Disk structure of a CVM
The CVM is the core of the AOS clusters, but have ever wondered how disk layout exists within them?
I provide a brief overview of its structure in the following post.
The main thing that most of you are familiar with already, is that the LSI controller is passed directly to the CVM in order for it to have direct access to the disks in the node. When combining all the disks space from all the CVMs within the cluster it provides the unified software define storage pool used to store your data.
That’s why when you run “df -h” in the CVMs you will see the data disks mounted in the following mount point:
/home/nutanix/data/stargate-storage/disks/<disk serial_number>
However, the CVM also needs to have space for itself in order work, so a portion of SSDs and HHD is reserved for that purpose.
In single SSDs system you will get the following partition layout:
-
sda1: ‘/ ‘ root partition
-
sda2: standby root partition used for upgrades
-
sda3: /home partition
-
Sda4: usable SSD partition for user data, this one however also gets subdivided for the space required by Cassandra, Oplog and the content cache and the remaining space will be for the extent store
In a dual SSD system, software raid 1 is implemented for data resiliency in case there are SSD failures, so the layout would look something like this:
The MD partitions would be made of two SSDs (sda and sdb)
-
md0: ‘/ ‘ root partition
-
md1: standby root partition used for upgrades
-
md2: /home partition
-
sda4/sdb4: usable SSD partition for user data, this one however also gets subdivided for the space required by Cassandra, Oplog and the content cache and the remaining space will be for the extent store
Due to the way AOS upgrades work, it is normal for the ‘/’ root partition order to change, sometimes it will be sda1/md0 or sda2/md1
The rest of the disks like HHDs will be purely used for the extent store (persistent storage) with only one partition created for them but with a small portion of the disk reserved for Curator’s Mapreduce Filesystem.
Additionally to the disk structure from the data drives, the CVM will boot from the iso located in the hypervisor’s boot device.
To know more about the structure of the Drives and how space is calculated for each one of the components that occupy space from the disks you can visit the Nutanix Bible in the “Drive Breakdown” section.