AHV - vDISK - Part - 1 : where are they stored?

  • 31 October 2019
  • 7 replies
  • 3059 views

Userlevel 3
Badge +4

 

In some scenarios, we might have a need to find the disks associated with a virtual machine hosted on a Nutanix AHV Cluster. 

Nutanix offers a distributed storage fabric, which forms a large storage pool across “N” number of nodes. we then create containers for different types of workloads or we can easily use a single container as well.

Virtual Machines are created via Prism Management and all vDisks associated with the VMs are hosted on the distributed storage pool. 

On a Nutanix AOS Cluster, Acropolis service runs across the cluster to manage virtual machine operations and configurations. As both Data & MetaData is distributed across the cluster, we can use the “Acropolis” service on almost any CVM to retrieve virtual machine information running on any node.

In order to interact with the Acropolis service, Nutanix offers “acli”. a powerful set of commands for virtual machine operations across your Nutanix cluster.

acli offers tab-completion once inside the acli shell or we can execute the acli commands from a normal cvm prompt as well.

 

Find disk or disks attached to a existing virtual machine:

I need to find the Disk (100GB) attached to a VM called “WServer2019” on a Nutanix AOS - AHV cluster.

To query Virtual Machine configuration via acropolis cli <acli>, we will use the “vm” namespace, which offers us vm.get or vm.disk_get  to find disk information of a virtual machine.

 

By retrieving Virtual Machine Information : acli vm.get

SSH to a CVM on the same cluster. “nutanix@cvm-ip” 

From the CVM command shell:
nutanix@NTNX-CVM:192.168.100.1:~$ acli vm.get <vm-name>

Above acli vm.get command will give us all the relevant information related to a virtual machine or in simpler terms, it will output virtual machine config, (CPU | RAM | Disks | Controllers | Location)

As we are interested in finding the disks and their location on one of the containers on our cluster, we could do the following:

acli vm.get WServer2019 include_vmdisk_paths=1

Above command will output the Entire VM Configuration for WServer2019, out of which we will be interested in the ‘disk_list’ section.

Note, we have used the “include_vmdisk_paths=1” in order for the output to include the actual path where the vdisk resides. Following section “disk_list” will give us the desired information:

disk_list {
addr {
bus: "scsi"
index: 0
}
container_id: 3878917992
container_uuid: "fcba5c88-71be-43cf-81c7-d6140c13ad26"
device_uuid: "6f861a2f-334a-414a-a930-6515707e712f"
naa_id: "naa.6506b8d9bad6b961b1f8219274aaacc9"
vmdisk_nfs_path: "/ctr01/.acropolis/vmdisk/958a3c70-7d99-4706-bee5-35cb70339ce0"
vmdisk_size: 107374182400
vmdisk_uuid: "958a3c70-7d99-4706-bee5-35cb70339ce0"
}

Looking at the output above, we can see the “vmdisk_nfs_path” is pointing to a container called “ctr01” and the hidden directory called “.acropolis” which holds all vmdisks inside the vmdisks folder.

 

By retrieving VIRTUAL DISK Information for a given VM : acli vm.disk_get

We can also drop inside the “acli” prompt and get the desired information (as above) via the “vm.disk_get” command:

nutanix@NTNX-CVM:192.168.1.1:~$ acli
acli vm.disk_get <vm-name>

in the following snippet we are inside the acli shell / prompt:

<acropolis> vm.disk_get WServer2019 include_vmdisk_paths=true
ide.1 {
addr {
bus: "ide"
index: 1
}
cdrom: True
device_uuid: "361b37ef-0681-45c8-9540-34cda27e3c1e"
empty: True
naa_id: "naa.6506b8db86621d2f872ca4f1126227c4"
}
scsi.0 {
addr {
bus: "scsi"
index: 0
}
container_id: 3878917992
container_uuid: "fcba5c88-71be-43cf-81c7-d6140c13ad26"
device_uuid: "6f861a2f-334a-414a-a930-6515707e712f"
naa_id: "naa.6506b8d9bad6b961b1f8219274aaacc9"
vmdisk_nfs_path: "/ctr01/.acropolis/vmdisk/958a3c70-7d99-4706-bee5-35cb70339ce0"
vmdisk_size: 107374182400
vmdisk_uuid: "958a3c70-7d99-4706-bee5-35cb70339ce0"
}
<acropolis>

the “vmdisk_uuid” and the location is of our interest here, which we can later use to either clone the disk as an image or download this disk by connecting to our cluster via WinSCP - more on that later.

Things to remember:

  • acli can be executed directly from a CVM
  • acli commands can also be executed by entering the acli shell  / prompt
  • acli can be executed from any cvm on a AOS cluster
  • acli vm.get (to List Virtual Machine info)
  • acli vm.disk_get <VM-NAME> (to retrieve disk information of a vm)

Stay tuned for the next part - where we will find and access disk of an existing virtual machine via WinSCP in order to download it to our workstation. 


7 replies

Any news on part 2, using scp? I could really use this right now to copy the files to another cluster.

Userlevel 3
Badge +4

Any news on part 2, using scp? I could really use this right now to copy the files to another cluster.

https://next.nutanix.com/server-virtualization-27/ahv-vdisk-part-2-accessing-and-downloading-vdisks-33672

Hi Elliott - Please find the link above - let me know if you require further info.

 

Userlevel 2
Badge +5

Hi Mutahir,

 

thank you very much for your comprehensive article. I wonder if there on the other hand is an easy way to find out which VM a particular vdisk belongs to. For example that last vdisk that still is on a container that is scheduled for deletion.

 

Kind regards,

Peter

Userlevel 3
Badge +4

Hi @MMSW_DE Thank you for the comments.

I wonder if there on the other hand is an easy way to find out which VM a particular vdisk belongs to. For example that last vdisk that still is on a container that is scheduled for deletion.

 

For your point above, we have to consider a few things:

Normally, we can list all the disks attached to a VM via acli vm.get or acli vm.disk_get commands, this will list all the vDisks attached with a given VM.

we can also include the full container path for a vdisk via:

acli vm.get <vm-name> include_vmdisk_paths=1

this will reveal the VM details (cpu | ram | nic | vdisks) along with the container path for all vdisks attached to this VM. Following string will be found in the output:

vmdisk_nfs_path: "/container-name/.acropolis/vmdisk/5b20b3f6-f83c-4e75-851b-3dc606c70379"

 

Now, if a vDisk is mark for deletion, that would most certainly, be part of a replication or snapshot cycle via protection domain for DR purposes. this is handled automatically.

For virtual machine disks which are part of a protection domain, their SNAPSHOTS are stored under:

/container-name/.snapshot/

If you take a Manual Snapshot via Prism > VM > SNAPSHOT, those Snapshots are stored under:

/container-name/.acropolis/snapshot/<UUID>/vmdisk/

Here you will see the snapshots of each vDISK (these are the snapshots you took manually) You can manage them in Prism > VM > Table view > VM Snapshot > (restore | delete | clone)  

vDisks can be part of a VM config, or a VM snapshot, or a protection domain snapshot, or an image, or catalog item, or volume group. 

So - automatically taken snapshots for any vdisk are handled automatically by the respective cluster component and it doesn’t impacts your running VM. 

I hope this helps, please let me know if you want further clarification on this or have any further queries on it.


BR

Userlevel 2
Badge +5

Hi,

thank you for this comprehensive information. I was looking for the other way round: I have a given vdisk residing on a container I’d like to delete, I can can see the vdisk path in sftp:

/SomeSuperfluousContainer/.acropolis/vmdisk/806fdad9-8684-41ea-a49c-fc956193bcff

I’d rather not execute  acli vm.get for all VMs on the cluster to see if I can spot this path anywhere, Hence my question whether there is a way to determine which VM this vdisk belongs to.

See you,

Peter

Userlevel 3
Badge +4

I was looking for the other way round: I have a given vdisk residing on a container I’d like to delete, I can can see the vdisk path in sftp:

 /SomeSuperfluousContainer/.acropolis/vmdisk/806fdad9-8684-41ea-a49c-fc956193bcff

I’d rather not execute  acli vm.get for all VMs on the cluster to see if I can spot this path anywhere, Hence my question whether there is a way to determine which VM this vdisk belongs to.

See you,

Peter

Hi @MMSW_DE 

Thank you for your comments. 

I understand that you would like to find the respective VM for a given vDisk.

So, to clarify, the WinSCP method is only for uploading or downloading files (if required). it is not recommended to delete anything when connected via winscp on the container. this can cause problems with the acropolis service.

On AHV you can only list vm details to find the respective vm disks. (acli vm.get or acli vm.disk_get). 

BR

 

 

 

Userlevel 3
Badge +4

Hi,

thank you for this comprehensive information. I was looking for the other way round: I have a given vdisk residing on a container I’d like to delete, I can can see the vdisk path in sftp:

 /SomeSuperfluousContainer/.acropolis/vmdisk/806fdad9-8684-41ea-a49c-fc956193bcff

I’d rather not execute  acli vm.get for all VMs on the cluster to see if I can spot this path anywhere, Hence my question whether there is a way to determine which VM this vdisk belongs to.

See you,

Peter

 

Hi Peter,

this might help you identify the VM for a given “vdisk uuid”.

acli vm.get "*" include_vmdisk_paths=1 | grep -A 20 <vDISK UUID> | egrep " name|nfs_path"

Hope this helps

Reply