Scripts

Welcome to the Nutanix NEXT community. To get started please read our short welcome post. Thanks!

cancel
Showing results for 
Search instead for 
Did you mean: 

Programatically retrieve VM Table information

SOLVED Go to solution
Explorer

Programatically retrieve VM Table information

I'm trying to figure out a way via PowerShell, or even the main Prism interface to export all of the table information from the VM > Table space within Prism.  Like this.

 

2017-10-09 08_50_49-nutanix-esxi - Prism Element.png

 

Doing an entire .CSV export of this would be just fine, but it appears to limit to 1000 rows.  I would expect something along the lines of grabbing all the VM's with Get-NTNXVM and piping that into Get-NTNXVMStat, but this cmdlet appears to need several metrics specified that I just can't dig into properly.  I receive 500 errors any time I try to retrieve information, and there's no Get-Help / example entries for the cmdlet.

 

Is there an easy way I'm overlooking to retrieve exactly that formatted table from prism?  If not, any canned PS scripts that can do it?  The individual I/O statistics are very helpful here, as a whole this is an excellent, granular snapshot of the environment.

 

Thanks!

1 ACCEPTED SOLUTION

Accepted Solutions
Nutanix Employee

Re: Programatically retrieve VM Table information

If you are looking for a powershell script you can use the following 

 

Connect-NTNXCluster -Server <ipaddress> -UserName admin  -AcceptInvalidSSLCerts
$static_stats=@("vmname","numVCpus","memoryCapacityInBytes","diskCapacityInBytes")
$dyn_stats=@("hypervisor_cpu_usage_ppm", "hypervisor_memory_usage_ppm","controller_avg_io_latency_usecs", `
"controller_num_read_iops", "controller_num_write_iops", "controller_io_bandwidth_kBps")
$vm_col=@()

Get-NtnxVM | %{
    $vmstat=New-Object System.Management.Automation.PSObject 
    $static_stats | %{Add-Member -InputObject $vmstat -MemberType NoteProperty -Name $_ -Value ""}
    $dyn_stats | %{Add-Member -InputObject $vmstat -MemberType NoteProperty -Name $_ -Value ""}
    $vmstat.("vmname")=$_.vmname
    $vmstat.("numvcpus")=$_.numvcpus
    $vmstat.("memorycapacityinbytes")=$_.memorycapacityinbytes
    $vmstat.("DiskCapacityInbytes")=$_.diskCapacityInBytes
    Get-NtnxVMstat -VmId $_.vmid -Metrics $($dyn_stats -join ',') | %{
        $vmstat.($_.metric)=$_.values[0]    }
    $vm_col+=$vmstat
}
$vm_col | select vmname,numvcpus,@{Name="Memory (GB)";Expression={$_.memoryCapacityInBytes/1GB}}, `
@{Name="Disk Size (GB)";Expression={[math]::Round($_.diskCapacityInBytes/1GB)}},@{Name="CPU Usg %";Expression={$_.hypervisor_cpu_usage_ppm/10000}},`
@{Name="Memory Usg %";Expression={$_.hypervisor_memory_usage_ppm/10000}},@{Name="read iops";Expression={$_.controller_num_read_iops}},`
@{Name="Write IOPS";Expression={$_.controller_num_write_iops}},@{Name="Bandwidth (kbps)";Expression={$_.controller_io_bandwidth_kBps}},`
@{Name="Avg Latency (ms)";Expression={$_.controller_avg_io_latency_usecs/1000}} | Export-csv -Path result.csv -NoTypeInformation

Please replace the <ipaddress> with CVM ip address.  This script will output a csv file "result.csv" with the required stats

10 REPLIES
Trailblazer

Re: Programatically retrieve VM Table information

 

I cobbled together a python script to list all the VM's and convert the json output to csv

 

It may help...

 

edit; it removed all the indentation Smiley Sad 

 

#!/usr/bin/env python

import csv
import pprint
import json
import os
import random
import sys
import traceback
import getpass
import time
import requests
from requests.packages.urllib3.exceptions import InsecureRequestWarning
requests.packages.urllib3.disable_warnings(InsecureRequestWarning)


t = (time.strftime("%H%M%S%d%m%Y"))
clus = raw_input("Enter Cluster Name: ")
user = raw_input("Enter your Prism username: ")
p = getpass.getpass("Enter you Prism Password: ")
rv = 'v2.0'

 

#This block initializes the parameters for the request.
class TestRestApi():
def __init__(self):

#Initializes the options and the logfile from GFLAGS.
self.serverIpAddress = clus
self.username = user
self.password = p
# Base URL at which REST services are hosted in Prism Gateway.
BASE_URL = 'https://%s:9440/PrismGateway/services/rest/'+rv
self.base_url = BASE_URL % self.serverIpAddress
self.session = self.get_server_session(self.username, self.password)

def get_server_session(self, username, password):

#Creating REST client session for server connection, after globally setting
#Authorization, content type, and character set for the session.
session = requests.Session()
session.auth = (username, password)
session.verify = False
session.headers.update(
{'Content-Type': 'application/json; charset=utf-8'})
return session

 

def getVMs(self):

vmsVMURL = self.base_url + "/vms/?include_vm_disk_config=false&include_vm_nic_config=false"
print "List of VMs in cluster %s" % self.serverIpAddress
serverResponse = self.session.get(vmsVMURL)
print "Response code: %s" % serverResponse.status_code
return json.loads(serverResponse.text)

 

def to_string(s):
try:
return str(s)
except:
#Change the encoding type if needed
return s.encode('utf-8')

 

def reduce_item(key, value):
global reduced_item

#Reduction Condition 1
if type(value) is list:
i=0
for sub_item in value:
reduce_item(key+'_'+to_string(i), sub_item)
i=i+1

#Reduction Condition 2
elif type(value) is dict:
sub_keys = value.keys()
for sub_key in sub_keys:
reduce_item(key+'_'+to_string(sub_key), value[sub_key])

#Base Condition
else:
reduced_item[to_string(key)] = to_string(value)


if __name__ == "__main__":

csv_file_path = t+'_'+clus+'_vms.csv'

try:
testRestApi = TestRestApi()
vms = testRestApi.getVMs()
raw_data = vms
data_to_be_processed = vms['entities']
except Exception as ex:
print ex
sys.exit(1)
data_to_be_processed = vms

processed_data = []
header = []
for item in data_to_be_processed:
reduced_item = {}
reduce_item('entities', item)

header += reduced_item.keys()

processed_data.append(reduced_item)

header = list(set(header))
header.sort()

with open(csv_file_path, 'wb+') as f:
writer = csv.DictWriter(f, header, quoting=csv.QUOTE_ALL)
writer.writeheader()
for row in processed_data:
writer.writerow(row)

print "Just completed writing csv file with %d columns" % len(header)

Trailblazer

Re: Programatically retrieve VM Table information

#!/usr/bin/env python

import csv
import pprint
import json
import os
import random
import sys
import traceback
import getpass
import time
import requests
from requests.packages.urllib3.exceptions import InsecureRequestWarning
requests.packages.urllib3.disable_warnings(InsecureRequestWarning)


t = (time.strftime("%H%M%S%d%m%Y"))
clus = raw_input("Enter Cluster Name: ")
user = raw_input("Enter your Prism username: ")
p = getpass.getpass("Enter you Prism Password: ")
rv = 'v2.0'



#This block initializes the parameters for the request.
class TestRestApi():
  def __init__(self):

    #Initializes the options and the logfile from GFLAGS.
    self.serverIpAddress = clus
    self.username = user
    self.password = p
    # Base URL at which REST services are hosted in Prism Gateway.
    BASE_URL = 'https://%s:9440/PrismGateway/services/rest/'+rv
    self.base_url = BASE_URL % self.serverIpAddress
    self.session = self.get_server_session(self.username, self.password)

  def get_server_session(self, username, password):

    #Creating REST client session for server connection, after globally setting
    #Authorization, content type, and character set for the session.
    session = requests.Session()
    session.auth = (username, password)
    session.verify = False
    session.headers.update(
        {'Content-Type': 'application/json; charset=utf-8'})
    return session



  def getVMs(self):

    vmsVMURL = self.base_url + "/vms/?include_vm_disk_config=false&include_vm_nic_config=false"
    print "List of VMs in cluster %s" % self.serverIpAddress
    serverResponse = self.session.get(vmsVMURL)
    print "Response code: %s" % serverResponse.status_code
    return json.loads(serverResponse.text)



def to_string(s):
    try:
        return str(s)
    except:
        #Change the encoding type if needed
        return s.encode('utf-8')



def reduce_item(key, value):
    global reduced_item

    #Reduction Condition 1
    if type(value) is list:
        i=0
        for sub_item in value:
            reduce_item(key+'_'+to_string(i), sub_item)
            i=i+1

    #Reduction Condition 2
    elif type(value) is dict:
        sub_keys = value.keys()
        for sub_key in sub_keys:
            reduce_item(key+'_'+to_string(sub_key), value[sub_key])

    #Base Condition
    else:
        reduced_item[to_string(key)] = to_string(value)


if __name__ == "__main__":

        csv_file_path = t+'_'+clus+'_vms.csv'

        try:
            testRestApi = TestRestApi()
            vms = testRestApi.getVMs()
            raw_data = vms
            data_to_be_processed = vms['entities']
        except Exception as ex:
            print ex
            sys.exit(1)
            data_to_be_processed = vms

        processed_data = []
        header = []
        for item in data_to_be_processed:
            reduced_item = {}
            reduce_item('entities', item)

            header += reduced_item.keys()

            processed_data.append(reduced_item)

        header = list(set(header))
        header.sort()

        with open(csv_file_path, 'wb+') as f:
            writer = csv.DictWriter(f, header, quoting=csv.QUOTE_ALL)
            writer.writeheader()
            for row in processed_data:
                writer.writerow(row)

        print "Just completed writing csv file with %d columns" % len(header)

I cobbled together this python script, it may help

 

Needs changing to get IO info though

Nutanix Employee

Re: Programatically retrieve VM Table information

If you are looking for a powershell script you can use the following 

 

Connect-NTNXCluster -Server <ipaddress> -UserName admin  -AcceptInvalidSSLCerts
$static_stats=@("vmname","numVCpus","memoryCapacityInBytes","diskCapacityInBytes")
$dyn_stats=@("hypervisor_cpu_usage_ppm", "hypervisor_memory_usage_ppm","controller_avg_io_latency_usecs", `
"controller_num_read_iops", "controller_num_write_iops", "controller_io_bandwidth_kBps")
$vm_col=@()

Get-NtnxVM | %{
    $vmstat=New-Object System.Management.Automation.PSObject 
    $static_stats | %{Add-Member -InputObject $vmstat -MemberType NoteProperty -Name $_ -Value ""}
    $dyn_stats | %{Add-Member -InputObject $vmstat -MemberType NoteProperty -Name $_ -Value ""}
    $vmstat.("vmname")=$_.vmname
    $vmstat.("numvcpus")=$_.numvcpus
    $vmstat.("memorycapacityinbytes")=$_.memorycapacityinbytes
    $vmstat.("DiskCapacityInbytes")=$_.diskCapacityInBytes
    Get-NtnxVMstat -VmId $_.vmid -Metrics $($dyn_stats -join ',') | %{
        $vmstat.($_.metric)=$_.values[0]    }
    $vm_col+=$vmstat
}
$vm_col | select vmname,numvcpus,@{Name="Memory (GB)";Expression={$_.memoryCapacityInBytes/1GB}}, `
@{Name="Disk Size (GB)";Expression={[math]::Round($_.diskCapacityInBytes/1GB)}},@{Name="CPU Usg %";Expression={$_.hypervisor_cpu_usage_ppm/10000}},`
@{Name="Memory Usg %";Expression={$_.hypervisor_memory_usage_ppm/10000}},@{Name="read iops";Expression={$_.controller_num_read_iops}},`
@{Name="Write IOPS";Expression={$_.controller_num_write_iops}},@{Name="Bandwidth (kbps)";Expression={$_.controller_io_bandwidth_kBps}},`
@{Name="Avg Latency (ms)";Expression={$_.controller_avg_io_latency_usecs/1000}} | Export-csv -Path result.csv -NoTypeInformation

Please replace the <ipaddress> with CVM ip address.  This script will output a csv file "result.csv" with the required stats

Explorer

Re: Programatically retrieve VM Table information

Thank you!  Pretty sure this is what I need, I appreciate it.  I'm currently trying to find a way to filter it down right now.  This cluster has about 2500 VM's in it, with a 4.5:1 CPU ratio and heavily utilization, I think this is causing the Get-NTNXVM cmdlet to time out.  I believe I can accomplish this by importing a .csv with chunks of names of VM's, and then combining the .CSV files later.

 

Thanks again : )

 

 

 

 

Nutanix Employee

Re: Programatically retrieve VM Table information

@mhutchison89  Since you have a large number of vm's i have changed the code to use Object collections instead of normal arrays which will be more memory efficient. The changes are in only two lines, please use the below script,

 

Connect-NTNXCluster -Server 10.63.4.113 -UserName admin  -AcceptInvalidSSLCerts
$static_stats=@("vmname","numVCpus","memoryCapacityInBytes","diskCapacityInBytes")
$dyn_stats=@("hypervisor_cpu_usage_ppm", "hypervisor_memory_usage_ppm","controller_avg_io_latency_usecs", `
"controller_num_read_iops", "controller_num_write_iops", "controller_io_bandwidth_kBps")
$vm_col= New-Object System.Collections.Arraylist

Get-NtnxVM | %{
    $vmstat=New-Object System.Management.Automation.PSObject 
    $static_stats | %{Add-Member -InputObject $vmstat -MemberType NoteProperty -Name $_ -Value ""}
    $dyn_stats | %{Add-Member -InputObject $vmstat -MemberType NoteProperty -Name $_ -Value ""}
    $vmstat.("vmname")=$_.vmname
    $vmstat.("numvcpus")=$_.numvcpus
    $vmstat.("memorycapacityinbytes")=$_.memorycapacityinbytes
    $vmstat.("DiskCapacityInbytes")=$_.diskCapacityInBytes
    Get-NtnxVMstat -VmId $_.vmid -Metrics $($dyn_stats -join ',') | %{
        $vmstat.($_.metric)=$_.values[0]    }
    $vm_col.add($vmstat) | Out-Null
}
$vm_col | select vmname,numvcpus,@{Name="Memory (GB)";Expression={$_.memoryCapacityInBytes/1GB}}, `
@{Name="Disk Size (GB)";Expression={[math]::Round($_.diskCapacityInBytes/1GB)}},@{Name="CPU Usg %";Expression={$_.hypervisor_cpu_usage_ppm/10000}},`
@{Name="Memory Usg %";Expression={$_.hypervisor_memory_usage_ppm/10000}},@{Name="read iops";Expression={$_.controller_num_read_iops}},`
@{Name="Write IOPS";Expression={$_.controller_num_write_iops}},@{Name="Bandwidth (kbps)";Expression={$_.controller_io_bandwidth_kBps}},`
@{Name="Avg Latency (ms)";Expression={$_.controller_avg_io_latency_usecs/1000}} | Export-csv -Path test.csv -NoTypeInformation
Highlighted
Explorer

Re: Programatically retrieve VM Table information

Thanks Chandru, I have a question..

 

The updated PS code using objects works great if I specify a single VM, like this, where I'm using -searchString to specify a single VM.  The .CSV will export perfectly for that VM.  As soon as I remove that search criteria the GET-NTNXVM cmdlet will time out within about 15-30 seconds.

 
 
Get-NtnxVM -SearchString 'vmname' | forEach{
    $vmstat=New-Object System.Management.Automation.PSObject 
    $static_stats | %{Add-Member -InputObject $vmstat -MemberType NoteProperty -Name $_ -Value ""}
    $dyn_stats | %{Add-Member -InputObject $vmstat -MemberType NoteProperty -Name $_ -Value ""}
    $vmstat.("vmname")=$_.vmname

 

 

I have tried substituting a variable with about 25 VM names in it, in the form of..

 

Get-NTNXVM -SearchString $vm.Name | % (etc...), but it appears to only retrieve the first name in the array, I assume because -searchString limits it to one item.  

 

Is there another way to specify a smaller collection of VM's?  I imagine querying the entire inventory is taxing that cmdlet.  We do have a lot of VM's in a single cluster.  

 

 

Thanks!  I'm thinking I may need to open a case to figure out why code is timing out though?

 

 

Nutanix Employee

Re: Programatically retrieve VM Table information

@mhutchison89 Does it work when you use regular arrays instead of the object colletions? You can try to limit the count of vm returned by Get-NTNXvm using the -Count parameter and check if it is getting output.

 

 

I tested it on a 120 vm test cluster didn't receive any timeouts , you can limit to the lesser number and see if it works.

 

Explorer

Re: Programatically retrieve VM Table information

Yep, the script runs if I limit the count to < 300.  If I could feed the script a list of VM names I could probably get around that limitation.  I've tried Get-NTNXVM -count 300 | Where {$_.VMName -like $arrayofnames} but the command times out with that as well, I assume because it's having to crawl the entire inventory again.

Nutanix Employee

Re: Programatically retrieve VM Table information

@mhutchison89 what happens when you just run Get-NTNXVM | select vmname does it timeout too? If that times out too then we may need to use rest api to query the vms and get a list of them.