How It works

You’re probably aware all the CVMs and hypervisor hosts in a cluster need to have IPs in the same subnet, but what about the IPMI? What are the requirements, and what’s involved in changing the IP? The requirements are actually quite flexible. The IPMI does not have to be in the same subnet as the hosts and CVMs. You’ll see an alert and a cluster health warning if you don’t restart the genesis service on the CVMs after making changes, but the configuration can be whatever works best for your organization. To restart the genesis service, log into the CVM via SSH or console as the user ‘nutanix’ and run the command “genesis restart”. This restart of the genesis service is non-disruptive.  Actually, you can plug the IPMI into an isolated network or configure it on a different VLAN. You could even leave it unplugged when not in use. The IPMI is very useful for installs and updates, and for troubleshooting hardware issues or an unexpected reboot but it is not required for day to day operati

Hi, I want to get these numbers with API call. What is the best way to achieve this? Thanks

I'm trying to create a vm from our CentOS75 gold image. Pallavi Akella from Nutanix created the CentOS gold image using Disk and not an ISO. When I provision vms using powershell script. update the Clone.csv where I enter the following info name,VlanID,VMRam,Vcpu,cores,VMIP,clone,Image name,VlanID,8,2,2,VMIP,CentOS75, and save the file. Next I run cloneNTNX.ps1 script to provision the vm. This script also runs connect-Nutainx.ps1 and Create-NTNXVM.ps1 script. When I run the cloneNTNX.ps1 script I get below error. PS C:\Users\srananavre-a> cd C:\_Mis\NTNX\scripts PS C:\_Mis\NTNX\scripts> .\cloneNTNX.ps1 Not connected to any server. Please connect to a server and try again. Nutanix Cluster: Username for : Password for [Warning]: Cluster version[5.10.1.1] and Cmdlets version[5.5] do not match. This might cause some cmdlets to not functin correctly. Do you still want to continue [Y/N]?: y Connected to Checking if VM already exists... No data found. Creating na

I want to exclude some disks from my snapshot, using Rubrik on my Nutanix VM. I use the script from rubrik (which is a Patch) to exclude some disks from my nutanix vm like this: { "excludeDiskIds": [ "diskid1", "diskid2" ] } I get these disks id from the nutanix API (which is a get), but when I snapshot the machine from Rubrik the disks are still included. Am I doing something wrong? Is the "Nutanix Disk ID" equal to the one requested from the Rubrik script?  Thank you

What is the recommended maximum storage utilization in a cluster? Customers can observe cluster issues when they use more than 90 percent of the total available storage on the cluster. Here is a brief explanation on the recommended storage utilization in respect to the replication factors 2 & 3.  For a cluster to be considered healthy and functioning as expected, the cluster has to tolerate at least one node failure for data resiliency.  Here is an explanation on how much free space you need and how to calculate it.   The formula for calculating the maximum recommended usage for clusters is one of the following: Recommended maximum utilization of a cluster with containers using replication factor (RF)=2 M = 0.9 x (T - N1) Recommended maximum utilization of a cluster with containers using replication factor (RF)=3 M = 0.9 x (T - [N1 + N2]) M Recommended maximum usage of the cluster T Total available physical storage capacity in the cluster N1 Storage capacity of the node with t

Let’s say you want  high availability at the NIC level  and failover capacity to the CVM and user VMs. We recommend using  balance-slb to take advantage of all adapters. Balance-tcp is a load balancing method that increases host and VM bandwidth utilization beyond a single 10 Gb adapter by balancing each VM NIC TCP session on a different adapter. Also used when network switches require LACP negotiation."   Advantages of Bond Mode for balance-slb Balance-slb bond mode in OVS (Open vSwitch) takes advantage of all the links. Rebalance VM traffic from the interface that is used extensively to less used interfaces. When the configurable bond-rebalance interval expires, OVS uses the measured load for each interface and the load for each source MAC hash to spread traffic evenly among links in the bond.  Traffic from some source MAC hashes may move to a less active link to more evenly balanced bond member utilization.   Here is an article that describes the impact of the change and the s

On your Nutanix and vSphere cluster you may have recently seen this alert stating “Storage Containers are not mounted on all nodes”. This may have started showing up after some cluster modifications or maintenance, or could have appeared after some recent upgrades.  You’re probably wanting to know, what is this alert for? How big of a problem is it and what do we need to do to fix it? I’d like to answer those questions for you. This alert is generated through a recurring NCC check “Storage Container Mount Configuration”. First off it’s useful to know that the main purpose of this check is to validate the configuration of  a Metro Availability pair of Nutanix/ESXi clusters. That’s why this check is categorized under the data protection checks. The command to run this check from the CVM CLI is “ncc health_checks data_protection_checks protection_domain_checks storage_container_mount_check”. As for the impact of the problem being reported I’d have to say it’s conditional. The briefest way

Hi,Our metro cluster is running AOS5.10.8.1 (since yesterday’s update from AOS 5.10.8) on x8 NX-8035-G6. During an LCM update for bios and HDD, a node failed due to a bad DIMM. The node pair were eventually evicted from the meta data ring at which point support were supplying a replacement DIMM. After a lot of coaxing both nodes were back in the cluster and hosting vSphere6.5 without any apparent issue, however, since the outage we are being warned in PC\PE and NCC that the active and standby PDs are not mounted on all nodes, even after manually ensuring that the containers are available and connected correctly. Support have advised that this is a known NCC error and just wondered if anyone else in the community has experienced this issue? Thanks in advance

 There may come a time when you must expand your cluster storage capacity since you predict more data will be added to the cluster in the near future.Also, you are already short of space and cannot gain any more space by deleting the old snapshots and removing VM not needed any more, etc.In that situation, assuming you have empty slots in your hardware chassis and depending on the type of the drive you are adding you will need to follow the instruction in your nutanix portal document titled “Adding a Drive”.If you do not have any more disk slot on your hardware to, you may have to add a new node to your cluster altogether, so it can expand the capacity of your cluster. This has the benefit of adding memory and cpu resources as well.

Hi, I am trying to create a VM using the v3 API and have a question about creating the vdisks. I can specify the disk details (size, type, etc.) in the create VM call, but don't seem to be able to specify the storage container to use. Is this correct?  The only work around that I've found is to use the v1 API to create the vdisk, grab its UUID, and use that in the v3 call to create the VM. I can't see a call to create a vdisk in either v2 or v3. Darren.

First of all, let’s take a look at what all-flash and hybrid nodes mean. All-flash:- Nodes containing only SSD drives Hybrid nodes:- Nodes containing both HDD and SSD drives.   Is it possible to have both kinds of nodes within the same cluster? – Provided that the two conditions below are met, it is: The AOS version must be 5.1 or later. The minimum number of each node type (all-flash/hybrid) must be equal to the cluster redundancy factor. For example, clusters with a redundancy factor of 2 must have a minimum of 2 hybrid and 2 all-flash nodes.   To check for any issues with mixing these nodes, we have the NCC check, “all_flash_nodes_intermixed_check”.   This checks if there are any all-flash (only SSD drives) nodes in the same cluster that have hybrid (HDD and SSD drives) nodes. This check is scheduled to run every day, by default and will generate an alert after 1 failure.   To know more about the check and its behavior, take a look at https://portal.nutanix.com/#/page/kbs/detai

The Unified Extensible Firmware Interface (UEFI) is a specification that defines a software interface between an operating system and platform firmware. UEFI replaces the legacy Basic Input/Output System (BIOS) firmware interface originally present in all IBM PC-compatible personal computers,with most UEFI firmware implementations providing support for legacy BIOS services. UEFI can support remote diagnostics and repair of computers, even with no operating system installed.Advantages :The interface defined by the EFI specification includes data tables that contain platform information, and boot and runtime services that are available to the OS loader and OS. UEFI firmware provides several technical advantages over a traditional BIOS system: Ability to use large disks partitions (over 2 TB) with a GUID Partition Table (GPT) CPU-independent architecture CPU-independent drivers Flexible pre-OS environment, including network capability Modular design Backward and forward compatib

You might receive an error of low free block count in your NCC report due to failed check sata_dom_fwv_check.   This plugin checks the SATA DOM SMART data and shows if anything is off from the expected result.   To free up blocks in the SATA DOM, we have a procedure called, “Trim Operation” which is effective and really quick. The trim operation consists of running a script called trim_satadom.sh which can be downloaded from the link provided in the below mentioned doc.   To get to know more about the NCC check and to see how the Trim operation works, take a look at- https://portal.nutanix.com/#/page/kbs/details?targetId=kA032000000PODRCA4

Oops, did you by mistake created a VM on SATA DOM? What will happen if a VM is intentionally created on SATA DOM?   For starters, it is not recommended to run VMs on the SATA DOM, as this accelerates the degradation. If by mistake you did it, how will you know?   The NCC health check sata_dom_uvm_check verifies if there are any VMs running on the SATA DOM, and reports a FAIL status if the check detects any VMs running on the SATA DOM.   Now as the check suggests that there is a VM running on SATA DOM, the only solution would be to take it out of there.   To know more about the check outputs and the solution, take a look at https://portal.nutanix.com/#/page/kbs/details?targetId=kA03200000098jUCAQ   Have any questions? Drop a comment and let’s start a discussion

IPMI version 2.0 is susceptible to exploitation that allows an attacker to obtain password hash information.   The vulnerability scan on the environment can give the below sample output:-  Synopsis :The remote host supports IPMI version 2.0.Description :The remote host supports IPMI v2.0. The Intelligent Platform Management Interface (IPMI) protocol is affected by an information disclosure vulnerability due to the support of RMCP+ Authenticated Key Exchange Protocol (RAKP) authentication. A remote attacker can obtain password hash information for valid user accounts via the HMAC from a RAKP message 2 response from a BMC.   To know the IPMI version on the host, login into the host, run the command for checking BMC version and you’ll get an output similar to:-  Device ID                 : 32Device Revision           : 1Firmware Revision         : 3.63IPMI Version              : 2.0 <<== IPMI versionManufacturer ID           : xxxxxManufacturer Name         : Supermicro   To know ho

Is it possible to share a disk between two or more Virtual Machines? Can we use the disk on the two VMs at the same time, for reading and writing?   The answer is Yes and No respectively.   Yes, Nutanix supports shared disk via Volume Group on AHV. This is supported on clustering solutions (Cluster at guest OS level) like My SQL etc.   You can use the disk on the two VMs at the same time for reading. However, two VMs writing at the same time is not allowed.    In a clustering solution, the master VM will write to the disk and all the other VMs in the cluster can read from the disk.    Take a look at the guide to have an overview of how storage management is done at Nutanix:- https://portal.nutanix.com/#/page/docs/details?targetId=Web-Console-Guide-Prism-v5_16:wc-storage-management-wc-c.html

Let’s say you want to add a new node with a newer processor class than the existing nodes in the cluster. In this case you must enable EVC (Enhanced vMotion Compatibility) feature.   What is EVC?   EVC stands for Enhanced vMotion Compatibility which is a vCenter Server cluster-centric feature allowing virtual machines to vMotion or migrate across ESXi hosts equipped with dissimilar processors in the same cluster. VMware EVC Mode works by masking unsupported processor features thus presenting a homogeneous processor front to all the virtual machines in a cluster. This means that a VM can vMotion to any ESXi host in a cluster irrespective of the host’s micro-architecture examples of which include Intel’s Sandy Bridge and Haswell. One caveat to remember is that all the processor(s) must be from a single vendor i.e. either Intel or AMD. You simply cannot mix and match.   What are the benefits?   The main benefit is that you can add servers with the latest processors to your existing cluste

Are you wondering which alerts generate a support case with Nutanix Support and whom Nutanix contacts when a case is generated? There is a particular subset of alerts that have been identified as being important, so action can be taken on them. These alerts are called Alert cases. When the cluster is under a current support contract, alert cases are generated. Alert cases are handled by Nutanix Support, who reach out to the primary Support Contact (also referred to as the break/fix contact) associated with the node or cluster in question. For customers with a third-party providing front-line support, it is normal to have a representative from that company listed as the primary Support Contact. Most of the alert cases are created with a P4 priority level, which has a target response time of 24 hours. NodeDegraded and NodeAvgLoadAboveThreshold are created with a P2 priority level, which has a target response time of two hours. To know which alerts generate cases, and how to change the co

On a Nutanix cluster running VMware vSphere, you may be seeing this alert while there is no other sign of issue: Warning : Some storage containers have a high number of NFS files Impact          : High number of NFS files may cause vpxa services on esxi hosts to restart. Cause           : Number of files for respective storage containers has increased beyond 20K. This is expected with large VDI setups. Resolution      : Reduce the number of files if you observe vpxa instability. Why do we see this alert? What needs to be done about it? As is noted on the relevant KB article this alert is set to be deprecated in an upcoming release of NCC. As suggested by the Impact and Resolution information above, the alert is related to a problem encountered in vpxa. The alert was created for an issue in vSphere 5.x that caused instability for the vpxa service if the number of files in a single NFS datastore was too high. The issue was resolved by VMware some time ago so you do not need to worry abou

Below are the top knowledge base articles for the month of February 2020 KB 4116 - Alert - A1187, A1188 - ECCErrorsLast1Day, ECCErrorsLast10Days KB 4141 - Alert - A1046 - PowerSupplyDown KB 7503 - G6, G7 platforms with BIOS 41.002 and higher - DIMM Error handling and replacement policy KB 1540 - What to do when /home partition or /home/nutanix directory is full KB 1113 - HDD/SSD Troubleshooting KB 4158 - Alert - A1104 - PhysicalDiskBad KB 4409 - LCM: (LifeCycle Manager) Troubleshooting Guide KB 4519 - NCC Health Check: check_ntp KB 4188 - Alert - A1050, A1008 - IPMIError KB 2090 - AHV | Host and Guest Networking KB 4541 - Alert - A101055 - MetadataDiskMountedCheck KB 1507 - Alert IPMI IP address on Controller VM {CVM-IP} was updated to ... without following the Nutanix IP Reconfiguration procedure, can be misleading KB 4273 - NCC Health Check: aged_third_party_backup_snapshot_check KB 8792 - NCC checks: same_hypervisor_version_check, duplicate_cvm_ip_check, same_timezone_check, esx_sio

Below are new knowledge base articles published on the week of February 23-29, 2020. KB 8900 - Prism Central Alert Manager and Cassandra Monitor services crashing, NCC health checks failing for same. KB 8909 - [Karbon] Airgap (dark site) FAQ and Troubleshooting Guide KB 8956 - AHV | How to get a list of virtual machines that were restarted due to an HA event KB 8976 - Nutanix Files Share might encounter mapping issue to macOS KB 8980 - VMs deleted via vCenter may fail to restore within 7 days of deletion KB 8988 - Calm - Calm UI is not accessible on PC 5.16 KB 8995 - Storage utilization on a Hyper-V cluster steadily increasing due to failed Hyper-V Replica replications KB 9005 - Nessus Plugin ID 33851 - Network daemons not managed by the package system KB 9029 - Changing LDAP port 389 authentication to Secure LDAP (LDAPS) ports 636 or 3269 Note: You may need to log in to the Support Portal to view some of these articles.

 Is there a way to create isolated networks within the nutanix cluster that would allow communication only for the VMs that has a NIC connected to that network? Find attached an example of what I mean if we were talking about ESXi Thanks in advance!

Can anyone let me know which steps are being involved when I expand Hyper-V cluster by using Prism? I have a Hyper-V cluster that being mixed by some AHV storage nodes and looks like I cannot use the Prism cluster expansion feature to add more nodes...

Dear All Is it mandatory to replace the node in a cluster with the same hardware? Can we replace it with a higher configuration(CPU)

If you have a Nutanix cluster on VMware vSphere which has been through some hypervisor upgrades, your CVM’s virtual hardware version may be a bit out of date. The main requirement from the Nutanix perspective is that CVM virtual hardware versions should be the same across the AOS cluster, and this is checked by NCC. Still, since newer virtual hardware versions include security updates and sometimes feature updates or improvements from VMware you may want to bring this up to date. If you have an AOS upgrade pending, completing that upgrade would be the easiest route to solution. If you don’t have any AOS upgrade needed, please note Nutanix does not recommend manually updating the VM hardware version for CVMs. The recommended method is to allow virtual hardware version to update during an AOS upgrade. Do be aware, if you manually updated the virtual hardware version to 13 or higher on ESXi 6.5 before Patch 4, or 6.7 before Update 3, this could cause the CVM to fail to boot. There is a kn