How It works

Hi, I want to get these numbers with API call. What is the best way to achieve this? Thanks

On your Nutanix and vSphere cluster you may have recently seen this alert stating “Storage Containers are not mounted on all nodes”. This may have started showing up after some cluster modifications or maintenance, or could have appeared after some recent upgrades.  You’re probably wanting to know, what is this alert for? How big of a problem is it and what do we need to do to fix it? I’d like to answer those questions for you. This alert is generated through a recurring NCC check “Storage Container Mount Configuration”. First off it’s useful to know that the main purpose of this check is to validate the configuration of  a Metro Availability pair of Nutanix/ESXi clusters. That’s why this check is categorized under the data protection checks. The command to run this check from the CVM CLI is “ncc health_checks data_protection_checks protection_domain_checks storage_container_mount_check”. As for the impact of the problem being reported I’d have to say it’s conditional. The briefest way

Let’s say you want  high availability at the NIC level  and failover capacity to the CVM and user VMs. We recommend using  balance-slb to take advantage of all adapters. Balance-tcp is a load balancing method that increases host and VM bandwidth utilization beyond a single 10 Gb adapter by balancing each VM NIC TCP session on a different adapter. Also used when network switches require LACP negotiation."   Advantages of Bond Mode for balance-slb Balance-slb bond mode in OVS (Open vSwitch) takes advantage of all the links. Rebalance VM traffic from the interface that is used extensively to less used interfaces. When the configurable bond-rebalance interval expires, OVS uses the measured load for each interface and the load for each source MAC hash to spread traffic evenly among links in the bond.  Traffic from some source MAC hashes may move to a less active link to more evenly balanced bond member utilization.   Here is an article that describes the impact of the change and the s

First of all, let’s take a look at what all-flash and hybrid nodes mean. All-flash:- Nodes containing only SSD drives Hybrid nodes:- Nodes containing both HDD and SSD drives.   Is it possible to have both kinds of nodes within the same cluster? – Provided that the two conditions below are met, it is: The AOS version must be 5.1 or later. The minimum number of each node type (all-flash/hybrid) must be equal to the cluster redundancy factor. For example, clusters with a redundancy factor of 2 must have a minimum of 2 hybrid and 2 all-flash nodes.   To check for any issues with mixing these nodes, we have the NCC check, “all_flash_nodes_intermixed_check”.   This checks if there are any all-flash (only SSD drives) nodes in the same cluster that have hybrid (HDD and SSD drives) nodes. This check is scheduled to run every day, by default and will generate an alert after 1 failure.   To know more about the check and its behavior, take a look at https://portal.nutanix.com/#/page/kbs/detai

Can I use USB Dongle in  a Nutanix VM?

When you on-ramp new team members they’re going to need access to all their tools. If those new teammates will be managing Nutanix systems they most likely need a login to the support portal for access to documentation, downloads, the knowledge base, and the ability to open a support case.   So how do we set up access for new users? There are a few simple steps covered in the article How to Gain User Access to Nutanix Support Portal .   You’ll need a block or software-only serial number and the user’s email address. The email address should be in the authorized domain for your customer account.   To find the block serial number log into Prism and navigate to Hardware > Diagram. The block serial number is displayed above each block in the diagram. If there are multiple blocks, pick one and note the serial number for later. For software-only licenses, The software asset serial number is mentioned in the acknowledgement email sent upon the fulfilment of your order.   Once you have the

The Unified Extensible Firmware Interface (UEFI) is a specification that defines a software interface between an operating system and platform firmware. UEFI replaces the legacy Basic Input/Output System (BIOS) firmware interface originally present in all IBM PC-compatible personal computers,with most UEFI firmware implementations providing support for legacy BIOS services. UEFI can support remote diagnostics and repair of computers, even with no operating system installed.Advantages :The interface defined by the EFI specification includes data tables that contain platform information, and boot and runtime services that are available to the OS loader and OS. UEFI firmware provides several technical advantages over a traditional BIOS system: Ability to use large disks partitions (over 2 TB) with a GUID Partition Table (GPT) CPU-independent architecture CPU-independent drivers Flexible pre-OS environment, including network capability Modular design Backward and forward compatib

You might receive an error of low free block count in your NCC report due to failed check sata_dom_fwv_check.   This plugin checks the SATA DOM SMART data and shows if anything is off from the expected result.   To free up blocks in the SATA DOM, we have a procedure called, “Trim Operation” which is effective and really quick. The trim operation consists of running a script called trim_satadom.sh which can be downloaded from the link provided in the below mentioned doc.   To get to know more about the NCC check and to see how the Trim operation works, take a look at- https://portal.nutanix.com/#/page/kbs/details?targetId=kA032000000PODRCA4

Giving thought to change your replication factor from 2 to 3? What are the impacts and things to consider?   First, let’s take a look at what replication factor is. Redundancy factor is a configurable option that allows a Nutanix cluster to withstand the failure of nodes or drives in different blocks.   By default, Nutanix clusters have redundancy factor 2, which means they can tolerate the failure of a single node or drive.  So RF3 means cluster can tolerate the failure of 2 nodes or drive… Basic Maths isn’t it?   Redundancy factor 3 has the following requirements: Redundancy factor 3 can be enabled at the time of cluster creation or after creation too. A cluster must have at least five nodes for redundancy factor 3 to be enabled. For guest VMs to tolerate the simultaneous failure of two nodes or drives in different blocks, the data must be stored on containers with replication factor 3. Controller VMs must be configured with a minimum of 28 GB(20 GB default+8 GB for the featur

Oops, did you by mistake created a VM on SATA DOM? What will happen if a VM is intentionally created on SATA DOM?   For starters, it is not recommended to run VMs on the SATA DOM, as this accelerates the degradation. If by mistake you did it, how will you know?   The NCC health check sata_dom_uvm_check verifies if there are any VMs running on the SATA DOM, and reports a FAIL status if the check detects any VMs running on the SATA DOM.   Now as the check suggests that there is a VM running on SATA DOM, the only solution would be to take it out of there.   To know more about the check outputs and the solution, take a look at https://portal.nutanix.com/#/page/kbs/details?targetId=kA03200000098jUCAQ   Have any questions? Drop a comment and let’s start a discussion

IPMI version 2.0 is susceptible to exploitation that allows an attacker to obtain password hash information.   The vulnerability scan on the environment can give the below sample output:-  Synopsis :The remote host supports IPMI version 2.0.Description :The remote host supports IPMI v2.0. The Intelligent Platform Management Interface (IPMI) protocol is affected by an information disclosure vulnerability due to the support of RMCP+ Authenticated Key Exchange Protocol (RAKP) authentication. A remote attacker can obtain password hash information for valid user accounts via the HMAC from a RAKP message 2 response from a BMC.   To know the IPMI version on the host, login into the host, run the command for checking BMC version and you’ll get an output similar to:-  Device ID                 : 32Device Revision           : 1Firmware Revision         : 3.63IPMI Version              : 2.0 <<== IPMI versionManufacturer ID           : xxxxxManufacturer Name         : Supermicro   To know ho

Is it possible to share a disk between two or more Virtual Machines? Can we use the disk on the two VMs at the same time, for reading and writing?   The answer is Yes and No respectively.   Yes, Nutanix supports shared disk via Volume Group on AHV. This is supported on clustering solutions (Cluster at guest OS level) like My SQL etc.   You can use the disk on the two VMs at the same time for reading. However, two VMs writing at the same time is not allowed.    In a clustering solution, the master VM will write to the disk and all the other VMs in the cluster can read from the disk.    Take a look at the guide to have an overview of how storage management is done at Nutanix:- https://portal.nutanix.com/#/page/docs/details?targetId=Web-Console-Guide-Prism-v5_16:wc-storage-management-wc-c.html

Let’s say you want to add a new node with a newer processor class than the existing nodes in the cluster. In this case you must enable EVC (Enhanced vMotion Compatibility) feature.   What is EVC?   EVC stands for Enhanced vMotion Compatibility which is a vCenter Server cluster-centric feature allowing virtual machines to vMotion or migrate across ESXi hosts equipped with dissimilar processors in the same cluster. VMware EVC Mode works by masking unsupported processor features thus presenting a homogeneous processor front to all the virtual machines in a cluster. This means that a VM can vMotion to any ESXi host in a cluster irrespective of the host’s micro-architecture examples of which include Intel’s Sandy Bridge and Haswell. One caveat to remember is that all the processor(s) must be from a single vendor i.e. either Intel or AMD. You simply cannot mix and match.   What are the benefits?   The main benefit is that you can add servers with the latest processors to your existing cluste

Hi, I am trying to create a VM using the v3 API and have a question about creating the vdisks. I can specify the disk details (size, type, etc.) in the create VM call, but don't seem to be able to specify the storage container to use. Is this correct?  The only work around that I've found is to use the v1 API to create the vdisk, grab its UUID, and use that in the v3 call to create the VM. I can't see a call to create a vdisk in either v2 or v3. Darren.

Hello, does anybody have experiences with a container OS like RancherOS on AHV? In the Compatibility Matrix of Nutanix I can see that RancherOS v1.5.5 is compatible with my AOS 5.10, but I don’t know which image of RancherOS I should use. The are some images for a specific cloud provider and some for a specific hypervisor, but no one for Nutanix, AHV or KVM.   I tought then I simply have to use the rancheros.iso, but there a some contradictions that I don’t understand. The good news is that the VM is working as expected, but RancherOS everytime enables the hyperv-vm-tools on startup and the container os-hypervvmtools is restarted every few seconds. The logs of this container are also empty. Does anybody know if there is something special to consider? For example do a need to enable the qemu-guest-tools or the kernel-extras?   Any information or tips are welcome.   Best regards H.Budde

Are you wondering which alerts generate a support case with Nutanix Support and whom Nutanix contacts when a case is generated? There is a particular subset of alerts that have been identified as being important, so action can be taken on them. These alerts are called Alert cases. When the cluster is under a current support contract, alert cases are generated. Alert cases are handled by Nutanix Support, who reach out to the primary Support Contact (also referred to as the break/fix contact) associated with the node or cluster in question. For customers with a third-party providing front-line support, it is normal to have a representative from that company listed as the primary Support Contact. Most of the alert cases are created with a P4 priority level, which has a target response time of 24 hours. NodeDegraded and NodeAvgLoadAboveThreshold are created with a P2 priority level, which has a target response time of two hours. To know which alerts generate cases, and how to change the co

On a Nutanix cluster running VMware vSphere, you may be seeing this alert while there is no other sign of issue: Warning : Some storage containers have a high number of NFS files Impact          : High number of NFS files may cause vpxa services on esxi hosts to restart. Cause           : Number of files for respective storage containers has increased beyond 20K. This is expected with large VDI setups. Resolution      : Reduce the number of files if you observe vpxa instability. Why do we see this alert? What needs to be done about it? As is noted on the relevant KB article this alert is set to be deprecated in an upcoming release of NCC. As suggested by the Impact and Resolution information above, the alert is related to a problem encountered in vpxa. The alert was created for an issue in vSphere 5.x that caused instability for the vpxa service if the number of files in a single NFS datastore was too high. The issue was resolved by VMware some time ago so you do not need to worry abou

Below are the top knowledge base articles for the month of February 2020 KB 4116 - Alert - A1187, A1188 - ECCErrorsLast1Day, ECCErrorsLast10Days KB 4141 - Alert - A1046 - PowerSupplyDown KB 7503 - G6, G7 platforms with BIOS 41.002 and higher - DIMM Error handling and replacement policy KB 1540 - What to do when /home partition or /home/nutanix directory is full KB 1113 - HDD/SSD Troubleshooting KB 4158 - Alert - A1104 - PhysicalDiskBad KB 4409 - LCM: (LifeCycle Manager) Troubleshooting Guide KB 4519 - NCC Health Check: check_ntp KB 4188 - Alert - A1050, A1008 - IPMIError KB 2090 - AHV | Host and Guest Networking KB 4541 - Alert - A101055 - MetadataDiskMountedCheck KB 1507 - Alert IPMI IP address on Controller VM {CVM-IP} was updated to ... without following the Nutanix IP Reconfiguration procedure, can be misleading KB 4273 - NCC Health Check: aged_third_party_backup_snapshot_check KB 8792 - NCC checks: same_hypervisor_version_check, duplicate_cvm_ip_check, same_timezone_check, esx_sio

Below are new knowledge base articles published on the week of February 23-29, 2020. KB 8900 - Prism Central Alert Manager and Cassandra Monitor services crashing, NCC health checks failing for same. KB 8909 - [Karbon] Airgap (dark site) FAQ and Troubleshooting Guide KB 8956 - AHV | How to get a list of virtual machines that were restarted due to an HA event KB 8976 - Nutanix Files Share might encounter mapping issue to macOS KB 8980 - VMs deleted via vCenter may fail to restore within 7 days of deletion KB 8988 - Calm - Calm UI is not accessible on PC 5.16 KB 8995 - Storage utilization on a Hyper-V cluster steadily increasing due to failed Hyper-V Replica replications KB 9005 - Nessus Plugin ID 33851 - Network daemons not managed by the package system KB 9029 - Changing LDAP port 389 authentication to Secure LDAP (LDAPS) ports 636 or 3269 Note: You may need to log in to the Support Portal to view some of these articles.

If you have a Nutanix cluster on VMware vSphere which has been through some hypervisor upgrades, your CVM’s virtual hardware version may be a bit out of date. The main requirement from the Nutanix perspective is that CVM virtual hardware versions should be the same across the AOS cluster, and this is checked by NCC. Still, since newer virtual hardware versions include security updates and sometimes feature updates or improvements from VMware you may want to bring this up to date. If you have an AOS upgrade pending, completing that upgrade would be the easiest route to solution. If you don’t have any AOS upgrade needed, please note Nutanix does not recommend manually updating the VM hardware version for CVMs. The recommended method is to allow virtual hardware version to update during an AOS upgrade. Do be aware, if you manually updated the virtual hardware version to 13 or higher on ESXi 6.5 before Patch 4, or 6.7 before Update 3, this could cause the CVM to fail to boot. There is a kn

Lets say that you want to monitor or modify your cluster by utilizing some of our REST APIs.  To view Nutanix REST APIs, go to Prism > click on the username on the top right > REST API Explorer.  Automating the API calls can make your daily tasks easier and faster, it's a script that you write once and you can use it via bots on chat platforms like Slack and Google Hangouts.  For example, if you have multiple clusters and you wish to see the alerts on all of them, you have the option to automate a script that sends you the alerts by simply writing "<cluster name> alerts" as opposed to logging in to each cluster and inserting the credentials every time.  Here is my example snippet of a server-side automation with Node.js, the call is to show the alerts on a cluster named Prolix where you can send a basic Auth credentials to pass the login page, the parsed result will be printed in the console. It’s important to include the first line:  process.env['NODE_TLS_REJECT_UNAUTHORIZ

You might receive an alert stating that “IP x.x.x.x is authenticating to the REST API with the admin user.” The cause of this alert is that an external client is configured to authenticate to REST API with the admin user.   The admin user will be disabled in a forthcoming release. REST API calls from the admin user will stop working. The resolution is to authenticate to the REST API with a different user that has administrative privileges.   To resolve the warning, create another user as per the below procedure. This user can authenticate to the REST API without an alert being raised.   To know how to do this, check out https://portal.nutanix.com/#/page/docs/details?targetId=Web-Console-Guide-Prism-v510:wc-user-management-wc-c.html   To know more about the alert, check out https://portal.nutanix.com/#/page/kbs/details?targetId=kA00e000000kAHyCAM

There can be many scenarios like PSU troubleshooting or upgrades where we might want to know the current BMC and BIOS version in the cluster. Let’s say for a tracking/planning sheet of firmware upgrades or sometimes its asked from the Nutanix support for an open case you have.   There are two ways to get this information. Using IPMI or cli to get the BIOS and BMC versions on individual nodes. Using NCC to determine the model, BIOS and BMC version for all nodes.   To know in detail about the above methods take a look at https://portal.nutanix.com/#/page/kbs/details?targetId=kA0600000008ZDeCAM   Planning to upgrade the firmware in your cluster? Take a look at https://portal.nutanix.com/#/page/docs/details?targetId=Acropolis-Upgrade-Guide-v5_16:upg-firmware-upgrades-c.html

Earlier we have discussed the Interoperability between the different software, we also discussed the compatibility between the different software solutions and Hardware-Software integration.   What about upgrading a specific software, what is the latest version you can upgrade to? In some cases if your software is too old, you can't directly upgrade it to the latest version since each version has some prerequisites. That's why we have the upgrade paths page. which works for AOS, Prism Central and Nutanix Files upgrades. You simply choose the component with your current version and you'll see the supported versions in green under the "Target Release Version" drop-down. For more information:  To view upgrade paths for old AOS releases, please visit here http://download.nutanix.com/alerts/upgradePaths.pdf To view AOS EOL versions, please visit here http://download.nutanix.com/misc/AOS_EOL/AOS_EOL.pdf To view Prism Central EOL versions, please click here http://download.nutanix.com/misc/P

Let’s say you recently acquired a brand new Nutanix Cluster that will allow your business-critical applications like Oracle to take full advantage of low-latency storage performance and linear scalability, high availability, and simplified management.   By shifting their workloads to Nutanix, DBAs and system administrators can achieve a lower TCO and eliminate the complexity of constantly managing and optimizing their underlying compute and storage in a multitier architecture.   With the RMAN TO PLATFORM feature, you can migrate your Oracle database across platforms, making it easier than ever to get started on Nutanix.   I hope that this note empowers you to begin moving Oracle user tablespaces from Solaris to Linux on Nutanix.   See the specific details on Step by Step Migration Instructions document to get the steps for converting user tablespaces from Solaris to Linux.   Also you can consult : The complete Oracle Database Migration from Solaris to Linux technical note. The Oracle o