How It works

LACP configuration is always a tricky business for every administrator and network engineer and enabling and verifying the configuration of LACP on NX Nodes should not send chills down your spine.First, let’s understand why we need LACP What are the advantages of LACP? A single user VM with multiple TCP streams could use up to 20 Gbps of bandwidth in an AHV node with two 10 GB adapters. A traffic-hashing algorithm such as balance-TCP can split traffic between multiple links in an active-active fashion. Because the uplinks appear as a single L2 link, the algorithm can balance the traffic among bond members without any regard for switch MAC address tables. With LACP, multiple links to separate physical switches appear as a single layer-2 link.Note: To use multiple upstream switches, you must configure MLAG or vPC on the physical switch  Points to seriously consider before jumping towards configuring LACP Read the following a guide to understand AHV Networking better            AHV Networ

Below are new knowledge base articles published on the week of November 3-9, 2019.KB 7563 - NCC Health Check: ofpfmfc_table_full_check KB 8279 - Alert - A130116 - Automatic Promote Metro Availability KB 8326 - AHV | How to convert VM running Windows 10/2016/2019 from BIOS to UEFI KB 8413 - Alert-A1087-Metadata Volume Snapshot Persistent Failure KB 8451 - Move Migration Plan fails due to duplicate VMUuid KB 8477 - NCC Health Check: remote_site_in_same_datacenter_check KB 8485 - Alert - A130088 - Failed To Snapshot Entities KB 8487 - Alert - A130197 - VmRestorationFailed KB 8492 - Nutanix Files - While trying to update permission using Windows Explorer "Enable inheritance" is not working KB 8495 - Missing vnics after AHV Protection Domain Failover. "Failed to find a network: Unknown network" error KB 8504 - Pre-upgrade check: test_vlan_detection KB 8506 - Era - Unable to register Cluster with due to non-UTC timezone on Era Server VM KB 8520 - Alert - A130199 - VM Migration Failed KB 8521

Cluster data usage grows and occasionally it grows more rapidly then planned and expected before a new node to be added to handle this growth. Nutanix offers data Compression and Deduplication to hold more data in the container, by reducing the stored size and avoiding duplicate data, respectively. Usually Compression should be used first, as de-duplication is recommended only on some specific scenarios (please, check documentation links provided below).   Compression: ============ You can enable compression on a storage container. Compression can save physical storage space and improve I/O bandwidth and memory usage which may have a positive impact on overall system performance. The following types of compression are available. -  Post-process compression:   Data is compressed after it is written. The delay time between write and compression is configurable, and Nutanix recommends a delay of 60 minutes. If compression is enabled in "Post-process", then existing data will also

I have had a vm fail and now I need access to its storage drive, is there a way to mount its storage drive to a new vm or browse and recover data?

NOTE: If you are running AOS 5.11.x Please upgrade to AOS 5.11.1.2 or higher before enabling the Remote Support TunnelWhat is the Remote Support TunnelThe Remote Support Tunnel allows Nutanix SREs and Engineers to access a cluster via a Secure SSH Tunnel.We can also send a public SSH key to allow access without the need for the cluster password to be exchanged. Enable the Remote Support Tunnel In Prism, go to; Settings, by clicking the gear wheel or from the Dashboard selection Select Remote Support Enable for n hour(s) (maximum of 24 hours)After 5 - 10 minutes, go back or refresh the Remote Support setting screen to see the Status. The Connection Status should be reporting “Success”.    If the status is reported as Disabled, (as seen below), please review the following Knowledge Base Article for Troubleshooting options. Nutanix Remote Support Tunnel Troubleshooting Guide   

Could we get some more documentation on this command?Clone-NTNXVirtualMachineOne of the required parameters is -SpecListI would like some clarification on this parameter and how to use it. Is this an array of values? Am I linking to a csv or json file?An example would be nice.Using the UI, I currently clone a reference image to create vm’s and I would love to get a PS script to work to do the same. I can get it to work with the ACLI commands but PS is a bit more versatile and I believe I could get it to do more if I understood this command and specifically this parameter better.

Hello Masters, Today I have this alert: System Non-Root Partitions Space Usage HighI read this KB https://next.nutanix.com/installation-configuration-23/cvm-non-root-partitions-space-usage-high-clear-space-safetly-33597and check my filesystem and all files inside of this directories are ok (small sizes):/home/nutanix/data/cores/ /home/nutanix/data/binary_logs/ /home/nutanix/data/ncc/installer/ /home/nutanix/data/log_collector/My problem look to be at the file  “/home/nutanix/data/logs/ncc_log_collector.log”. It has actually this size (14 and 6 GB): $ allssh du -h ~/data/logs/ncc_log_collector.log================== 192.168.150.92 =================5.7G /home/nutanix/data/logs/ncc_log_collector.log================== 192.168.150.94 =================14G /home/nutanix/data/logs/ncc_log_collector.log================== 192.168.150.96 =================238M /home/nutanix/data/logs/ncc_log_collector.logthe file don’t show error, only INFO. It safe to delete it ?? can I delete it using “rm /home/n

Occasionally you will need to deal with nutanix support who needs access to your cluster but not over a webex or zoom sessions. You can always setup a remote support channel so nutanix SREs can perform troubleshooting in the background, leaving you free to work on other more important issues in your environment. Simply run:To check if it is already set up:nutanix@cvm$ ncli cluster get-remote-support-statusTo start itnutanix@cvm$ ncli cluster start-remote-supportTo close itnutanix@cvm$ ncli cluster stop-remote-support For troubleshooting  instructions, please refer to public nutanix KB titled: “Nutanix remote Support Channel Troubleshooting Guide”. 

In order to check Nutanix AOS, AHV or 3rd Party Hypervisor Compatibility with certified hardware, you can visit the Nutanix portal and select "Compatibility Matrix". Home > Documentation > Compatibility MatrixYou can check compatibility for:Hardware model, for e.g. NX, HPE, Dell, Cisco UCS AOS Version Hypervisor Version (AHV, ESXi, Hyper-V, XEN)You can filter by hardware model, AOS version or Hypervisor Version.Visit Compatibility Matrix (Nutanix portal account required) to check for hardware compatibility with AOS and Hypervisors (AHV / ESXi / Hyper-V / XEN).From the same page, you can also check AHV Guest OS compatibility as well. 

Hi everybody. Is it possible to know if SATADOM is broken or has anomalies?Or the only way to know is to restart a node? thanks!

Below are new knowledge base articles published on the week of October 27-November 2, 2019.KB 7837 - How to update expired licenses KB 8266 - Alert - A1037 - Node Forwarding Metadata Requests KB 8275 - Alert - A130161 - Protection Rule Conflict Occurred KB 8369 - AHV: Create Tenable Core with Nessus VM from OVA KB 8388 - Expected Inputs for Alerts Notify Creation on API v3 KB 8393 - CVM inaccessible via SSH after removing node or destroying cluster KB 8434 - Host Kernel panic due to BIOS setting change (CPU Configuration settings) KB 8439 - LSI Firmware Upgrade for Hyper-V 2012R2 KB 8441 - Snapshot Chain Height has exceeded the threshold limit KB 8445 - AHV: Ubuntu Cloud Image VM creation KB 8446 - Portable Foundation Failing "ERROR Exception in <ImagingStepInitIPMI(<NodeConfig(x.x.x.x) @af70>)" KB 8453 - Alert - A1044 - DiskBad KB 8454 - High utilization noticed on CVM loopback interface KB 8459 - [Karbon] Password with $ symbol breaks K8s cluster deployment KB 8460 - Alert-

Below are the top knowledge base articles for the month of October 2019.KB 4141 - Alert - A1046 - PowerSupplyDown KB 1540 - What to do when /home partition or /home/nutanix directory is full KB 4905 - NCC Health Check: notifications_dropped_check KB 4188 - Alert - A1050, A1008 - IPMIError KB 4409 - LCM: (LifeCycle Manager) Troubleshooting Guide KB 1113 - HDD/SSD Troubleshooting KB 4116 - Alert - A1187, A1188 - ECCErrorsLast1Day, ECCErrorsLast10Days KB 4519 - NCC Health Check: check_ntp KB 2090 - AHV | Host and Guest Networking KB 7503 - G6, G7 platforms with BIOS 41.002 -DIMM Error handling and replacement policy KB 4158 - Alert - A1104 - PhysicalDiskBad KB 2608 - Cluster Health Checks Incorrectly Show Warnings and Failures KB 4273 - NCC Health Check: aged_third_party_backup_snapshot_check KB 3523 - How to create a Phoenix ISO or AHV ISO from a CVM or Foundation VM KB 4541 - Alert - MetadataDiskMountedCheck KB 1863 - NCC Health Check: sufficient_disk_space_check KB 3357 - NCC Health Ch

 In some scenarios, we might have a need to find the disks associated with a virtual machine hosted on a Nutanix AHV Cluster. Nutanix offers a distributed storage fabric, which forms a large storage pool across “N” number of nodes. we then create containers for different types of workloads or we can easily use a single container as well.Virtual Machines are created via Prism Management and all vDisks associated with the VMs are hosted on the distributed storage pool. On a Nutanix AOS Cluster, Acropolis service runs across the cluster to manage virtual machine operations and configurations. As both Data & MetaData is distributed across the cluster, we can use the “Acropolis” service on almost any CVM to retrieve virtual machine information running on any node.In order to interact with the Acropolis service, Nutanix offers “acli”. a powerful set of commands for virtual machine operations across your Nutanix cluster.acli offers tab-completion once inside the acli shell or we can execu

Hi, is there a way how to get VM/Host/Cluster/etc. statistics from API in bulk? We know about the /stats endpoints, but those only allow to get stats for single entity, so in case there are hundreds of VMs in cluster that would mean hundreds of separate requests to API. We don't really want to put lot of strain on the cluster just by polling stats. Our default polling interval is several minutes, so using projection=stats doesn't really work for us (as I believe it provides current value) and we'd need to know the stats for the time interval between our polls. Thanks so much for your time. Best regards,Marek

Confused regarding crash-consistent snapshot and application-consistent snapshot? Crash-Consistent SnapshotVM snapshots are by default crash-consistent, which means that the vDisks captured are consistent with a single point in time. The snapshot represents the on-disk data as if the VM crashed or the power cord was pulled from the server—it doesn’t include anything that was in memory when the snapshot was taken. Today, most applications can recover well from crash-consistent snapshots. Application-Consistent SnapshotApplication-consistent snapshots capture the same data as crash-consistent snapshots, with the addition of all data in memory and all transactions in process. Because of their extra content, application-consistent snapshots are the most involved and take the longest to perform.While most organizations find crash-consistent snapshots to be sufficient, Nutanix also supports application-consistent snapshots. The Nutanix application-consistent snapshot uses Nutanix Volume Shad

Sometimes, NCC (Nutanix Cluster Check) will alert for PSU down and a case is automatically created when a PSU down is detected.This can be a false positive alert or are transient errors (when relocating a node or some maintenance which requires a PSU down).However, a Power supply alert should never be taken lightly. One can always verify the physical status of Power supply by checking the LED indicators on the PSU unit itself, but in certain cases this may not be possible right away - due to the equipment location etc.  To ensure Power Supplies of a Block or Node are in optimal state, you can login to the physical host or execute a command from the CVM to check their status:From a AHV Host:[root@AHV-HOST~]# ipmitool sdr | grep -i psPS1 Status       | 0x01              | okPS2 Status       | 0x01              | okFrom a CVM on AHV (this will query all HOSTS in the cluster):nutanix@cvm$ hostssh "ipmitool sdr | grep -i ps"From a CVM on a ESXi Host:You can query all hosts PSU status:nutani

Hey guys,I have uploaded some of ISO and disk image files including VirtIO to container via Prism Central and they all are marked as ACTIVE means that the upload process has completed successfully.But when I looked into the container to find the uploaded files via command line or WinSCP, all of them are being kept in container except for VirtIO iso file. Do you know how I can find it? Thanks and Regards. 

 Hi, can someone be kind enough to explain what is Downstroking? this effects my SSD config. It happened when I had configured 1x 1.92TB SSD and changed to 2x 960GB SSD (can’t), and forced me to configure to 2x 1.92TB SSD instead. TIA. 

Nutanix offers a monitoring & alerting utility, NCC - Nutanix Cluster check, which comes    pre-installed on every CVM.Nutanix Cluster Check is a powerful framework which includes several plugins to monitor & alert on different hardware and software components. It is installed on every CVM across a single cluster on all nodes.It comes with almost all Nutanix products, such as Prism Central, Files etc. NCC will include the relevant checks for these products, enabling Nutanix administrators to keep on top of any software, configuration issues. Following are some of the important Hardware (Physical Components) checks and alerting offered byNCC:a) Periodic Disk checksb) Periodic BIOS / BMC Firmware checks (Versions / Upgrades required)c) Boot Devices Monitoring (Sata DOM or M.2)d) SSD and HDD wear and tear checkse) BIOS, BMC checks to ensure all hardware components are current and stablef) Network Interface card checksg) IPMI / Out of band management interface checksh) Power Supply

Below are new knowledge base articles published on the week of October 20-26, 2019.KB 4639 - How to place CVM and Host in maintenance mode KB 8232 - Network Segmentation with isolated physical NIC KB 8285 - Alert - A130152 - Two Node Cluster changed state to stand-alone mode KB 8308 - Cluster expansion with Network Segmentation KB 8370 - Determining expected bandwidth for Pulse data KB 8376 - [Karbon] Node image upgrade stalls at 5% when an HTTP proxy is configured on Prism Element KB 8380 - Remote Site Connectivity Test - Test Connection Feature in PRISM KB 8386 - NCC check failure with "ERR : Failed to fetch NIC list on X.X.X.X for 00:0c:29:b2:a3:20" KB 8399 - SSR volume detach may issue the Windows event log "Error code: 0xc000009d" if FSRM quota is enabled KB 8400 - Interpreting advertised capacity configuration and logical usages of Storage Containers KB 8401 - Manage IPMI Configuration > Alerts through SMCIPMITOOL KB 8403 - Alert - A130108 - Cloud remote site failed to start K

Let’s say you’re running a Nutanix cluster with ESXi as a hypervisor and noticing a lot of files in your datastore and different service crashing.You’re running a VDI workload and number of files in the datastore is exceeding 30,000 and the vpxa service is crashing. This can be critical for your infrastructure as vpxa is responsible for communication between the ESXi host and the VirtualCenter Server. Does Nutanix give you a warning before the file limit is reached?Absolutely yes!The NCC health check nfs_file_count_check verifies if an NFS datastore has more than 20,000 files.Want to run the check individually?  Login from any CVM and try running the following command: nutanix@cvm$ ncc health_checks hypervisor_checks nfs_file_count_check Nutanix recommends that the number of files in any datastore should be less than 20,000 if vpxa instability issues occur.Want to manually check the number of files in any datastore?Try running the following command from an ESXi host shellfind /vmfs/vol

Hello,We plan to move some nutanix cluster to a new location and we already know that it will not be possible to keep the network vlan and IP setup for the installation.We found the document on how to change the CVM IP and you first advice is to keep the initial network if possible.So we should understood what are the argument pro and against regarding the IP reconfiguration process, can we have some performance issue?In some case we want to keep the data on the cluster and other don't take care of them. So what are the best choice to manage this different move.And if some of yours have already perform this kind of operation, could you share your feedback or advice on that.Thanks for your answers.Regards.David ALEXANDRE.

Hi We currently use Files and I have created an SMB share. The share is setup to store disaster recovery information in the event a production failure. We use Data protection to replicate the share to another Nutanix cluster at another site.We want the ability to access this share in the event that our Active Directory isn’t available. I was looking for a location either in PE or PC where I could create local accounts and assign them as a user to this share but have been unable to find.Is this possible to do? Thanks 

If you are running AOS 5.10 or later, you may notice some user VM in Prism show two IP addresses. This could happen if the VM releases its older IP address and acquire a new one. You will see both old and new IP addresses in Prism for this VM, even if VM uses a temporary IP address like APIPA addresses (APIPA addresses fall into 169.254.1.0 - 169.254.254.255 range and use 255.255.0.0 netmask)This is by design and should nto last more than 10 minutes as we scan the arp cache every 10 minutes as we scan the ARP cache every 10 minutes to detect duplicate entries, and remove all entries corresponding to an IP address except for the most recent one. Tis has been explained in your nutanix portal kb 7652.

 There may come a time when you must expand your cluster storage capacity since you predict more data will be added to the cluster in the near future.Also, you are already short of space and cannot gain any more space by deleting the old snapshots and removing VM not needed any more, etc.In that situation, assuming you have empty slots in your hardware chassis and depending on the type of the drive you are adding you will need to follow the instruction in your nutanix portal document titled “Adding a Drive”.If you do not have any more disk slot on your hardware to, you may have to add a new node to your cluster altogether, so it can expand the capacity of your cluster. This has the benefit of adding memory and cpu resources as well.