How It works

Below are new knowledge base articles published on the week of February 26-March 4, 2023.KB 13178 - NCC Health Check: cluster_node_count KB 13481 - Increasing /dev/sda3 filesystem partition size on Policy VM upgraded to 3.6.1 KB 13653 - Nutanix Database Service | Pulse Telemetry KB 14018 - Alert - A130365 - PauseStretchTriggeredByWitness KB 14032 - Upgrading to NDB 2.5.1 for HA-enabled setups fails KB 14303 - Unable to delete a Nutanix Storage Container KB 14326 - AOS upgrade to 6.1.x (or above) stuck on network segmentation enabled cluster with mixed hypervisors. KB 14366 - Nutanix Database Service - MongoDB brownfield to greenfield conversion fails after upgrade from a version lower than 2.5.1 to a higher one KB 14367 - IPMI Network Configuration Requirements & Best Practices KB 14372 - Data Lens - how to re-enable file server after been disabled KB 14373 - Storage Container Usage may increase after AOS upgrade in ESXi clusters with Thick Provisioned virtual disks larger than 4Ti

We have our Nutanix 5 node cluster installed with VMware vSphere 7.0U3. RF=2 so I should be bale to withstand a 1 node failure. I would like to test the HA capability of VMware/Nutanix by taking a node offline abruptly and ensure everything works as expected before placing production workloads on it.I was thinking of placing a test VM on a node and then pulling the power cables and ensuring that VM is powered up on a remaining host.Does anyone have any better or preferred way to test HA?

Below are the top knowledge base articles for the month of February 2023.KB 1540 - [AOS Only] What to do when /home partition or /home/nutanix directory on a Controller VM (CVM) is full KB 7503 - NX Hardware [Memory] - DIMM Error handling and replacement policy KB 8885 - Alert - A15039 - IPMI SEL UECC Check KB 1381 - NCC Health Check: host_nic_error_check KB 4409 - LCM: Life Cycle Manager Troubleshooting Guide KB 3827 - Alert - A130087 - Node Degraded KB 1113 - HDD or SSD disk troubleshooting KB 2090 - AHV host networking KB 2473 - NCC Health Check: cvm_memory_usage_check KB 13870 - Prism Virtual IP is configured but unreachable alert and VIP becomes permanently unreachable observed on AOS 6.5.1 and later KB 3786 - Alert - A1081 - CuratorScanFailure KB 8514 - NCC Health Check: fs_inconsistency_check KB 4519 - NCC Health Check: check_ntp KB 13150 - NCC Health Check: cfs_fatal_check KB 5228 - NCC Health Check: pcvm_disk_usage_check KB 8094 - NCC Health Check: disk_status_check KB 4158 -

Below are new knowledge base articles published on the week of February 19-25, 2023.KB 13146 - Flow Network Security may block traffic using virtual IP address or forwarded traffic KB 13587 - List of IAM Migration Alerts during Upgrade to Prism Central pc.2022.9 KB 14017 - Alert - A130354 - FailoverTriggeredByWitness KB 14308 - Network performance of VXLAN interfaces inside VMs severely degraded if VM is running on AHV host with Broadcom BCM57414 NICs with Hardware Generic Receive Offload (GRO) feature enabled KB 14332 - Metric IO stats missing from Prism or the Insights Portal KB 14343 - Prism Central - Templates view is empty for LDAP Users on Prism Central with enabled Microservices Platform (CMSP) KB 14344 - Prism Central: report is missing values or shows only a dash (-) KB 14351 - How to obtain quota limits via API after enabling Calm Policy Engine KB 14353 - Unable to change the VPN route priority in Nutanix DRaaS KB 14369 - AHV Metro/SyncRep : Services on Pacemaker unable to ru

I got existing 3 nodes with Dell XC740 running with AHV. And I’ve order new 3 nodes x Dell XC750 but it came with factory install ESXi 7.0.Question is any special instruction do before adding new node to the existing cluster and I would like to new node XC750 to run the same AHV hypervisor in existing cluster too. So, can I follow expand step here Prism 6.5 - Expanding a Cluster (nutanix.com)?Thank you.

Below are new knowledge base articles published on the week of February 12-18, 2023.KB 14236 - PSOD on Ice Lake processor platforms after upgrading to ESXi 7.0 U3i due to an issue with microcode 0x0d000375 KB 14297 - Security Dashboard throws the error "Enable microservices infrastructure with internet connectivity for Security Dashboard to work" KB 14309 - NDB - MySQL software profile creation fails for commercial MySQL versions KB 14310 - NDB - Failed to create the software profile with error message: "local variable \'db_version\' referenced before assignment" KB 14318 - User defined Life cycle rule does not work in object bucketsNote: You may need to log in to the Support Portal to view some of these articles.

I’d like to sort my powered off VMs under AHV by last date they were powered on?Can you share your thoughts on how to display through AHV?

Hello All,I tried to install AOS 6.5.x with foundation 5.3.2 and nutanix but I faced from some issues. please find the below logs for reference2023-02-16 09:10:32,277Z INFO [1081/2430] Hypervisor installation in progress2023-02-16 09:11:02,318Z INFO [1111/2430] Hypervisor installation in progress2023-02-16 09:11:32,358Z INFO [1141/2430] Hypervisor installation in progress2023-02-16 09:12:02,371Z INFO [1171/2430] Hypervisor installation in progress2023-02-16 09:12:32,413Z INFO [1201/2430] Hypervisor installation in progress2023-02-16 09:12:32,595Z WARNING Hypervisor installation takes longer than usual2023-02-16 09:13:02,453Z INFO [1231/2430] Hypervisor installation in progress2023-02-16 09:13:32,486Z INFO [1261/2430] Hypervisor installation in progress2023-02-16 09:14:02,525Z INFO [1291/2430] Hypervisor installation in progress2023-02-16 09:14:32,563Z INFO [1321/2430] Hypervisor installation in progress

Hi there, I need some explanation about fault tolerance in a particular situation.The configuration is one 4-nodes block with each nodes configured with 2 SSDs and 4 HDDs. Also Fault Tolerance FT=1 is configured.In that case, how many disk/node are protected ?

Below are new knowledge base articles published on the week of February 5-11, 2023.KB 13748 - Accessing the IPMI on-site KB 13807 - NCC Health Check: witness_fault_domain_check KB 14190 - SQL Server Standard Edition is not able to utilize all the CPUs assigned to the VM KB 14218 - Nutanix Database Service - VM provisioning fails if AD gMSA whitelist is used KB 14229 - After upgrading to pc.2022.9, issue with upgrade of microservice infrastructure OR MSP base services KB 14233 - Enabling Distributed Autorid on systems upgraded to Nutanix Files 4.2 or later KB 14243 - Nutanix Database Service operations fail with invalid authentication credentials KB 14278 - Alert - A20032 - Containers are marked for removal KB 14284 - NDB - Clone Database Failed KB 14295 - NDB Postgres DB Server provisioning fails with "Failed to configure storage for database" with 1Gb memory profile KB 14306 - After upgrading Prism Central from pc.2022.6 to pc.2022.9, the Security Dashboard might fail to runNote: You

Hi!We are running about 100 vm’s in our cluster - so find out a specific information manually for each vm is pretty hard.I want to get the information which vm’s are stored in a specific storage container with acli on cvm.I know that i could perform acli vm.get <vm-name>  and look for the source_nfs_path of the vmdisks but it is way too much effort for 100 vm’s. Another way would be establishing a connection via SSH to /storage-container/.acropolis/vmdisk of of the cvm but i only see the vmdisk-uuids there, not the vm names i need.Is there any possibility, maybe with a for loop on each vm and a grep filter?

Hello Team,I hope you are doing well,I am learning more about Nutanix platform ,  can please share document(s) or KB(s) on how the physical disks are used in a Nutanix host? ie if the disk in slot 1 is reserved for CVM or m.2 is reserved to install the OS,I want to have my own guide that shows all models with their own disk layout according to their possible combinations -single SSD , dual or quad SSDs the rest of the slots are populated with HDDs -hybrid-or all slots are populated with SSDs ..ect.BRAdel

Below are new knowledge base articles published on the week of January 29-February 4, 2023.KB 12456 - NCC Health Check: ahv_gateway_check KB 13445 - Alert - A130377 - StaleVolumeGroup KB 13611 - Alert - A130382 - SynchronousReplicationPausedOnVM KB 13739 - Alert - A300441 - RecoveryPlanDynamicSubnetExtensionDeletionFailure KB 13773 - Alert-A130385 - SyncedVolumeGroupRunningAtSubOptimalPerformance KB 13796 - Alert - A130386 - AutoPlannedFailoverOfVolumeGroup KB 14126 - LCM Pre-check: test_ptagent_status KB 14173 - The cluster raises "A130184: Data At Rest Encryption key backup warning: Last user backup taken at MMM DD" even after performing the backup. KB 14211 - OpenShift 4.12 installation via the “openshift-install create install-config” command fails KB 14222 - Nutanix Database Service - Ubuntu/Debian DB Provisioning is failing due to static IP allocation failure KB 14224 - LCM UI is not available and inventory task stops at 0% KB 14234 - Alert - A160054 - File Server Partner Server

Below are the top knowledge base articles for the month of January 2023.KB 1540 - [AOS Only] What to do when /home partition or /home/nutanix directory on a Controller VM (CVM) is full KB 7503 - NX Hardware [Memory] - DIMM Error handling and replacement policy KB 8885 - Alert - A15039 - IPMI SEL UECC Check KB 2473 - NCC Health Check: cvm_memory_usage_check KB 13870 - Prism Virtual IP is configured but unreachable alert and VIP becomes permanently unreachable observed on AOS 6.5.1 and later KB 4409 - LCM: Life Cycle Manager Troubleshooting Guide KB 1113 - HDD or SSD disk troubleshooting KB 2090 - AHV host networking KB 3827 - Alert - A130087 - Node Degraded KB 13150 - NCC Health Check: cfs_fatal_check KB 4519 - NCC Health Check: check_ntp KB 8514 - NCC Health Check: fs_inconsistency_check KB 5228 - NCC Health Check: pcvm_disk_usage_check KB 3786 - Alert - A1081 - CuratorScanFailure KB 6153 - NCC Health Check: default_password_check and pc_default_password_check KB 4141 - Alert - A1046 -

Below are new knowledge base articles published on the week of January 22-28, 2023.KB 12548 - NCC Health Check: attached_vm_volume_different_recovery_clusters_check KB 12922 - Alert - A130369 - DistributedOplogFeatureDisabled KB 13040 - NCC Health Check: attached_vm_vg_protection_check KB 13100 - Alert - A130371/A130372/A130373/A130374/A130375 - DrNetworkSegmentationRemote KB 13513 - Alert - A160157 - FileServerVMcoreDetected KB 13889 - CRUD operations are not permitted on a stale Volume Group KB 13956 - CMSP enabled PC upgrade fails with error "Max retries done: unable to push docker image msp-registry.cmsppc.nutanix.com:5001/alpine:3.12" KB 13983 - Allow Out-of-band (OOB) snapshot for 3rd patry backups on RF1 VM KB 14177 - ESXi upgrade stuck at 14% due to vMotion taking longer than 15 minutes KB 14186 - Nutanix Files - Manual DNS verification fails even if the DNS entries are properly created KB 14195 - Acropolis service may crash with "Unexpected CAS error while trying to delete an

Below are new knowledge base articles published on the week of January 15-21, 2023.KB 10596 - Prism Central Pre-Upgrade Check: test_prism_central_cmsp_enablement_check KB 14074 - LCM Pre-check: test_empty_ilo_task_queue KB 14105 - After AOS upgrade to 6.x, users with the Prism Element Backup Admin role are unable to use AD credentials to log in to Prism Element KB 14167 - AHV upgrade is blocked in LCM 2.5.0.4 or newer if running vGPUs VMs are found KB 14185 - Power state of VMs managed by the Nutanix AHV Plugin for Citrix on AHV clusters changed to "Unknown" after upgrading AOS cluster to version AOS 6.6 or newer KB 14194 - Metrics-enabled (vhostmd) VMs fail to start on AHV 20220304.x KB 14198 - UpdateVmDbState task can often be seen on AHV clustersNote: You may need to log in to the Support Portal to view some of these articles.

Below are new knowledge base articles published on the week of January 8-14, 2023.KB 12052 - NCC Health Check: Prism Central disaster recovery PE-PC version compatibility check KB 13210 - NCC Health Check: pcvm_reboot_check KB 13272 - Pre-Upgrade Check: test_max_pe_nodes_registered_to_pc KB 13301 - Pre-Upgrade Check: test_prism_central_run_cmsp_preupgrade_check KB 13666 - Pre-Upgrade Check: test_prism_central_pcdr_compatibility_check KB 13676 - NCC Health Check: category_threshold_alert KB 13766 - Prism Central Pre-check: test_event_audit_counts_under_threshold KB 13872 - Pre-upgrade check: test_prism_central_pcdrv1_enabled_check KB 13979 - Custer creation fails on HPE DX G10 Plus nodes if there are no disks installed in a storage controller KB 14076 - In microservice infrastructure enabled Prism Centrals, /home directory space can be exhausted due to docker folders KB 14130 - LCM is running in Offline mode KB 14142 - Nutanix Kubernetes Engine - Pod to service communication does not wo

As title, Does AOS 6.x support Fault-Tolerance on ESXi 7.x ?If positive, does it support more than 4 FVTM in an ESXi 7 host ?   I had read https://docs.vmware.com/en/VMware-vSphere/7.0/com.vmware.vsphere.avail.doc/GUID-57929CF0-DA9B-407A-BF2E-E7B72708D825.html . Can I change parameters to support more than 4 FTVMs in an ESXi host?

Hi, One of my employees deleted 2 VMs with the name like xxxxxxxxxxxB1234 and xxxxxxxxxxxxxB1235 and at the same time xxxxxxxxxxxxxT1234 and xxxxxxxxxxxxxxxT1235 were deleted. (Logs show that he deleted them like less than a minute later) He claims that he did not touch those two. Is it possible that two VMs can be corelated somehow and deleting one triggers deletion of another one? One was bench and one was test. The only difference in name was one letter.

nutanix ERROR ipv4config.py:1811 Unable to get the KVM device configuration, ret 1, stdout , stderr br0-backplane: error fetching interface information: Device not found\n Anyone know what to do with this error.  I turned on remote syslogging to see why a host has been crashing and I see this as one of the errors in the logs.  The host is a Dell PE R630.Not sure if KVM means the host OS, since AHV basically is KVM or an actually connecting to iDRAC for Keyboard Mouse Video.  

What metrics are considered when Acropolis Dynamic Scheduling makes a migration decision on AHV related to CPU usage? Does ADS consider only CPU usage on the Host? Or does ADS take into consideration other VM or Host metrics such as CPU Ready, Co Stop, and/or Steal Time? 

Below are new knowledge base articles published on the week of January 1-7, 2023.KB 14008 - NDB - Operation for Oracle DB Patch not applied on the DB but NDB UI shows successful KB 14134 - Foundation and Phoenix Fail Immediately With Error "livecd files not found" KB 14147 - Active Directory, SNMP, or SMTP configuration may be lost following BIOS firmware upgrade with LCM when using G6/G7 BMC 7.11Note: You may need to log in to the Support Portal to view some of these articles.

Below are the top knowledge base articles for the month of December 2022.KB 7503 - NX Hardware [Memory] - DIMM Error handling and replacement policy KB 1540 - [AOS Only] What to do when /home partition or /home/nutanix directory on a Controller VM (CVM) is full KB 8885 - Alert - A15039 - IPMI SEL UECC Check KB 4409 - LCM: Life Cycle Manager Troubleshooting Guide KB 1113 - HDD or SSD disk troubleshooting KB 3827 - Alert - A130087 - Node Degraded KB 2090 - AHV host networking KB 3786 - Alert - A1081 - CuratorScanFailure KB 8514 - NCC Health Check: fs_inconsistency_check KB 2473 - NCC Health Check: cvm_memory_usage_check KB 4141 - Alert - A1046 - PowerSupplyDown KB 4519 - NCC Health Check: check_ntp KB 6153 - NCC Health Check: default_password_check and pc_default_password_check KB 13150 - NCC Health Check: cfs_fatal_check KB 13870 - Prism Virtual IP is configured but unreachable alert and VIP becomes permanently unreachable observed on AOS 6.5.1 and later KB 5228 - NCC Health Check: pcvm

Below are new knowledge base articles published on the week of December 25-31, 2022.KB 13517 - Cluster fetching the old SMTP server details after it has been changed KB 14084 - Nutanix Kubernetes Engine - Node Image upgrade in clusters with K8s 1.22.9+ may wipe a proxy configurationNote: You may need to log in to the Support Portal to view some of these articles.

Below are new knowledge base articles published on the week of December 18-24, 2022.KB 13919 - Nutanix Files - Removing ICAP servers causing performance issue and access denied while trying to access smb share KB 13921 - Ultimate trial expiry alert on the Prism central KB 14012 - Error "Cannot read properties of undefined (reading 'length')" while setting Category based Affinity Policies on PC KB 14033 - Upgrading NGT through Prism Central shows the task failed but NGT shows the latest version on guest VM KB 14038 - Src and Dest incorrectly show Any/Any for all rules in Flow Virtual Networking ANC VPC policies on PC 2022.6.x KB 14064 - SSH SCP function via TeraTerm fails with the error SSH2_MSG_CHANNEL_OPEN_FAILURE KB 14071 - Hypervisor does not boot after a chassis or a node replacement with mirrored NVMe drives KB 14080 - Minimum AOS version Support for HPDX FSC Platforms KB 14091 - On IAMv2 enabled Prism Central AD or LDAP user unable to edit VM or open VNC console KB 14099 - Disks