AOS manual upgrade / “pre-upgrade” task

  • 22 January 2023
  • 3 replies

Badge +1

 tl;dr how to perform pre-upgrade checks from the command line?

Hello Guys,
I have a question related to the performing manual / cli upgrade (command line). Here’s the scenario:

  • AOS with ESXi as a hypervisor,
  • multiple ROBO offices / Dark Site (no access to the internet),
  • Dark Site server with LCM,
  • very limited broadband: 16Mbit with static shaping (4Mbit) for management purposes and 12Mbit for business

It is very difficult to upload via AOS GUI 4-5GB of  upgrade and .json file. 3-4% / 10minutes and upload task informing about a ‘server issue’. I’m 99% sure that it related to the broadband shaping.

Solution is simple:

  • upload tar.gz install/upgrade bundle,
  • ‘untar’,
  • install ( ./install/bin/cluster -i ./install upgrade )

My question is: is any magic command to start pre-upgrade tasks with is normally presented in GUI if upgrade package is available? It will be very helpfull to start pre-upgrade check in-advance the main upgrade.

I tried to find the command in lab:
-- uploaded the bundle in ‘normal GUI’ way,
-- started pre-upgrade check from GUI
-- ps aux from CVM:

It looks that install/bin folder is keeping script “pre-upgrade”. I tried with different ways run it from the ‘nutanix’ account: /usr/bin/python2.7 -B /home/nutanix/software_uncompressed/nos/

No luck...



Best answer by Moustafa Hindawi 22 January 2023, 10:56

View original

This topic has been closed for comments

3 replies

Userlevel 5
Badge +6

Hello @Dominik.Siedlak 

LCM Pre-Checks


Verifies that AOS and Nutanix Files are compatible.


Verifies that the cluster statuscommand returns a healthy state.


Checks zeus_config_proto (run zeus_config_printer for equivalent) for the following:

  • Node cassandra_status is not Normal.
  • No nodes are in maintenance.
  • The NCC check described in KB 4301 returns a PASS.


Checks that all CVMs have sufficient space on /home/nutanix.


Verifies that CVM IPs are reachable.


Verifies that SSH can connect to the hypervisor.

  • AHV: verifies libvirtd on AHV hosts.
  • ESXi: checks if there is a VmkNic for all ESXi hosts in the same subnet as eth0/eth2. Verifies pyVim connectivity to ESXi hypervisor.


Verifies that at least three zk hosts are available, on a cluster having three or more nodes.


Verifies that no containers in the cluster have RF=1.


Checks for under-replication in the cluster.


Checks that the registered Prism Central is compatible with Prism Element by reading prism_central_compatibility.json.


Checks that Prism Central has boot and data partitions on different disks.


Verifies that no nodes are degraded.


Verifies that LCM can fetch VLAN information from all nodes using NodeManager.


Checks to see if a non-LCM upgrade is in progress.


Checks to see if the Foundation service is running on any node in the cluster.


Checks that an ESXi or Hyper-V host has vMotion enabled.


Verifies that the cluster has vSphere HA enabled on vCenter mapped to a Nutanix cluster. (Make sure that you disable admission control.)


Verifies that the same management server (vCenter) manages all hosts.


  • All hosts: verifies that the host has sufficient space.
  • ESXi:
    • Verifies that the host does not have APD/VMCP enabled. (If yes, disable it.)
    • Verifies that no CD-ROMs (ISO image stored in local storage) or other physical devices are attached to the VM. (If yes, remove the physical device from UVM.)
  • AHV: verifies that the host can enter maintenance mode.
  • Hyper-V: none.


Checks that the CVM can revoke the shutdown token after an LCM update completes on a node in a cluster.


Checks that the configured NTP servers are reachable before starting LCM updates.


Check that Prism Central is configured with the correct license and has enough memory to support all licensed features.


Check that the configuration on a two-node cluster allows LCM updates.

  • Check that both nodes are up and the cluster is functional.
  • Check that the cluster has selected an LCM leader.
  • Check that the cluster has configured a witness VM.
  • Check that there are at least two meta disks available on each CVM.


Check to see if there is a Zookeeper migration in progress.


Check that the most commonly used catalog workflows for LCM are currently performable.


Check whether the current version of Foundation on the cluster is on the LCM denylist.


Check to make sure that every node in the cluster is running the same version of Foundation.


Check that the ESXi bootbank has enough space for LCM upgrades that involve restarting into Phoenix.


Check to see if any node in the cluster contains a VM that cannot be live migrated to another node.

Note: For inventory scans, LCM skips the aos_afs_compatibility and under_replication tests, because inventory scans are not disruptive.

Badge +1

Hi Moustafa,
Thank you for answering me in such a short time :-) I am asking about a pre-checks related to the new release of the AOS and upgrade. I’ve downloaded upgrade package tar xzf to it with is keeping installer in the ‘install’ folder.

Install is very simple: start the command: “./install/bin/cluster -i ./install upgrade”

Process will start pre-check and if all is fine upgrade.
I  am asking how to start ‘Pre-upgrade’ from the command line instead of from GUI.


Command-line way of installation: it is showing progress in CLI and in GUI including health checks related to the upgrade:


I am asking if is any single command to process in-advance  pre-checks with no installation. Thanks!

Userlevel 5
Badge +6

Hello @Dominik.Siedlak 

All the above commands used before upgrade or update, if you want only to check the compatibility then refer to compatibility matrix as below, also you need to check the hardware itself and Hypervisor version supported or not