AOS manual upgrade / “pre-upgrade” task

Hello @Dominik.Siedlak 

LCM Pre-Checks

test_aos_afs_compatibility

Verifies that AOS and Nutanix Files are compatible.

test_cluster_status

Verifies that the cluster statuscommand returns a healthy state.

test_cassandra_status

Checks zeus_config_proto (run zeus_config_printer for equivalent) for the following:

• Node cassandra_status is not Normal.
• No nodes are in maintenance.
• The NCC check described in KB 4301 returns a PASS.

test_nutanix_partition_space

Checks that all CVMs have sufficient space on /home/nutanix.

test_network

Verifies that CVM IPs are reachable.

test_hypervisor_tests

Verifies that SSH can connect to the hypervisor.

• AHV: verifies libvirtd on AHV hosts.
• ESXi: checks if there is a VmkNic for all ESXi hosts in the same subnet as eth0/eth2. Verifies pyVim connectivity to ESXi hypervisor.

test_zk_status

Verifies that at least three zk hosts are available, on a cluster having three or more nodes.

test_replication_factor

Verifies that no containers in the cluster have RF=1.

test_under_replication

Checks for under-replication in the cluster.

test_prism_central_prism_element_compatibility

Checks that the registered Prism Central is compatible with Prism Element by reading prism_central_compatibility.json.

test_prism_central_ova_compatibility

Checks that Prism Central has boot and data partitions on different disks.

test_degraded_node

Verifies that no nodes are degraded.

test_vlan_detection

Verifies that LCM can fetch VLAN information from all nodes using NodeManager.

test_upgrade_in_progress

Checks to see if a non-LCM upgrade is in progress.

test_foundation_service_status_in_cluster

Checks to see if the Foundation service is running on any node in the cluster.

test_vmotion_support/test_live_migration_support

Checks that an ESXi or Hyper-V host has vMotion enabled.

test_cluster_config

Verifies that the cluster has vSphere HA enabled on vCenter mapped to a Nutanix cluster. (Make sure that you disable admission control.)

test_hosts_managed_by_same_mgmt_server

Verifies that the same management server (vCenter) manages all hosts.

test_hypervisor_config

• All hosts: verifies that the host has sufficient space.
• ESXi:
  • Verifies that the host does not have APD/VMCP enabled. (If yes, disable it.)
  • Verifies that no CD-ROMs (ISO image stored in local storage) or other physical devices are attached to the VM. (If yes, remove the physical device from UVM.)
• AHV: verifies that the host can enter maintenance mode.
• Hyper-V: none.

test_check_revoke_shutdown_token

Checks that the CVM can revoke the shutdown token after an LCM update completes on a node in a cluster.

test_ntp_server_reachability

Checks that the configured NTP servers are reachable before starting LCM updates.

test_prism_central_minimum_resource_check

Check that Prism Central is configured with the correct license and has enough memory to support all licensed features.

test_two_node_cluster_checks

Check that the configuration on a two-node cluster allows LCM updates.

• Check that both nodes are up and the cluster is functional.
• Check that the cluster has selected an LCM leader.
• Check that the cluster has configured a witness VM.
• Check that there are at least two meta disks available on each CVM.

test_zk_migration

Check to see if there is a Zookeeper migration in progress.

test_catalog_workflows

Check that the most commonly used catalog workflows for LCM are currently performable.

test_foundation_workflows

Check whether the current version of Foundation on the cluster is on the LCM denylist.

test_uniform_foundation_across_cluster

Check to make sure that every node in the cluster is running the same version of Foundation.

test_esxi_bootbank_space

Check that the ESXi bootbank has enough space for LCM upgrades that involve restarting into Phoenix.

test_esx_entering_mm_pinned_vms

Check to see if any node in the cluster contains a VM that cannot be live migrated to another node.

Note: For inventory scans, LCM skips the aos_afs_compatibility and under_replication tests, because inventory scans are not disruptive.