Skip to main content
Question

VMs - Load Balancing

  • September 3, 2026
  • 5 replies
  • 46 views

Hello.

After update to AHV 11.0.1.6, AOS 7.5.1.10 the behaviour of our cluster changes.

Before the update, the machines were distributed evenly across all hosts at startup.

Now the clients started on small number of hosts. When the host reaches 90% CPU-Util, the next host will be used.

On our hosts we have nVidia A16 installed. With 60VMs on one host (were using 2Q and 4Q profiles) the VMs don’t work as expected anymore.

Nutanix support means, that this behaviour is normal.. 

I don’t think so.

Is there a chance to achieve a better distribution of the VMs?
With the actual situation a normal work is for our users not possible.

 

Thanks in advance for your answers.

 

Regards

Dirk Emmermacher

5 replies

Forum|alt.badge.img+3
  • DrewPlaster
  • Trailblazer
  • September 4, 2026

Acropolis Dynamic Scheduling (ADS) is responsible for the initial placement of VMs and runtime optimization. When multiple hosts are available, ADS should select the host with the overall highest availability (CPU, memory, storage-controller, and vGPU).  The 90% CPU utilization threshold you mentioned is close to ADS 85% threshold. ADS specifically monitors host CPU usage and triggers migration tasks to resolve contention only if resource utilization remains greater than 85% for a 10-minute span.

Here are some things to investigate that may be causing abnormal ADS behaviour:
1) Verify ADS is Enabled
ADS should be enabled by default but something/someone may have disabled it although if you're getting VM relocations at 90% it is more than likely enabled.
2) Check for VM-VM Anti-Affinity Policies
During initial startup placement ADS soft enforces these policies; it will prefer to start the VM on a separate host but will prioritize starting the VM over non-compliance if resources are constrained.  At runtime to resolve contention, ADS will reject a migration if it results in a violation of a VM-VM anti-affinity policy.
3) Monitor and Resolve Dynamic Scheduling Failures
If the cluster is imbalanced but VMs are not migrating, check for ClusterHotspotAvoidanceNotPossible alert in Prism. This occurs if ADS cannot build a plan to resolve contention and could be due to insufficient CPU or CVM resources, affinity policy constraints that prevent migration targets, the presence of non-migratable VMs (VMs with GPU/CPU pass-through or vNUMA configured)

JeroenTielen
Forum|alt.badge.img+8
  • JeroenTielen
  • Vanguard
  • September 5, 2026

What you are facing is DFS (Depth First Scheduling). Nutanix will first fill up 1 GPU and then go to the other. 

 

More info here: https://www.nutanix.com/blog/mastering-the-power-of-rtx-pro-blackwell-server-edition-gpus-with-nutanix-ahv-hypervisor

Nutanix NTC 2022+2023+2024+2025+2026 || Community Excellence Award 2025+2026 || Nutanix NUG Champion || Nutanix Certified Instructor

Acropolis Dynamic Scheduling (ADS) is responsible for the initial placement of VMs and runtime optimization. When multiple hosts are available, ADS should select the host with the overall highest availability (CPU, memory, storage-controller, and vGPU).  The 90% CPU utilization threshold you mentioned is close to ADS 85% threshold. ADS specifically monitors host CPU usage and triggers migration tasks to resolve contention only if resource utilization remains greater than 85% for a 10-minute span.

Here are some things to investigate that may be causing abnormal ADS behaviour:
1) Verify ADS is Enabled
ADS should be enabled by default but something/someone may have disabled it although if you're getting VM relocations at 90% it is more than likely enabled.
2) Check for VM-VM Anti-Affinity Policies
During initial startup placement ADS soft enforces these policies; it will prefer to start the VM on a separate host but will prioritize starting the VM over non-compliance if resources are constrained.  At runtime to resolve contention, ADS will reject a migration if it results in a violation of a VM-VM anti-affinity policy.
3) Monitor and Resolve Dynamic Scheduling Failures
If the cluster is imbalanced but VMs are not migrating, check for ClusterHotspotAvoidanceNotPossible alert in Prism. This occurs if ADS cannot build a plan to resolve contention and could be due to insufficient CPU or CVM resources, affinity policy constraints that prevent migration targets, the presence of non-migratable VMs (VMs with GPU/CPU pass-through or vNUMA configured)

Hello Drew.

Thanks for your answer.

I suppose, that ADS is on. We’re running acli ads.update enable=true on one of eight hosts.

Policies for VM-VM Anti-Affinity Policies are not declared.

For our CAD-Machine I added VM-Host Affinity Policies (That’s only a workaround).

The event ClusterHotspotAvoidanceNotPossible is not present.

 

Sorry, but my knowledge about Nutanix is a little poor :(.

 

Regards

Dirk

JeroenTielen
Forum|alt.badge.img+8
  • JeroenTielen
  • Vanguard
  • September 7, 2026

Please read my post above. This is by design. And is related to DFS. 

Nutanix NTC 2022+2023+2024+2025+2026 || Community Excellence Award 2025+2026 || Nutanix NUG Champion || Nutanix Certified Instructor

What you are facing is DFS (Depth First Scheduling). Nutanix will first fill up 1 GPU and then go to the other. 

 

More info here: https://www.nutanix.com/blog/mastering-the-power-of-rtx-pro-blackwell-server-edition-gpus-with-nutanix-ahv-hypervisor

Hello Jeroen.

The idea of DFS is ok. I suppose, it don’t run well on our cluster. Our servers has 2 nVidia A16. But when you have more than 50 machines on one host with 2 and $G profiles, that’s too much for this host.

 

The cluster starts the VMs on one host. The host, where the VMs starts, changes every day.

Here is an example from last week:

Server                 Number of VMs

1                         2

2                         3

3                         48

4                         55

5                         9

6                         50

7                         9

8                         2

 

The load balancing isn’t well here.

Regards

Dirk

Terms & ConditionsAccessibility statement