When network I/O control is enabled, distributed switch traffic is divided into Customer Network Resource pools and/or the following predefined network resource pools: Fault Tolerance traffic, iSCSI traffic, vMotion traffic, management traffic, vSphere Replication (VR) traffic, NFS traffic, and virtual machine traffic.
From vSphere 5.0 onwards, you can also create custom network resource pools for virtual machine traffic. You can control the bandwidth each network resource pool is given by setting the physical adapter shares and host limit for each network resource pool.
The physical adapter shares assigned to a network resource pool determine what share of the total available bandwidth will be guaranteed to the traffic associated with that network resource pool in the event of contention.
A important thing to understand about Network I/O control (NIOC) is it has no impact on the network traffic unless there is contention. So during times where the network is less than 100% utilized, NIOC has no advantage or disadvantage.
The share of transmit bandwidth available to a network resource pool is determined by the network resource pool's shares and what other network resource pools are actively transmitting.
Host Limits can be applied to selected traffic types if required, however Nutanix recommends not to set limits as this may unnecessarily limit performance for given workloads when there is available bandwidth. Using shares ensures burst workloads such as vMotion can complete their workload (in this example the migration of a VM to a different ESXi host) as fast as possible where bandwidth is available, and NIOC shares will prevent other workloads being significantly impacted in the event available bandwidth is limited.
The following table shows the recommended Network Resource Pool Share values recommended for Network I/O Control (NIOC) in a Nutanix environment.
For more information, see the Tech Note VMware vSphere Networking on Nutanix