Solved

Limit or maximum NICs on VM

  • 6 August 2020
  • 9 replies
  • 694 views

Badge

I wonder if there is any limit on the number of NICs (VLANs) in a guest VM using AHV \ AOS?

As well as, what would be the implications of a VM with multiple NICs, for example, over 20 NICs.

icon

Best answer by Paul Ilavarasu 6 August 2020, 16:17

Hi ,

vNICs per VM: You’re limited by virtual PCI bus space, and the limit varies based on VM configuration. There are 32 slots total, one goes to the scsi controller, another goes to the IDE controller (if you have an CD-ROM drive, for example), one or two go to USB controllers. You probably usually have ~25 slots free for vNICs or so.

The PCI bus itself (ie on the device side of the PCI controller circuits) is a parallel shared bus - the same 32 address/data + N control wires run to each PCI device on the motherboard, and to each extension slot. There can be a maximum of 32 devices on a PCI bus, although some of them can be a PCI “bridge” that provides a subsidiary PCI bus.

http://moi.vonos.net/linux/the-pci-bus/#:~:text=The%20PCI%20bus%20itself%20(ie,and%20to%20each%20extension%20slot.

Any limitation on VM hardware is also depend on the Guest OS , if a VM supports then it will be supported by hypervisor

For an Example: 

Windows Server 2016 supports 12 total:
- 8 Hyper-V specific network adapters
- 4 legacy network adapters

https://docs.microsoft.com/en-us/windows-server/virtualization/hyper-v/plan/plan-hyper-v-scalability-in-windows-server

View original

This topic has been closed for comments

9 replies

Userlevel 5
Badge +4

Hi ,

vNICs per VM: You’re limited by virtual PCI bus space, and the limit varies based on VM configuration. There are 32 slots total, one goes to the scsi controller, another goes to the IDE controller (if you have an CD-ROM drive, for example), one or two go to USB controllers. You probably usually have ~25 slots free for vNICs or so.

The PCI bus itself (ie on the device side of the PCI controller circuits) is a parallel shared bus - the same 32 address/data + N control wires run to each PCI device on the motherboard, and to each extension slot. There can be a maximum of 32 devices on a PCI bus, although some of them can be a PCI “bridge” that provides a subsidiary PCI bus.

http://moi.vonos.net/linux/the-pci-bus/#:~:text=The%20PCI%20bus%20itself%20(ie,and%20to%20each%20extension%20slot.

Any limitation on VM hardware is also depend on the Guest OS , if a VM supports then it will be supported by hypervisor

For an Example: 

Windows Server 2016 supports 12 total:
- 8 Hyper-V specific network adapters
- 4 legacy network adapters

https://docs.microsoft.com/en-us/windows-server/virtualization/hyper-v/plan/plan-hyper-v-scalability-in-windows-server

Badge

So, not taking into account the guest VM's OS, would it only be a maximum of 20-25 NICs?

Userlevel 5
Badge +4

This again depends on the other devices connectivity , if other devices(scsi controller, IDE controller (if you have an CD-ROM drive, for example), USB controllers.)count goes high then you NIC count will go less 

Badge

Is this information official from Nutanix?

Userlevel 5
Badge +4

AHV or Nutanix has nothing to do with Guest OS , its completely depend on your Guest OS , IF you have available resources at hardware level then your hypervisor make a visible to guest VM

Badge

We are doing tests to virtualize the Pfsense firewall on Nutanix, with each VLAN being a vNIC on the VM, currently being 28 vNICs.

 

https://docs.netgate.com/pfsense/en/latest/interfaces/using-a-large-number-of-interfaces.html

Userlevel 5
Badge +4

What is the purpose of having individual vnic for all vlans? Does it require only access ports ? you can have trunk port on VM to allow all the VLANS , Rather than having individual vnics,

 

We have trunk port configuration to meet requirements of  Virtual firewall,IPS etc 

Refer this

https://next.nutanix.com/blog-40/ahv-vm-networking-enhancements-vlan-trunking-10217

Badge

I wanted to make fewer changes to the cli level in Nutanix.

 

@bbbburns comments in https://next.nutanix.com/blog-40/ahv-vm-networking-enhancements-vlan-trunking-10217 :

For performance reasons I wouldn't recommend this approach in a high traffic environment. In production or high performance scenarios you'll have more available CPU for processing network packets on different VLANs with one vNIC per VLAN.

 

Anyway, we will do tests and see how it behaves.

 

Thank you.

Userlevel 5
Badge +4

Yes, you can configure Trunk port only on CLI,

Follow this Guide

https://portal.nutanix.com/page/documents/details?targetId=AHV-Admin-Guide-v5_17:ahv-vm-nic-set-mode-to-trunk-or-access-t.html

 

If this helps you then pls mark this as your answer , if you still need additional help then pls do let me know