What Firewall Rules needed for ESXi, CVM and IPMI?

B
Badge +4
Hello Everyone,

There is no documentation on how to assign the firewall rules if ESXi, CVM and IPMI is on different VLANs and subnets. There is the port documentation on http://download.nutanix.com/guides/c_3_5/xhtml/index.html#oxy_ex-1/topics/appliance/block_networking_r.html
But it doesn't explain from which source address to destination address the firewall ports will be. It would be nice to have an Excel Sheet that can be shared with the customer. Can you help on that?

Thank you very much and have a wonderful day...

13 replies

dlink7
Rank
Userlevel 4
Badge +21
Traffic between CVM's hopefully doesn't have to be firewalled

THe IPMI ports are

HTTP: 80 (TCP)HTTPS: 443 (TCP)IPMI: 623 (UDP)Remote console: 5900 (TCP)Virtual media: 623 (TCP)SMASH: 22 (TCP)WS-MAN: 8889 (TCP)


Addtional CVM ports

2010 - curator
2011 - chronos
7000 - medusa
B
Badge +4
Hello dlink7,

Yes, I have all the ports need to be open, but from which source to where? CVMs, ESXi Hosts and IPMIs are all in different VLANs, but both CVMs are on the same VLAN. The firewall ports that may be helpful is that:

Source:CVM
Destination:ESXi
Bidirectional:Yes
Ports: ?

Source:CVM
Destination:IPMI
Bidirectional:Yes
Ports: ?

Source:ESXi
Destination: IPMI
Bidirectional:Yes
Ports: ?

Your help is really appreciated.

Thank you and have a wonderful day...

Burak
shuguet
Userlevel 4
Badge +21
I don't have the answer, but a question instead: Is it even supported to have the CVM and the ESXi on different VLANs? What about autopathing? And even if it is, is it wise to put a firewall in the storage path? Like, when you're gonna upgrade you nutanix cluster, or have a CVM crash, the NFS traffic will flow through the firewall?

Sylvain.
Sylvain Huguet - 2x Nutanix Technology Champion 2015-2016, French Nutanix Connect Chapter Champion, French VMUG Leader, 6x VMware vExpert 2011 to 2016
B
Badge +4
Hello Sylvain,

It would be nice if all CVM, ESXi and IPMI are on the same VLAN, but not all the customers allow this kind of configuration. You can insist about ESXi and CVM but most of them are using IPMI on different VLANs. So for this kind of situation and for all the other traffic, what would be the firewall settings as I explained before is the biggest question.

Thank you for those points also.

Hello Everyone,

Do you have any idea on Sylvains thoughts?

Have a wonderful day...
christie
Rank
Badge +6
Hello Sylvian,

The CVM and ESXi must be on the same subnet, this is to facilitate autopathing as you have stated.

Please refer to our setup guide for more details.

"All Controller VMs and hypervisor hosts must be on the same subnet. If the IPMI interfaces are connected, Nutanix recommends that they be on the same subnet as the Controller VMs and hypervisor hosts.
Guest VMs can be on a different subnet."
http://download.nutanix.com/guides/c_3_5/xhtml/index.html#oxy_ex-1/topics/ip_config/ip_config_web_c.html

-christie
dlink7
Rank
Userlevel 4
Badge +21
The local CVM will use IPMItools on the esxi host to log into IPMI to grab health data. You acutally don't need the IPMI plugged in for the Cluster to run but of course is very handy to have.
B
Badge +4
So Dlink7,

Just by plugging only 1 10GbE port, the cluster can be up and running without any problem, right? We do not need them to plug the IPMIs? Even while using foundation, the 1 GbE port acts as the IPMI and when connecting the 1GbE port, the cluster can be up and running with IPMI access and cluster access, is that correct?

Thank you very much for that wonderful explanations.
shuguet
Userlevel 4
Badge +21
I don't know about the "only 1 10GbE port" part... @dlink7 comment seems to imply that the cluster will be working fine because each CVM will query IPMI status via the IPMItool command on the underlying hypervisor.

As for the "only 1 1GbE port", it is working as you describe.
If I'm not mistaken, using only 1GbE is only supported on the NX-1xxx range.
But you are right, if you have a cable plugged in the first 1GbE NIC and not in the IPMI port (and with the right network configuration) you can access the IPMI interface using this connection.

It's very useful for setup (even if it's not supported for production), because you can use a standard 1GbE switch, connect the first 1GbE NIC of each node, then assign all the IPs (IPMI, Hypervisor, CVM) for all nodes in one fell swoop using the cluster init page.

Sylvain.
Sylvain Huguet - 2x Nutanix Technology Champion 2015-2016, French Nutanix Connect Chapter Champion, French VMUG Leader, 6x VMware vExpert 2011 to 2016
F
Badge +4
Hello dlink7,

i had used Supermicro IPMI View V2.0 due to ATEN KVM switch console is not functional well months ago and had ignored a message from IPMI View to update firmware at that time.

Will be any issues for Nutanix Block in case customers do a update of firmware from Supermicro?
dlink7
Rank
Userlevel 4
Badge +21
Just saw the last question now, IPMI firnware should follow support guidlines. Please don't just upgrade it.
J
Badge
I am interested too about it, I found this article but I am not sure it is still up date : http://vmwaremine.com/2014/09/19/nutanix-network-port-diagram/#sthash.lCbtWS7l.KlNTvjRD.dpbs
T
Badge +2
I would expect Nutanix to answer this question with a link to appropriate documentation, even it needs to be created. Understanding the flow of traffic within an infrastructure solution is critical, absolutely no excuse for not having a diagram available for its customers. This deserves immediate escalation.
T
Badge +2
FYI on the VMwaremine link, it shows port 22 needs to be open between Prism central and the CVMs, yet no where do I see this requirement is Nutanix documentation, and it is this type of discrepancy that makes a customer question the accuracy of the posting.

Also, the diagram offered up also shows ESXi connecting to IMPI but no ports showing in the diagram.

Reply