Question

Networking problem


Badge +2
Hello,
I installed Nutanix CE (fresh install) on my x3400 M2 server. Everything seems to be ok, except network. I can ping CVM from Nutanix host and Nutanix host from CVM. I can ssh CVM and I see 9440, 80 ports are open and LISTEN.
But, I cannot access Nutanix host and CVM (no ping, no SSH) from my home network. Also, I cannot access any pc on my home network from Nutanix host and CVM.
My home network is 192.168.8.0/24, Nutanix host 192.168.8.20, CVM 192.168.8.21.
Network adapters (2) - Broadcom Gigabit Ethernet.
Pls, advise a solution for a problem.
Thank you in advance,
Mindaugas

30 replies

Badge +2
How do you connect your home network to the server? As I recall several IP Classes (192.x.y.z being one of them) are non-routable.
Badge +5
I assume that your CE cluster is located at your home and therefroe we are talking about a single physical network here.

To get things started, you may want to check if your uplink bond/s is/are configured correctly, so the cluster's virtual switch is actually able to communicate with the physical switch. Basic checks are done from CVM:

code:
manage_ovs show_interfaces

shows all physical interfaces with link status and speed.
code:
manage_ovs show_uplinks

shows bridges and their uplink interfaces (if properly assigned).

CU
Peter
Badge +2
Hello and thank you for fast response,

  1. PC I am using to connect Nutanix is on the same network (192.168.8.118) - LAN. Nutanix server and PC are connected through TP-Link switch.
  2. show_interfaces
eno1 1000 False None
eno2 1000 False None
????? No link...
show_uplinks
Uplink ports: bond0
Uplink interfaces: eno1, eno2

Regards,
Mindaugas
Badge +2
No link seems strange to me.

Could we also ask you review your manage_ovs show_lldp as well please.
Badge +2
With no link connections showing (your show interfaces = false) please review the following:

https://www.tp-link.com/us/faq-736.html

From my limited experience show_uplinks normally shows me:

Bridge: br0
Bond: br0_up
bond_mode: active-backup
interfaces: eth3, eth2, eth1, eth0
lacp: off


etc.

Your seems to imply that no bond has been created or the physical connection is not made. As always please check the blinking lights on the NIC connections.
Badge +2
I had reinstall Nutanix node and CVM as well and enabled second adapter on a server. Checked - all lamps on server and switch are blinking.
Show_interfaces
eno1 1000 False None
eno2 1000 False None
enp0s26f1u2 0 False none
Show_uplinks
Bridge br0:
Uplink ports: bond0
Uplink interfaces: eno1,eno2
... I see interfaces eno1, eno2 instead of eth1,eth2...
manage_ovs show_lldp
provide a list of subcommands, tested some of them - they works (except update options - interface not have a link state).
Badge +2
Passed all tests on switch. Before I have Esxi 6.5 on my server - all environment was doing ok and with the same cables.
Badge +2
All ovs commands I can run on CVM. On host - "command not found"...
Badge +2
Thank you for the additional information.

It bothers me that show_interfaces seems to indicate no connectivity. Let me see what other commands I might be able to dig up to help pinpoint this issue.

No duplicate IPs on the switch and ... ?
Badge +2
Please try:

ssh {root}@{AHV IP} "ovs-appctl bond/show"



near the end it should say:

slave eno0: {enabled | disabled}
may_enable: {true | false}



this should be a quick and dirty way of seeing if the cluster is seeing the links as connected or not...
Badge +2
On host
ovs-appctl bond/show bond0
slave eno1 and eno2 disabled. may_enable: false
Badge +2
To me that seems to indicate that the cluster is not seeing them as connected (that is how my ipmis, without rj-45, interfaces look to me).

If you were to take three other rj-45 enabled devices and just plug them into the cables do the devices see each other? (trying to eliminate hw vs. cluster issue)
Badge +2
No duplicate IP's. I have reloaded switch too (off/on).
Badge +2
I can see two devices router and tp-link switch. That's all on my home network today.
Tomorrow I shall do:
  • take another switch (do I need reinstall of Nutanix ce?)
  • investigate the configuration of current switch.
If you have more ideas - welcome. Thank you for your time and support. This is my first Nutanix installation. Our company is looking for cloud platform. I myself prefer Nutanix.
Badge +2
Sounds good. I will give it some thought and maybe we can try a few things again tomorrow.

Hopefully other might have some ideas too.

Take care.
Badge +2
Before you take anything apart tomorrow please try:

ifconfig



to see if flags=4163

thx.
Badge +2
Hello Roberteo,
I took a look at current switch configuration, nothing special. No VLANS used. No filtering and so on.
Later I have changed a switch and reinstalled Nutanix from scratch - everything remain as previous...

Flags before and after reinstall 4163.
Regards
Badge +2
Can Broadcom gigabit ethernet adapter could not be supported?
Badge +2
That is a good question. Broadcom is a well known adapter and I would expect it would be but let me see what I can find out.
Badge +2
What version of the AOS are you using?

I did find the following indicating previous issues with Broadcom:

"Hello Andre,

I found a guide to install the nodes,
but I made a step by step guide to make a cluster off it.
Because there is a raid controller where is not the support of HBA you'll need to make a raid 0 for all the disks.
Also for the Nutanix software I used a high performance sd card.
If you want to use 5.5 ensure you have intel nics installed in the server, otherwise it won't work.
So please use the 5.1 version with Broadcom.

I will embed a link wich is available for 3 days."

Badge +2
and from


"Network interface card (NIC)Intel-based NIC Some Community Edition users have reported success using Broadcom NICs."

Do you know your ven and dev numbers for those NICs? All signs currently are pointing toward unsupported NICs...
Badge +2
Hi,

Nutanix CE version 5.6.
My conf is:
1x240 gb ssd
3x500 gb disks (cold tier)
and 2x140 disks.
I cannot create raid 0 of one ssd disk....
Badge +2
Are things okay?
Badge +2
Network adapters Broadcom NetXtreme II Gigabit Ethernet.
I'll take a look at log files too...
Badge +2
Some messages from acropolis_ovs.log:
Creating bridge for br0 failedbr0.local is not a bridge or a socket
error running pre-migrate for br0
no bridge named br0.local

Reply