Solved

CVM node unreachable

  • 12 August 2021
  • 6 replies
  • 1162 views
D
Badge +1

Hi Everyone,

 My Nutanix cluster had 4 node running AHV version Nutanix 20170830.337 and AOS version 5.10.9.1 LTS.

Suddenly, one of the node had CVM unreachable issue. When I’m running cluster status command got message 1 CVM down.

I’m try to run nodetool -h 0 ring got something like this. 

When I’m issue cmd ncli host list. I’m notice the problem node got this message “Node marked to be added back to metadata store”

1 of my colleague try to repair host boot device but no luck.

 

What should we do now?

 

 

icon

Best answer by Magdy 15 August 2021, 08:10

View original
This topic has been closed for comments

6 replies

B
Rank
Badge +1

Hi @dot_slash, it looks like, host is down for a long time. Login to host console vi IPMI and check the current status of the host. Based on the "nodetool -h 0 ring" output, node is detached from metadata ring due to unavailability.

D
Badge +1

Yes, this host is down for a long time. Any idea how to do?

B
Rank
Badge +1

Hi @dot_slash is host still down? if yes, could you please post screen shot for the IPMI console? Also provide output of cs | grep -v UP.

if host is down, this might be a hardware issue. Please open case with Nutanix for further assistance.

D
Badge +1

Hi @Bala Murali Krishna 

host is up. AHIV running, but CVM no connection.

cs | grep -v UP as below

This host is eol & eosl by nutanix but we still using it as DR cluster 

M

@dot_slash Please upgrade your cluster as I can see the AHV and AOS versions are old and this can cause lots of issues. Prism 5.15 - Acropolis Upgrade Guide (nutanix.com)You might as well come across cluster downtime if not upgraded soon.

S

@dot_slash First troubleshooting tip is to keep your cluster up-tp-date as the older versions can cause lots of issues which you cannot comprehend (If you are having a bad day of course).Please follow the official doc for upgrading your cluster and the order of the upgrade is also to keep in mind.

Terms & Conditions