Skip to main content

Hi everyone, I have run into the issue described in the KB-18070, Case 2: CRITICAL memory_model_manager.py:83 X == 1 failed, after the power outage. Description and symptoms fits and I confirmed it in the logs. 

Problem is, that solution is to upgrade to next version and contact support for the recovery of the impacted cluster. Upgrade is not possible and its CE so support is also not an option. I was wondering if someone have similar issue or might have a clue hwo to move forward, as I would like to avoid nuking the whole cluster.  Any advice or help would be appreciated.

Why cant you upgrade? That is possible in CE. 

I can perform inventory but when i pick what to upgrade, it just hangs at “Generating Upgrade Plan...”

Hi ​@a.ovesny  Since you're hitting the upgrade block at "Generating Upgrade Plan...", it's likely related to either the upgrade service being in a bad state or metadata corruption due to the power outage.

Here are a few steps you can try before considering a full rebuild:

1.Restart the LCM service manually:

cluster stop
cluster start

or on the CVM:

genesis restart

Then try the upgrade again.

2.Clear any stuck upgrade state:

On a CVM:

rm -rf /home/nutanix/software_uploads/*

Then restart the upgrade process via Prism or CLI.

3.Check cluster services:

Make sure all services are running using:

cluster status

4.Try LCM upgrade via CLI:

Sometimes using LCM via CLI works better than the UI when it's stuck.

lcm_inventory
lcm_upgrade

5.Logs to review:

/home/nutanix/data/logs/lcm.log

/home/nutanix/data/logs/host_agent.out

If none of this helps, the last-resort path would be to manually export your VM configs and data, then rebuild the cluster and re-import. I hope this helps you avoid that scenario.

Let me know how it goes or if you see any specific log errors!

Terms & Conditions