Foundation installation process fails and IPMI issue on Lenovo HX

Hello All,

I was trying to deploy Nutanix AOS 5.15.1 and HyperV2016 on 03 Lenovo HX-2321 nodes using Foundation 4.5.4.

At the first time, the deployment failed with this message : "Fatal running pre-install sanity tests" for each of the 03 nodes. Please see below all the related log messages :

==================================================================

2020-08-18 17:02:12,832Z DEBUG Setting state of <ImagingStepValidation(<NodeConfig(10.105.254.200) @b390>) @be70> from PENDING to RUNNING
2020-08-18 17:02:12,838Z INFO Running <ImagingStepValidation(<NodeConfig(10.105.254.200) @b390>) @be70>
2020-08-18 17:02:12,845Z DEBUG Cache MISS: key(<function common_validations at 0x0399D2B0>_()_{'global_config': <foundation.config_manager.GlobalConfig object at 0x0518B890>})
2020-08-18 17:02:12,845Z INFO Validating parameters. This may take few minutes
2020-08-18 17:02:12,845Z INFO Validating parameters. This may take few minutes
2020-08-18 17:02:12,846Z INFO Validating parameters. This may take few minutes
2020-08-18 17:03:52,963Z DEBUG Validating hyperv: C:\Program Files (x86)\Nutanix\Portable Foundation\isos\hypervisor\hyperv\SW_DVD9_Win_Server_STD_CORE_2016_64Bit_English_-4_DC_STD_MLF_X21-70526.iso
2020-08-18 17:03:52,964Z DEBUG Starting whitelist_check
2020-08-18 17:03:52,964Z DEBUG in_memory iso_whitelist version: 1594717124
2020-08-18 17:03:52,964Z DEBUG Cache HIT: key(<function get_nos_version_from_tarball at 0x030EFE30>_()_{'nos_package_path': u'C:\\Program Files (x86)\\Nutanix\\Portable Foundation\\nos\\nutanix_installer_package-release-euphrates-5.15.1.1-stable-x86_64.tar.gz'})
2020-08-18 17:03:52,966Z DEBUG Finished whitelist_check
2020-08-18 17:03:52,966Z INFO metadata update: {"iso_entry": {"min_foundation": "3.11.2", "deprecated": null, "min_nos": "5.5", "friendly_name": "Windows 2016 VL (SW_DVD9_Win_Server_STD_CORE_2016_64Bit_English_-4_DC_STD_MLF_X21-70526.ISO)", "compatible_versions": {"hyperv": v".*2016.*"]}, "version": "2016", "hypervisor": "hyperv", "unsupported_hardware": e], "skus": s"datacenter", "standard", "standard_gui", "datacenter_gui"]}}
2020-08-18 17:03:52,967Z INFO Ensuring there is no Haswell, Broadwell mix in the same chassis for cluster CLUSTER-NU01
2020-08-18 17:03:53,285Z WARNING tmap was called <function read_hardware_config_from_any at 0x03350870> with empty args and kwargs, ignoring
2020-08-18 17:03:53,285Z INFO Validating whether all members of cluster CLUSTER-NU01 belong to the same license class
2020-08-18 17:04:35,319Z ERROR Foundation is unable to determine state of node with ip 10.105.254.200, postponing validation of intermixing of software only/appliance nodes to cluster creation stage
2020-08-18 17:04:35,319Z INFO Running one and two node validations
2020-08-18 17:05:17,348Z DEBUG Setting state of <ImagingStepValidation(<NodeConfig(10.105.254.200) @b390>) @be70> from RUNNING to FINISHED
2020-08-18 17:05:17,351Z INFO Completed <ImagingStepValidation(<NodeConfig(10.105.254.200) @b390>) @be70>
2020-08-18 17:05:17,357Z DEBUG Setting state of <GetNosVersion(<NodeConfig(10.105.254.200) @b390>) @b450> from PENDING to RUNNING
2020-08-18 17:05:17,357Z INFO Running <GetNosVersion(<NodeConfig(10.105.254.200) @b390>) @b450>
2020-08-18 17:05:17,358Z INFO Node IP: CVM(10.105.254.200) HOST(10.105.254.243) IPMI(10.105.254.227)
2020-08-18 17:05:17,358Z DEBUG Cache HIT: key(<function get_nos_version_from_tarball at 0x030EFE30>_(u'C:\\Program Files (x86)\\Nutanix\\Portable Foundation\\nos\\nutanix_installer_package-release-euphrates-5.15.1.1-stable-x86_64.tar.gz',)_{})
2020-08-18 17:05:17,361Z INFO NOS Version is: 5.15.1.1
2020-08-18 17:05:17,361Z INFO metadata update: {"nos_version": "5.15.1.1"}
2020-08-18 17:05:17,364Z DEBUG Setting state of <GetNosVersion(<NodeConfig(10.105.254.200) @b390>) @b450> from RUNNING to FINISHED
2020-08-18 17:05:17,365Z INFO Completed <GetNosVersion(<NodeConfig(10.105.254.200) @b390>) @b450>
2020-08-18 17:05:17,368Z DEBUG Setting state of <ImagingStepTypeDetection(<NodeConfig(10.105.254.200) @b390>) @bc10> from PENDING to RUNNING
2020-08-18 17:05:17,371Z INFO Running <ImagingStepTypeDetection(<NodeConfig(10.105.254.200) @b390>) @bc10>
2020-08-18 17:05:17,371Z INFO Attempting to detect device type on 10.105.254.227
2020-08-18 17:05:17,372Z INFO Factory mode is False
2020-08-18 17:05:17,930Z INFO Manufacturer ID = 19046
2020-08-18 17:05:17,976Z INFO Detected class lenovo_asu for node with IPMI IP 10.105.254.227
2020-08-18 17:05:18,023Z ERROR Could not execute '<obfuscated_command>'
Traceback (most recent call last):
File "bmc_utils\tools.py", line 78, in system
File "site-packages\subprocess32.py", line 617, in __init__
File "site-packages\subprocess32.py", line 922, in _execute_child
WindowsError: EError 2] Le fichier sp�cifi� est introuvable
2020-08-18 17:05:18,028Z ERROR Command '<obfuscated_command>' returned error code -1
stdout:

stderr:

2020-08-18 17:05:18,029Z DEBUG Setting state of <ImagingStepTypeDetection(<NodeConfig(10.105.254.200) @b390>) @bc10> from RUNNING to FINISHED
2020-08-18 17:05:18,030Z INFO Completed <ImagingStepTypeDetection(<NodeConfig(10.105.254.200) @b390>) @bc10>
2020-08-18 17:05:18,032Z DEBUG Setting state of <ImagingStepPrepareVendor(<NodeConfig(10.105.254.200) @b390>) @ba50> from PENDING to RUNNING
2020-08-18 17:05:18,035Z INFO Running <ImagingStepPrepareVendor(<NodeConfig(10.105.254.200) @b390>) @ba50>
2020-08-18 17:05:18,036Z DEBUG Setting state of <ImagingStepPrepareVendor(<NodeConfig(10.105.254.200) @b390>) @ba50> from RUNNING to FINISHED
2020-08-18 17:05:18,038Z INFO Completed <ImagingStepPrepareVendor(<NodeConfig(10.105.254.200) @b390>) @ba50>
2020-08-18 17:05:18,039Z DEBUG Setting state of <ImagingStepInitCVM(<NodeConfig(10.105.254.200) @b390>) @b9b0> from PENDING to RUNNING
2020-08-18 17:05:18,040Z DEBUG Setting state of <ImagingStepInitIPMI(<NodeConfig(10.105.254.200) @b390>) @bf30> from PENDING to RUNNING
2020-08-18 17:05:18,042Z INFO Running <ImagingStepInitCVM(<NodeConfig(10.105.254.200) @b390>) @b9b0>
2020-08-18 17:05:18,043Z INFO Running <ImagingStepInitIPMI(<NodeConfig(10.105.254.200) @b390>) @bf30>
2020-08-18 17:05:18,046Z DEBUG skipped, this node will be imaged via IPMI
2020-08-18 17:05:18,046Z INFO Making node specific Phoenix image
2020-08-18 17:05:18,048Z DEBUG Setting state of <ImagingStepInitCVM(<NodeConfig(10.105.254.200) @b390>) @b9b0> from RUNNING to FINISHED
2020-08-18 17:05:18,051Z INFO Completed <ImagingStepInitCVM(<NodeConfig(10.105.254.200) @b390>) @b9b0>
2020-08-18 17:05:48,118Z INFO Using foundation-layouts egg C:\Program Files (x86)\Nutanix\Portable Foundation\lib/foundation-platforms\foundation_layout-1.8+r.1235.fabe063-py2.7.egg
2020-08-18 17:05:48,213Z DEBUG Injecting aurora-1.2.6-fde6167.tar.gz to phoenix
2020-08-18 17:05:48,384Z DEBUG Injecting tartarus-0.7.1-f6a45e22.tar.gz to phoenix
2020-08-18 17:05:48,654Z DEBUG Cache HIT: key(<function get_md5sum at 0x03348E70>_('C:\\Program Files (x86)\\Nutanix\\Portable Foundation\\lib\\phoenix\\x86_64\\squashfs.img',)_{})
2020-08-18 17:05:48,657Z DEBUG Cache HIT: key(<function get_md5sum at 0x03348E70>_('C:\\Program Files (x86)\\Nutanix\\Portable Foundation\\lib\\phoenix\\x86_64\\squashfs.img',)_{})
2020-08-18 17:05:48,661Z DEBUG using C:\Program Files (x86)\Nutanix\Portable Foundation\lib\bin\cygwin\mkisofs.exe for mkisofs
2020-08-18 17:05:49,252Z INFO Setting boot mode
2020-08-18 17:05:49,253Z DEBUG No implementation found for setting boot mode to BootMode.LEGACY. Skipping
2020-08-18 17:05:49,255Z INFO Powering off node
2020-08-18 17:05:49,266Z ERROR Could not execute '<obfuscated_command>'
Traceback (most recent call last):
File "bmc_utils\tools.py", line 78, in system
File "site-packages\subprocess32.py", line 617, in __init__
File "site-packages\subprocess32.py", line 922, in _execute_child
WindowsError: WError 2] Le fichier sp�cifi� est introuvable
2020-08-18 17:05:49,269Z ERROR Command '<obfuscated_command>' returned error code -1
stdout:

stderr:

2020-08-18 17:05:54,278Z ERROR Could not execute '<obfuscated_command>'
Traceback (most recent call last):
File "bmc_utils\tools.py", line 78, in system
File "site-packages\subprocess32.py", line 617, in __init__
File "site-packages\subprocess32.py", line 922, in _execute_child
WindowsError: dError 2] Le fichier sp�cifi� est introuvable
2020-08-18 17:05:54,280Z ERROR Command '<obfuscated_command>' returned error code -1
stdout:

stderr:

2020-08-18 17:05:59,290Z ERROR Could not execute '<obfuscated_command>'
Traceback (most recent call last):
File "bmc_utils\tools.py", line 78, in system
File "site-packages\subprocess32.py", line 617, in __init__
File "site-packages\subprocess32.py", line 922, in _execute_child
WindowsError: tError 2] Le fichier sp�cifi� est introuvable
2020-08-18 17:05:59,292Z ERROR Command '<obfuscated_command>' returned error code -1
stdout:

stderr:

2020-08-18 17:06:04,299Z ERROR Could not execute '<obfuscated_command>'
Traceback (most recent call last):
File "bmc_utils\tools.py", line 78, in system
File "site-packages\subprocess32.py", line 617, in __init__
File "site-packages\subprocess32.py", line 922, in _execute_child
WindowsError: Error 2] Le fichier sp�cifi� est introuvable
2020-08-18 17:06:04,302Z ERROR Command '<obfuscated_command>' returned error code -1
stdout:

stderr:

2020-08-18 17:06:09,306Z ERROR Could not execute '<obfuscated_command>'
Traceback (most recent call last):
File "bmc_utils\tools.py", line 78, in system
File "site-packages\subprocess32.py", line 617, in __init__
File "site-packages\subprocess32.py", line 922, in _execute_child
WindowsError: 9Error 2] Le fichier sp�cifi� est introuvable
2020-08-18 17:06:09,308Z ERROR Command '<obfuscated_command>' returned error code -1
stdout:

stderr:

2020-08-18 17:06:14,321Z ERROR Could not execute '<obfuscated_command>'
Traceback (most recent call last):
File "bmc_utils\tools.py", line 78, in system
File "site-packages\subprocess32.py", line 617, in __init__
File "site-packages\subprocess32.py", line 922, in _execute_child
WindowsError: ,Error 2] Le fichier sp�cifi� est introuvable
2020-08-18 17:06:14,325Z ERROR Command '<obfuscated_command>' returned error code -1
stdout:

stderr:

2020-08-18 17:06:14,332Z ERROR Could not execute '<obfuscated_command>'
Traceback (most recent call last):
File "bmc_utils\tools.py", line 78, in system
File "site-packages\subprocess32.py", line 617, in __init__
File "site-packages\subprocess32.py", line 922, in _execute_child
WindowsError: sError 2] Le fichier sp�cifi� est introuvable
2020-08-18 17:06:14,332Z ERROR Command '<obfuscated_command>' returned error code -1
stdout:

stderr:

2020-08-18 17:06:14,332Z ERROR Exception in <ImagingStepInitIPMI(<NodeConfig(10.105.254.200) @b390>) @bf30>
Traceback (most recent call last):
File "foundation\decorators.py", line 77, in wrap_method
File "foundation\imaging_step_init_ipmi.py", line 265, in run
File "foundation\imaging_step_init_ipmi.py", line 110, in boot_phoenix
File "bmc_utils\_remote_boot_ipmi.py", line 54, in poweroff
File "bmc_utils\tools.py", line 178, in ipmitool_with_retry
StandardError: Command '<obfuscated_command>' returned error code -1
stdout:

stderr:

2020-08-18 17:06:14,388Z ERROR Exception in running <ImagingStepInitIPMI(<NodeConfig(10.105.254.200) @b390>) @bf30>
Traceback (most recent call last):
File "foundation\imaging_step.py", line 161, in _run
File "foundation\decorators.py", line 77, in wrap_method
File "foundation\imaging_step_init_ipmi.py", line 265, in run
File "foundation\imaging_step_init_ipmi.py", line 110, in boot_phoenix
File "bmc_utils\_remote_boot_ipmi.py", line 54, in poweroff
File "bmc_utils\tools.py", line 178, in ipmitool_with_retry
StandardError: Command '<obfuscated_command>' returned error code -1
stdout:

stderr:

2020-08-18 17:06:14,392Z DEBUG Setting state of <ImagingStepInitIPMI(<NodeConfig(10.105.254.200) @b390>) @bf30> from RUNNING to FAILED
2020-08-18 17:06:14,394Z DEBUG Setting state of <ImagingStepRAIDCheckPhoenix(<NodeConfig(10.105.254.200) @b390>) @b1b0> from PENDING to NR
2020-08-18 17:06:14,394Z WARNING Skipping <ImagingStepRAIDCheckPhoenix(<NodeConfig(10.105.254.200) @b390>) @b1b0> because dependencies not met, failed tasks: .<ImagingStepInitIPMI(<NodeConfig(10.105.254.200) @b390>) @bf30>]
2020-08-18 17:06:14,398Z DEBUG Setting state of <ImagingStepPreInstall(<NodeConfig(10.105.254.200) @b390>) @0c70> from PENDING to NR
2020-08-18 17:06:14,398Z WARNING Skipping <ImagingStepPreInstall(<NodeConfig(10.105.254.200) @b390>) @0c70> because dependencies not met
2020-08-18 17:06:14,400Z DEBUG Setting state of <InstallHypervisorHYPERV(<NodeConfig(10.105.254.200) @b390>) @0990> from PENDING to NR
2020-08-18 17:06:14,401Z DEBUG Setting state of <ImagingStepPhoenix(<NodeConfig(10.105.254.200) @b390>) @0bf0> from PENDING to NR
2020-08-18 17:06:14,401Z WARNING Skipping <InstallHypervisorHYPERV(<NodeConfig(10.105.254.200) @b390>) @0990> because dependencies not met
2020-08-18 17:06:14,401Z WARNING Skipping <ImagingStepPhoenix(<NodeConfig(10.105.254.200) @b390>) @0bf0> because dependencies not met

==================================================================

I did some research on Internet, and on some nutanix discussions forums and I found that I have to reset all the 3 IPMI to the factory state.

I did it, and after that, Foundation failed with a new error message : "Failed to connect to the following IPMIs. Here are some possible reasons..…

Please below all the related screenshots :

Please need your help.

Page 1 / 1

Hello,

Sorry to hear about your distress.

Are you able to login to the IPMIs?

How many network connections are on these modes? The msg. about the shared IPMI vs the Dedicated one might indicate that you are not connecting to the active IPMI.

If you start with a basic switch connected to the Dedicated IPMIs can you login to the IPMI?

If you start with a basic switch connected to the Shared IPMIs can you login to the IPMI?

Let us know...

Hello Roberto,

Please find below answers to your questions :

Are you able to login to the IPMIs? ==> Yes

How many network connections are on these modes? ==> 02 connections per node (01 x 10 Gig SFP+ and the IPMI 1Gig RJ45 port) to a same basic Switch (Cisco 9200 with 24 RJ45 ports and 04 x 10 Gig SFP+ ports)

If you start with a basic switch connected to the Dedicated IPMIs can you login to the IPMI? ==> Yes

If you start with a basic switch connected to the Shared IPMIs can you login to the IPMI? ==> Unlike Supermicro models, Lenovo models do not have shared IPMI, that’s why I connected both IPMI and Data ports to the same switch.

Thank you for your responses… I was kind of hoping that it might be a shared/dedicated IPMI issue but this information seems to preclude this…

Let me look over your info again and see if there is anything else that stands out.

If anyone else has ideas please feel free to join in...

Just as a quick note… you seem to be installing an Eng Win2016 OS?

Contained within the log there is Francais (the Error 2 code response)?

Will keep looking...

I use the English version of Windows 2016. honestly I don’t why this line appears in french (perhaps because I am using Portable FOundation and my computer OS is in french...)

Hello,

Just an update on the IPMI issue. I solved it by enabling “IPMI over LAN” in the BMC menu.

When you perform a factory reset of the IPMI, this option becomes “disabled”. Even when this option is disabled you can still ping the IPMI.

I am still troubleshooting for the second issue...

That is excellent news! If I think of anything I will let you know...

Hello All,

I finally solved my issue about the error message "Fatal running pre-install sanity tests" while using Foundation to install my Nuta cluster.

Please find below all the actions taken by myself :

+ Downloading Foundation VM 4.5.4.1

+ Using VirtualBox instead of VMware Workstation to deploy the foundation VM

+ Using the lastest AOS (AOS 5.15.2) instead of AOS 5.15.1.1

+ Launching foundation and providing all informations (IP adressses, hostnames, IPMI credentials, etc).

After following these steps, all things were OK. All the 3 hypervisors are installed, the 3 CVMs too and the nutanix cluster is UP. Thanks

Sign up

Login to the community

Scanning file for viruses.

This file cannot be downloaded