Cannot install freshly AOS | Nutanix Community
Skip to main content

Hello. i Have a 4 nodes Lenovo HX3320.

 

When i want to install AOS it fails.

 

Foundation 4.6

AOS 5.15.4

 

I updated bmc firmware and reset it. But it didnt work.

Can you help me with this problem

here is the log

 

2020-12-10 12:38:39,191Z DEBUG Setting state of <ImagingStepValidation(<NodeConfig(192.168.1.22) @e2d0>) @e5f0> from PENDING to RUNNING2020-12-10 12:38:39,197Z INFO Running <ImagingStepValidation(<NodeConfig(192.168.1.22) @e2d0>) @e5f0>2020-12-10 12:41:01,398Z DEBUG Cache HIT: key(<function common_validations at 0x03D71230>_()_{'global_config': <foundation.config_manager.GlobalConfig object at 0x054D0E90>})2020-12-10 12:41:01,405Z DEBUG Setting state of <ImagingStepValidation(<NodeConfig(192.168.1.22) @e2d0>) @e5f0> from RUNNING to FINISHED2020-12-10 12:41:01,410Z INFO Completed <ImagingStepValidation(<NodeConfig(192.168.1.22) @e2d0>) @e5f0>2020-12-10 12:41:01,415Z DEBUG Setting state of <GetNosVersion(<NodeConfig(192.168.1.22) @e2d0>) @e550> from PENDING to RUNNING2020-12-10 12:41:01,417Z INFO Running <GetNosVersion(<NodeConfig(192.168.1.22) @e2d0>) @e550>2020-12-10 12:41:01,418Z INFO Node IP: CVM(192.168.1.22) HOST(192.168.1.18) IPMI(192.168.1.14)2020-12-10 12:41:01,421Z DEBUG Cache HIT: key(<function get_nos_version_from_tarball at 0x03479230>_(u'C:\\Program Files (x86)\\Nutanix\\Portable Foundation\\nos\\nutanix_installer_package-release-euphrates-5.15.4-stable-x86_64.tar.gz',)_{})2020-12-10 12:41:01,423Z INFO NOS Version is: 5.15.42020-12-10 12:41:01,426Z INFO metadata update: {"nos_version": "5.15.4"}2020-12-10 12:41:01,428Z DEBUG Setting state of <GetNosVersion(<NodeConfig(192.168.1.22) @e2d0>) @e550> from RUNNING to FINISHED2020-12-10 12:41:01,430Z INFO Completed <GetNosVersion(<NodeConfig(192.168.1.22) @e2d0>) @e550>2020-12-10 12:41:01,438Z DEBUG Setting state of <ImagingStepTypeDetection(<NodeConfig(192.168.1.22) @e2d0>) @e5d0> from PENDING to RUNNING2020-12-10 12:41:01,440Z INFO Running <ImagingStepTypeDetection(<NodeConfig(192.168.1.22) @e2d0>) @e5d0>2020-12-10 12:41:01,441Z INFO Attempting to detect device type on 192.168.1.142020-12-10 12:41:01,443Z INFO Factory mode is False2020-12-10 12:41:01,946Z INFO Manufacturer ID = 190462020-12-10 12:41:02,073Z INFO Detected class lenovo_asu for node with IPMI IP 192.168.1.142020-12-10 12:41:02,078Z ERROR Could not execute '<obfuscated_command>'Traceback (most recent call last):  File "bmc_utils\tools.py", line 78, in system  File "site-packages\subprocess32.py", line 617, in __init__  File "site-packages\subprocess32.py", line 922, in _execute_childWindowsError: rError 2] The system cannot find the file specified2020-12-10 12:41:02,081Z ERROR Command '<obfuscated_command>' returned error code -1stdout:stderr:2020-12-10 12:41:02,082Z DEBUG Setting state of <ImagingStepTypeDetection(<NodeConfig(192.168.1.22) @e2d0>) @e5d0> from RUNNING to FINISHED2020-12-10 12:41:02,085Z INFO Completed <ImagingStepTypeDetection(<NodeConfig(192.168.1.22) @e2d0>) @e5d0>2020-12-10 12:41:02,089Z DEBUG Setting state of <ImagingStepPrepareVendor(<NodeConfig(192.168.1.22) @e2d0>) @e390> from PENDING to RUNNING2020-12-10 12:41:02,094Z INFO Running <ImagingStepPrepareVendor(<NodeConfig(192.168.1.22) @e2d0>) @e390>2020-12-10 12:41:02,095Z DEBUG Setting state of <ImagingStepPrepareVendor(<NodeConfig(192.168.1.22) @e2d0>) @e390> from RUNNING to FINISHED2020-12-10 12:41:02,096Z INFO Completed <ImagingStepPrepareVendor(<NodeConfig(192.168.1.22) @e2d0>) @e390>2020-12-10 12:41:02,107Z DEBUG Setting state of <ImagingStepInitCVM(<NodeConfig(192.168.1.22) @e2d0>) @e7d0> from PENDING to RUNNING2020-12-10 12:41:02,108Z DEBUG Setting state of <ImagingStepInitIPMI(<NodeConfig(192.168.1.22) @e2d0>) @4d30> from PENDING to RUNNING2020-12-10 12:41:02,109Z INFO Running <ImagingStepInitCVM(<NodeConfig(192.168.1.22) @e2d0>) @e7d0>2020-12-10 12:41:02,114Z INFO Running <ImagingStepInitIPMI(<NodeConfig(192.168.1.22) @e2d0>) @4d30>2020-12-10 12:41:02,114Z DEBUG skipped, this node will be imaged via IPMI2020-12-10 12:41:02,118Z INFO Making node specific Phoenix image2020-12-10 12:41:02,118Z DEBUG Setting state of <ImagingStepInitCVM(<NodeConfig(192.168.1.22) @e2d0>) @e7d0> from RUNNING to FINISHED2020-12-10 12:41:02,122Z INFO Completed <ImagingStepInitCVM(<NodeConfig(192.168.1.22) @e2d0>) @e7d0>2020-12-10 12:41:03,493Z INFO Using C:\Program Files (x86)\Nutanix\Portable Foundation\lib/foundation-platforms\aurora-1.4.0-7954202.tar.gz2020-12-10 12:41:03,493Z DEBUG Cache HIT: key(<function get_md5sum at 0x0370DFB0>_('C:\\Program Files (x86)\\Nutanix\\Portable Foundation\\lib\\foundation-platforms\\aurora-1.4.0-7954202.tar.gz',)_{})2020-12-10 12:41:03,496Z INFO Using C:\Program Files (x86)\Nutanix\Portable Foundation\lib/foundation-platforms\tartarus-1.0.1-39d983b3.tar.gz2020-12-10 12:41:03,496Z DEBUG Cache HIT: key(<function get_md5sum at 0x0370DFB0>_('C:\\Program Files (x86)\\Nutanix\\Portable Foundation\\lib\\foundation-platforms\\tartarus-1.0.1-39d983b3.tar.gz',)_{})2020-12-10 12:41:03,497Z INFO Using C:\Program Files (x86)\Nutanix\Portable Foundation\tmp\sessions\20201210-163836-3\files\features_node_192.168.1.22.json2020-12-10 12:41:03,499Z DEBUG Cache MISS: key(<function get_md5sum at 0x0370DFB0>_('C:\\Program Files (x86)\\Nutanix\\Portable Foundation\\tmp\\sessions\\20201210-163836-3\\files\\features_node_192.168.1.22.json',)_{})2020-12-10 12:41:03,500Z INFO Using C:\Program Files (x86)\Nutanix\Portable Foundation\lib/foundation-platforms/hcl.json2020-12-10 12:41:03,500Z DEBUG Cache HIT: key(<function get_md5sum at 0x0370DFB0>_('C:\\Program Files (x86)\\Nutanix\\Portable Foundation\\lib\\foundation-platforms\\hcl.json',)_{})2020-12-10 12:41:03,502Z INFO Using c:\program files (x86)\nutanix\portable foundation\lib\foundation-platforms\foundation_layout-2.0+r.1297.728c9b5-py2.7.egg2020-12-10 12:41:03,503Z DEBUG Cache HIT: key(<function get_md5sum at 0x0370DFB0>_('c:\\program files (x86)\\nutanix\\portable foundation\\lib\\foundation-platforms\\foundation_layout-2.0+r.1297.728c9b5-py2.7.egg',)_{})2020-12-10 12:41:03,505Z DEBUG Generating updates config json C:\Program Files (x86)\Nutanix\Portable Foundation\tmp\sessions\20201210-163836-3\files\updates_config_node_192.168.1.22.json2020-12-10 12:41:03,509Z DEBUG Cache MISS: key(<function get_md5sum at 0x0370DFB0>_('C:\\Program Files (x86)\\Nutanix\\Portable Foundation\\tmp\\sessions\\20201210-163836-3\\files\\updates_config_node_192.168.1.22.json',)_{})2020-12-10 12:41:03,517Z DEBUG Cache HIT: key(<function get_md5sum at 0x0370DFB0>_('C:\\Program Files (x86)\\Nutanix\\Portable Foundation\\lib\\phoenix\\x86_64\\squashfs.img',)_{})2020-12-10 12:41:03,523Z DEBUG Cache HIT: key(<function get_md5sum at 0x0370DFB0>_('C:\\Program Files (x86)\\Nutanix\\Portable Foundation\\lib\\phoenix\\x86_64\\squashfs.img',)_{})2020-12-10 12:41:03,528Z DEBUG using C:\Program Files (x86)\Nutanix\Portable Foundation\lib\bin\cygwin\mkisofs.exe for mkisofs2020-12-10 12:41:03,815Z INFO Setting boot mode2020-12-10 12:41:03,818Z DEBUG No implementation found for setting boot mode to BootMode.LEGACY. Skipping2020-12-10 12:41:03,819Z INFO Powering off node2020-12-10 12:41:03,825Z ERROR Could not execute '<obfuscated_command>'Traceback (most recent call last):  File "bmc_utils\tools.py", line 78, in system  File "site-packages\subprocess32.py", line 617, in __init__  File "site-packages\subprocess32.py", line 922, in _execute_childWindowsError: rError 2] The system cannot find the file specified2020-12-10 12:41:03,826Z ERROR Command '<obfuscated_command>' returned error code -1stdout:stderr:2020-12-10 12:41:08,852Z ERROR Could not execute '<obfuscated_command>'Traceback (most recent call last):  File "bmc_utils\tools.py", line 78, in system  File "site-packages\subprocess32.py", line 617, in __init__  File "site-packages\subprocess32.py", line 922, in _execute_childWindowsError: rError 2] The system cannot find the file specified2020-12-10 12:41:08,854Z ERROR Command '<obfuscated_command>' returned error code -1stdout:stderr:2020-12-10 12:41:13,868Z ERROR Could not execute '<obfuscated_command>'Traceback (most recent call last):  File "bmc_utils\tools.py", line 78, in system  File "site-packages\subprocess32.py", line 617, in __init__  File "site-packages\subprocess32.py", line 922, in _execute_childWindowsError: rError 2] The system cannot find the file specified2020-12-10 12:41:13,868Z ERROR Command '<obfuscated_command>' returned error code -1stdout:stderr:2020-12-10 12:41:18,877Z ERROR Could not execute '<obfuscated_command>'Traceback (most recent call last):  File "bmc_utils\tools.py", line 78, in system  File "site-packages\subprocess32.py", line 617, in __init__  File "site-packages\subprocess32.py", line 922, in _execute_childWindowsError: rError 2] The system cannot find the file specified2020-12-10 12:41:18,878Z ERROR Command '<obfuscated_command>' returned error code -1stdout:stderr:2020-12-10 12:41:23,894Z ERROR Could not execute '<obfuscated_command>'Traceback (most recent call last):  File "bmc_utils\tools.py", line 78, in system  File "site-packages\subprocess32.py", line 617, in __init__  File "site-packages\subprocess32.py", line 922, in _execute_childWindowsError: rError 2] The system cannot find the file specified2020-12-10 12:41:23,894Z ERROR Command '<obfuscated_command>' returned error code -1stdout:stderr:2020-12-10 12:41:28,908Z ERROR Could not execute '<obfuscated_command>'Traceback (most recent call last):  File "bmc_utils\tools.py", line 78, in system  File "site-packages\subprocess32.py", line 617, in __init__  File "site-packages\subprocess32.py", line 922, in _execute_childWindowsError: rError 2] The system cannot find the file specified2020-12-10 12:41:28,911Z ERROR Command '<obfuscated_command>' returned error code -1stdout:stderr:2020-12-10 12:41:28,914Z ERROR Could not execute '<obfuscated_command>'Traceback (most recent call last):  File "bmc_utils\tools.py", line 78, in system  File "site-packages\subprocess32.py", line 617, in __init__  File "site-packages\subprocess32.py", line 922, in _execute_childWindowsError: rError 2] The system cannot find the file specified2020-12-10 12:41:28,914Z ERROR Command '<obfuscated_command>' returned error code -1stdout:stderr:2020-12-10 12:41:28,915Z ERROR Exception in <ImagingStepInitIPMI(<NodeConfig(192.168.1.22) @e2d0>) @4d30>Traceback (most recent call last):  File "foundation\decorators.py", line 77, in wrap_method  File "foundation\imaging_step_init_ipmi.py", line 269, in run  File "foundation\imaging_step_init_ipmi.py", line 110, in boot_phoenix  File "bmc_utils\_remote_boot_ipmi.py", line 54, in poweroff  File "bmc_utils\tools.py", line 178, in ipmitool_with_retryStandardError: Command '<obfuscated_command>' returned error code -1stdout:stderr:2020-12-10 12:41:28,948Z ERROR Exception in running <ImagingStepInitIPMI(<NodeConfig(192.168.1.22) @e2d0>) @4d30>Traceback (most recent call last):  File "foundation\imaging_step.py", line 161, in _run  File "foundation\decorators.py", line 77, in wrap_method  File "foundation\imaging_step_init_ipmi.py", line 269, in run  File "foundation\imaging_step_init_ipmi.py", line 110, in boot_phoenix  File "bmc_utils\_remote_boot_ipmi.py", line 54, in poweroff  File "bmc_utils\tools.py", line 178, in ipmitool_with_retryStandardError: Command '<obfuscated_command>' returned error code -1stdout:stderr:2020-12-10 12:41:28,950Z DEBUG Setting state of <ImagingStepInitIPMI(<NodeConfig(192.168.1.22) @e2d0>) @4d30> from RUNNING to FAILED2020-12-10 12:41:28,953Z DEBUG Setting state of <ImagingStepRAIDCheckPhoenix(<NodeConfig(192.168.1.22) @e2d0>) @4bd0> from PENDING to NR2020-12-10 12:41:28,954Z WARNING Skipping <ImagingStepRAIDCheckPhoenix(<NodeConfig(192.168.1.22) @e2d0>) @4bd0> because dependencies not met, failed tasks: s<ImagingStepInitIPMI(<NodeConfig(192.168.1.22) @e2d0>) @4d30>]2020-12-10 12:41:28,957Z DEBUG Setting state of <ImagingStepPreInstall(<NodeConfig(192.168.1.22) @e2d0>) @4a10> from PENDING to NR2020-12-10 12:41:28,959Z WARNING Skipping <ImagingStepPreInstall(<NodeConfig(192.168.1.22) @e2d0>) @4a10> because dependencies not met2020-12-10 12:41:28,963Z DEBUG Setting state of <InstallHypervisorKVM(<NodeConfig(192.168.1.22) @e2d0>) @0590> from PENDING to NR2020-12-10 12:41:28,963Z DEBUG Setting state of <ImagingStepPhoenix(<NodeConfig(192.168.1.22) @e2d0>) @5b70> from PENDING to NR2020-12-10 12:41:28,963Z WARNING Skipping <InstallHypervisorKVM(<NodeConfig(192.168.1.22) @e2d0>) @0590> because dependencies not met2020-12-10 12:41:28,964Z WARNING Skipping <ImagingStepPhoenix(<NodeConfig(192.168.1.22) @e2d0>) @5b70> because dependencies not met

Where are u running Foundation? Portable on Windows machine it seams. Try downloading Foundation for Virtual Box, install the foundation Appliance on Virtual box and try again, i spent six hours back in october with troubbleshooting Portable Foundation for windows. Downloaded VirtualBox and ran foundation from that Linux Appliance,

 

worked first try after changeing to that approach! 
 

Look at this KB how to set up VirtualBox 

https://portal.nutanix.com/page/documents/details?targetId=Field-Installation-Guide-v4-0%3Av40-cluster-environment-foundation-t.html

I second Jonas on this.

 

Portable foundation does not have support for Lenovo server.

Source: Field Installation Guide v4.6: Foundation Use Case Matrix

Always use the guide for the version you are using Field Installation Guide v4.6: Prepare Bare-Metal Nodes for Imaging

Let us know how you go.

@Farhad978 how did things go with Foundation VM on VirtualBox?

Hope you got it working :)

Now i am installed foundation 4.6 on esxi. when it gets booting installer media. it gives error The target is not in ssh able or not in the phoenix. What is the phoenix ?

Now i am installed foundation 4.6 on esxi. when it gets booting installer media. it gives error The target is not in ssh able or not in the phoenix. What is the phoenix ?

Phoenix is the os on the node when it ships from factory! I have none exprience with Lenovo nodes, but with NX nutanix nodes you have DiscoveryOS on the nodes from factory.

 

Does the Foundation VM have Layer2 access to the network where the NICs on the nodes are connected?

Does Foundation discover the nodes automatically? If yes, try adding them manually, with serial and IPMI-IP.

Also another question: What subnet do you intend to configure your CVMs with?
 

I need the following info to get a better picture:

 

  1. Foundation VM-IP and netmask
  2. IPMI-IPs and Netmask
  3. intended CVM IP and netmask

Thanks in Advance!

 

  1. Foundation vm ip is 192.168.1.5 netmask: 255.255.255.0
  2. 4 Lenovo hx3320 nodes  ipmi ip: 192.168.1.13 -16 netmask: 255.255.255.0
  3. CVM ip: 192.168.1.21-24 netmask: 255.255.255.0

 

Foundation not discovering automatically  nodes. I manually added it with ipmi ip adresses.

 

From Foundation installed OS (redhat) i can ping all of these ipmi ip adresses.

 

Sorry for my english. 

Okay!

sounds like you have a decent setup!

and you can browse the IPMI-s from the Foundation VM and you’re setting the correct passwords for the IPMI interfaces in the Foundation GuI.

 

i would still like you to test out VirtualBox with foundation on your laptop with a direct cable in to the same VLAN as the new nodes and to try running from there instead of an esxi appliance thats because the networking seams to be in order what i can se.

 

let me know how it goes!

Hi Farhad978,

just to be sure, are you using 10G data ports for the foundation process?

As far as i know Lenovo’s nodes need that kind of connection to be safely founded.

Please let me know.

Have a nice day

The servers are connected to each other using 10 gigabit DAC cables. Foundation 4.6 itself is installed on a laptop on a virtual box. and connects to servers using 1 gigabit ethernet cable

And just to be sure, you’re connecting the 1g cable and the 10g cables to the same switch? 
 

Where are the IPMIs connected?

 

Can you draw a network diagram

of how you connected everything?

Would help us a lot i guess!

The correct cabling should be something like that

4 x 10G to nodes

4 X 1G to XCCs

1 x 1G Uplink cable

1 x 1G Link to your notebook

 

 

The correct cabling should be something like that

4 x 10G to nodes

4 X 1G to XCCs

1 x 1G Uplink cable

1 x 1G Link to your notebook

 

Nice UPX :)

 

:) Thank you, 

I’ve to say, during the last quarter of the year i’ve been busy doing a lot of docs and graph for customers so i’m used to and i did that in just few minutes 

:) Thank you, 

I’ve to say, during the last quarter of the year i’ve been busy doing a lot of docs and graph for customers so i’m used to and i did that in just few minutes 


Above and beyond! Nice :)

@Farhad978 any update on the Foundation process? Did you have any luck? Take some time to review the network diagram from UPX above. 
 

You can allways log a support case to get a SRE look at your issue as well, let us know how things are going. 

Here is the Diagramm

 

Just to make sure, can you try to move the Laptop to the 10gb switch using 1G SFP,

Also from the switches, if you run ”show mac address vlan xx) do you se the Macs of all of the nics nd IPMIS?

@Farhad978 how did things end up with the foundation for your Lenovo Clusteri?

Reply


Terms & Conditions