Struggling with Foundation

Badge +4
Good morning.

I'm in the process of doing a green field installation at a customer, and unexpectingly I ran into problems.
We first staged the nodes with KVM in our lab, and than shipped it off to our customer. We had some issues, as the cluster was not detected on IPv6, combined with some other strange errors, and I decided to scratch everything and rerun foundation (with per documentation you can do after a cluster destroy and scratch everything)
Foundation could not detected any nodes, so I resorted to entering MAC addresses of the IPMI interfaces and adding the nodes. This worked as far as that IPMI got configured, I got the point of the pre-phoenix reboot, and than it stops with an unble to connect.. (so after a progress of about 18% foundation stops..)
I do see some time out messages in the logs.
At the end of day, I followed the last advice in the field deployment guide, where it states "if you're unable to deploy through foundation" you can deploy manually through IPMI. I tried that one also.
Using the process deployment seems to go will, up the point where it states "please unplug all virtual or physical media and press y to reboot". The node should than reboot, which it does, and should afterwards reboot once again (after 15-30mins), which it doesnt.

Looking in the firstboot logs I see that the script doesnt complete and stops with this error
NFO: Running cmd ['virsh net-destroy NTNX-Local-Network']
INFO: Running cmd ['virsh net-undefine NTNX-Local-Network']
INFO: Running cmd ['virsh net-define /root/net-NTNX-Local-Network.xml']
INFO: Running cmd ['virsh net-start NTNX-Local-Network']
FATAL: Execution of cmd [['virsh net-start NTNX-Local-Network']] failed for rea
son [error: Failed to start network NTNX-Local-Network

We had IPMI and Gigabit adapters connected. Everything is connected on the same switch, same VLan, the VM running Foundation is also connected on the same switch (you can call it a flat switch)
I didn't notice that STP was active, which I saw at the end of day could give problems;
Was I stuped? (yes, I broke everything :S) and are all issues related to too many connected cables, or am I missing something else?
(this is the first time I deploy nutanix with KVM, with VMware it seems a bit less tricky, but that may be a feeling)
Any opinions will be hugely appreciated
Thank you very much

6 replies

Userlevel 3
Badge +14
It is best to have STP disabled, however we can check what errors did you actually get when you ran the foundation the first time.

Also, what version was the Foundation, Phoenix and the NOS?

With just boot from Phoenix, at what stage does the installation fail?
Userlevel 1
Badge +10
HI Mario,

i struggled with nutanix cluster imaging with foundation initially as you mentioned.then i opened a ticket and the second case owner was able to resolve the issue. should use the most simple switch for the imaging.simple makes things easy.

2.try to image the cluster using a simple switch before putting nutanix in the customer network since STP may be enabled in their network .
3.from latest nos i think 4.1 phoenix iso is not used for the imaging rather nos image is used for the imaging of all nodes to cluster and if you want to add a single node then you should you phoenix iso.i tried with phoenix iso image for imaging cluster rather than nos image which was the main problem.

4.try to stop the antivirus in your pc before doing the imaging since some time it can show you error like ipmi interface not found

5.only connect 1gb interface into the switch.dont tried to connect all interface.

6.always use the mac address of ipmi and enter it in the foundation web page since ipv6 is not enabled in most of the networks.

think these suggestions will help you and others to sort out problems with installation.

 please comment if i mentioned any thing wrong so that it wont mislead to others who read this post
Userlevel 4
Badge +18
you have covered most of the points which can make foundation imaging process successful. :)

Thanks for the contribution in the community.
Userlevel 1
Badge +10
@donniethanks man...
Badge +1
Thanks for the solution!

Nutanix Support, where can i find the "Field Installation Guide" for download?

Userlevel 1
Badge +9
Field Install Guide can be found under Docs,
Change the "Filter" to show docs related to Foundation.