Shutdown cluster via script


Badge +5
Hey all,

Our site has limited power availability for when the lights go out - enough to shutdown everything manually at the least. However there isnt always someone proficient onsite to be able to shutdown the infrastructure.

Ive been writing a powershell script that gets called by our APC virtual appliance - the script shutsdown all the VMs in the right order etc, however I have yet to be able to shutdown the cluster scripted.

Running a 'cluster stop' has a confirmation prompt that my script cannot handle - as the powershell command doesnt run a session, but just a single command.

The method im using is as follows:
$result = Invoke-SshCommand -ComputerName $cvmIPs[0] -Command '/home/nutanix/cluster/bin/cluster status | grep state'
etc...

Has anyone a script for shutting down the whole cluster? I'd think sending all the CVMs a 'sudo shutdown -h 0' will certainly shut them down, but id be a little worried about the state of the cluster when they are started again.

Thanks

- Scott

9 replies

Userlevel 7
Badge +30
I think there is a --force option on the cluster command, just for this type of reason.

Once you are done with your script, post it up here, or to the github.com/nutanix repo and we'd be happy to review it.
Badge +5
Unfortunatly as this is a system in production i cannot just give that a go.
I tried 'cluster stop --help' and the listed flags didnt include a --force.
I have at least 8 weeks to wait before i can try typing 'cluster stop --force' - unless the power goes out prior...
Hopefully someone has experience with this.
Userlevel 7
Badge +30
I pinged a few folks internally, I'm pretty sure there's a force option, but I'll double check
Userlevel 7
Badge +30
Thanks to Fermin from support for digging this up

From /usr/local/nutanix/cluster/bin/cluster : # Check if the command is excuted from a terminal. Piping for destructive # operations, eg: "$ echo y | cluster destroy" is not allowed. The stop cmd # allows piping for automation purpose as it is not destructive. if cmd != "stop" and not sys.stdin.isatty(): log.ERROR("Destructive operations like %s should get the confirmation " "(Y/[n]) from terminal. Seems like the confirmation is piped " "or redirected from another process into this command." % cmd) sys.exit(1)

So, give this a shot:
echo Y | cluster stop
Badge +5
Awesome - ah linux voodoo.
I'll give it a go in my script (its a modification of Josh Sinclair's from here: https://joshsinclair.com/?p=290) along with some extra scripting to do VM shutdowns from vCenter.
I'll post it if it works - i get to try in about 6 weeks.
Badge +2
I realize this is an old post, but as much for my knowledge as anything else, is the APC PowerChute Network Shutdown for virtual clusters an option in your stated situation? http://www.apc.com/shop/us/en/categories/power/ups/ups-management/powerchute-network-shutdown/_/N-auzzn7
Badge +5
Hey,

That is exactly what im using. Possibly an older version though.
Got powerchute triggering a batch file, that runs a powershell script that shuts things down in order and then execute the above mentioned ssh commands on the hosts. Seems to work ok - not as consistiant as i'd like - but ive also been onsite for each power outage since.
Badge +1
Hi Jon, I've been struggling with a script that stops the nutanix cluster service. I can run other commands no problem such as cluster status etc. , however, I cannot get "echo Y | cluster stop" to work . I see that others are having this issue too but I havn't seen anyone provide a solution. Do you know of one for this? Thanks !
Badge +1
I am also able to get the cluster status result but when I issue the cluster stop command the cluster does not stop and I don't get any errors.

Has anyone solved this ?
Thanks

Reply