Recently i have changed my CVM IP address following Manual Method in Documentation 4.6 , i can start the cluster after changed the IP , but the zookeeper seem like not stable.
Prism show error "Desired fault tolerance for Zookeeper is 1 but we can tolerate only 0 node failure(s)" on the Data Resiliency Status dashboard.
NOS 4.6 , ESXI 6.0
When i grep the zookeeper PIDs , it only show 5 PIDs instead of 6 , and sometimes it will be change to 4 PIDs.
Command:
nutanix@NTNX-14SM15430005-C-CVM:10.20.20.118:~$ allssh genesis status | grep zookeeper zookeeper: [10354, 10367, 12412, 12438, 12439] zookeeper: [12250, 12263, 15454, 15506, 15507] zookeeper: [19055, 19069, 21163, 21189, 21190]
nutanix@NTNX-14SM15430005-C-CVM:10.20.20.118:~$ allssh genesis status | grep zookeeper zookeeper: [12412, 12438, 12439, 26481, 26494] zookeeper: [15454, 15506, 15507, 27119] zookeeper: [21163, 21189, 21190, 32528, 32542]
Anyone has any idea on this ?
Thanks.
Best answer by mgauch
Looks like you may have missed a step in the instructions. Please double check your zk_server_config_file (I didn't see genesis in there it still shows zookeeper_monitor):
==================================
Open the zookeeper configuration file.nutanix@cvm$ vi /home/nutanix/data/zookeeper_monitor/zk_server_config_filePress A to edit values in the file.Update the fields in the file.Change # LAST MODIFIER: field_entry to # LAST MODIFIER: GENESISIncrement the existing number by one in the # ZOOKEEPER CONFIG VERSION field. For example, you would change # ZOOKEEPER CONFIG VERSION 3 to # ZOOKEEPER CONFIG VERSION 4Update the entries for each Controller VM (zk1, zk2, zk3, and so on) to match the IP addresses (ip_address) of the Controller VMs.# LAST MODIFIER: GENESIS# ZOOKEEPER CONFIG VERSION existing_number+1ip_address zk2 # DON'T TOUCH THIS LINEip_address zk1 # DON'T TOUCH THIS LINEip_address zk3 # DON'T TOUCH THIS LINEip_address zkN # DON'T TOUCH THIS LINE
==================================
Link: https://portal.nutanix.com/#/page/docs/details?targetId=Advanced_Admin-Acr_v4_6:ip__cvm_ip_addr_change_t.html
View original
==================================
Open the zookeeper configuration file.nutanix@cvm$ vi /home/nutanix/data/zookeeper_monitor/zk_server_config_filePress A to edit values in the file.Update the fields in the file.Change # LAST MODIFIER: field_entry to # LAST MODIFIER: GENESISIncrement the existing number by one in the # ZOOKEEPER CONFIG VERSION field. For example, you would change # ZOOKEEPER CONFIG VERSION 3 to # ZOOKEEPER CONFIG VERSION 4Update the entries for each Controller VM (zk1, zk2, zk3, and so on) to match the IP addresses (ip_address) of the Controller VMs.# LAST MODIFIER: GENESIS# ZOOKEEPER CONFIG VERSION existing_number+1ip_address zk2 # DON'T TOUCH THIS LINEip_address zk1 # DON'T TOUCH THIS LINEip_address zk3 # DON'T TOUCH THIS LINEip_address zkN # DON'T TOUCH THIS LINE
==================================
Link: https://portal.nutanix.com/#/page/docs/details?targetId=Advanced_Admin-Acr_v4_6:ip__cvm_ip_addr_change_t.html