Cluster creation or initialization is a process of bootstrapping the cluster by configuring the unconfigured nodes, load some node information into Zeus configuration file and start the services. Let's break it down.
What is an Unconfigured node?
-
Node which are factory shipped.
-
Node which was removed from an existing cluster.
These nodes are typically Pre-installed with CVM/Hypervisor
NO IPv4 address is configured But IPv6 link local address configured on eth0. This will always remain on a host.
Before you begin to create a cluster with your brand-new nodes,
You Must Have
-
IPv4 Address configuration
-
IPMI IP address
-
Hypervisor IP address
-
CVM IP address
-
DNS/NTP IP address (Required while creating cluster via foundation)
-
-
Hypervisor and CVM should be installed.
What are the methods of Cluster initialization?
-
Foundation:
-
One click process for cluster creation
-
Re-Images multiple nodes
-
Assigns IP address on each node
-
-
Manual
-
Manual Hypervisor installation
-
CVM installation using Phoenix
-
Configuring IP address
-
Some prechecks listed below but not an exhaustive list:
-
Validate the IP addresses configured.
-
Checks if the required number of IPs are provided for cluster creation.
-
Validate the AOS version installed on each node.
-
-
Test for cluster homogeneity
-
SX and NX nodes cannot be in same cluster.
-
Check SX cluster size
-
Validates that G4 and G5 nodes are not on the same block.
-
-
Hyper-V specific check – Hyper-V cluster Cannot be created with mixed version of Hyper-V
-
NX nodes and other hardware vendor nodes cannot be in same cluster.
-
All the nodes must have same hypervisor – this restriction is not applicable if Storage nodes are
Cluster Create:
-
Assigning IP addresses process is part of Foundation.
-
When manually creating cluster, this process does not happen.
-
IP addresses to be assigned for CVM, Host and IPMI are provided in Foundation GUI which will be saved in a JSON file.
-
These IP addresses are validated and checked to confirm that they do not overlap with reserved subnet, After which IP addresses are configured on the CVM, Host and IPMI
Node configuration :
-
Zookeeper mapping are updated based on RF in /etc/hosts file
-
Once IP address and ZK mapping is complete, zookeeper service is started.
-
Foundation service is stopped.
-
Initial cluster configuration JSON file is created and is used by zookeeper to update cluster config.
Cluster configuration :
-
Zookeeper will initialize cluster with information to update the configuration proto
This will take the following parameters as input:
-
Internal and External subnet
-
Cluster ID and UUID, cluster name
-
Node Info (node_list) - IP addresses of CVM, Host, IPMI, node UUID, node position,
-
software version, node serial number etc
-
If the CVM is a zk node then “myid” will also be added
-
Update cluster fault tolerance state.
-
Create and add default storage pool and container
At this point Cluster is ready to be started
Cluster start :
-
Cluster manager will mark zookeeper that the cluster needs to be started
-
Other services will read this and will be started on the node.
-
The order depends on the AOS version and other dependencies between components/services.
Creating a cluster :
Getting started with Nutanix Community Edition:
Filed installation guide: