Nutanix Kubernetes Volume Plugin For On-Demand Choice

  • 12 March 2018
  • 1 reply
Nutanix Kubernetes Volume Plugin For On-Demand Choice
Userlevel 7
Badge +35
This post was authored by Dwayne Lessner, Sr Technical Marketing Engineer at Nutanix

Nutanix has had a Docker Volume Plugin for a long time and now is adding the option to use a Kubernetes Volume Plugin. The Nutanix Kubernetes plugin gives more options to users that may want to use Kubernetes as their orchestrator of choice. Like Docker Inc, we have realized that even within the same companies' different departments and even individual developers don't want to be tied to anyone solution.

Nutanix supports the Kubernetes PersistentVolume for both static and dynamic provisioning. From the Kubernetes documentation:

"A PersistentVolume (PV) is a piece of storage in the cluster that has been provisioned by an administrator. It is a resource in the cluster just like a node is a cluster resource. PVs are volume plugins like Volumes, but have a lifecycle independent of any individual pod that uses the PV. This API object captures the details of the implementation of the storage, be that NFS, iSCSI, or a cloud-provider-specific storage system.

A PersistentVolumeClaim (PVC) is a request for storage by a user. It is similar to a pod. Pods consume node resources and PVCs consume PV resources. Pods can request specific levels of resources (CPU and Memory). Claims can request specific size and access modes (e.g., can be mounted once read/write or many times read-only)."

The Nutanix Kubernetes Volume Plugin uses iSCSI based storage from Nutanix ABS (Acropolis Block Storage). With the upcoming support of NFS v4 for Acropolis File Services (AFS ) will enable even more options. With advanced intelligence used for data placement Nutanix Enterprise Cloud OS can run on all flash nodes, hybrid nodes, or mixed clusters all flash and hybrid nodes. You can even add storage only nodes to accommodate a variety of needs.

As an example if you have 8 All Flash Nutanix nodes running VMs and Containers but you also might have 4 storage only hybrid nodes for extra capacity, Nutanix ABS can utilize the storage controllers from all of the nodes including the storage only nodes if other resources are busier.

To test out the new plugin, I used the latest build of Docker Datacenter that was running a beta of their upcoming support of both Swarm and Kubernetes. I had 4 worker nodes with a valid IP, and made sure every node had the same iSCSI initiator name set. You can set the iSCSI initiator name in /etc/iscsi/initiatorname.iscsi, the default name for the plugin is

I decided to run some of the commands directory in the manager without the need to setup kubectl on my local client. Below is a picture of starting to setup the service account inside of Docker Datacenter (DDC). What I like about this method is that security is enforced from the login into DDC and you don't have another entry point out in the wild.

I thought the most interesting thing about the Nutanix plugin was the ability for the developers to select what level of service that want for their persistent storage. When you setup the plugin you have to create your storage class. The plug-in provides two types of storage classes: gold and silver. If you're running on a hybrid cluster (both SSD's and HDD's) you can select the gold tier and it will automatically pin the backing vdisk to the SSD tier. Under metadata in the name field all you have to do is select what level of service you want, gold or silver.

Below is how you would create the StorageClass on Nutanix.

kind: StorageClass
name: class_type provisioner:
nutanix/abs parameters:
prismEndPoint: prism_ip_address:port
dataServiceEndPoint: data_service_ip_address:port
user: user_name
password: password
secretName: kubernetes_secret_name
storageContainer: storage_container
fsType: file_system_type
chapAuthEnabled: "true_or_false"
iscsiSecretName: iscsi_secret_name
defaultIqn: kubernetes_worker_node_iqn

I was testing on an all flash system, so there wasn't any point to pin my storage to SSD. I selected the silver storage class. To use the storage class, you have to create and use a Persistent Volume Claim. When you create a claim with dynamic persistent volume definitions, the plug-in automatically provisions the volume. Persistent Volume Claim (PVC) APIs are used to request storage. By using PVC, you can specify the size and access mode for storage. When the pod is deleted, the volume is considered released and deleted. You can change the default behavior by changing the ReclaimPolicy to Retain.

The picture above is from Prism. Under the volume group details you can see that flash mode is disabled. If the policy was Gold you would see this as enabled.

So with Nutanix and Docker today you can try out running both Swarm and Kubernetes on the same cluster giving choice to the ultimate choice to the developer.

Full documentation for the Nutanix Kubernetes volume plugin can be found on

Forward-Looking Statements Disclaimer
This blog includes forward-looking statements, including but not limited to statements concerning our plans and expectations relating to product features and technology that are under development or in process and capabilities of such product features and technology. These forward-looking statements are not historical facts, and instead are based on our current expectations, estimates, opinions and beliefs. The accuracy of such forward-looking statements depends upon future events, and involves risks, uncertainties and other factors beyond our control that may cause these statements to be inaccurate and cause our actual results, performance or achievements to differ materially and adversely from those anticipated or implied by such statements, including, among others: failure to develop, or unexpected difficulties or delays in developing, new product features or technology on a timely or cost-effective basis; delays in or lack of customer or market acceptance of our new product features or technology; the failure of our software to interoperate on different hardware platforms; failure to form, or delays in the formation of, new strategic partnerships and the possibility that we may not receive anticipated results from forming such strategic partnerships; the introduction, or acceleration of adoption of, competing solutions, including public cloud infrastructure; a shift in industry or competitive dynamics or customer demand; and other risks detailed in our Form 10-Q for the fiscal quarter ended October 31, 2017, filed with the Securities and Exchange Commission. These forward-looking statements speak only as of the date of this presentation and, except as required by law, we assume no obligation to update forward-looking statements to reflect actual results or subsequent events or circumstances.

This blog may contain links to external websites that are not part of Nutanix does not control these sites and disclaims all responsibility for the content or accuracy of any external site. Our decision to link to an external site should not be considered an endorsement of any content on such site.

© 2018 Nutanix, Inc. All rights reserved. Nutanix and the Nutanix logo are registered trademarks or trademarks of Nutanix, Inc. in the United States and other countries. All other brand names mentioned herein are for identification purposes only and may be the trademarks of their respective holder(s).

This topic has been closed for comments

1 reply