Special thanks: Subodh Mathur (Engineering) and Denis Guyadeen (PM) for reviewing and contributing to this article. Special thanks to Christophe Jauffret (DevOps SME) for the Calm and Helm demo.
When Nutanix was founded in 2009, we set off to address a major problem in virtualization: storage. At the time, most performance problems revolved around storage contention. Whether the issue was complexity that led to poor configurations or storage controller resources being exhausted, storage was a mess to fix, scale, and plan for. To meet the goal of customer delight, Nutanix addressed these problems with its own distributed file system: Acropolis Distributed Storage Fabric (ADSF). Developers need more agile, policy-driven, and software-based storage for containers.
Making storage the bedrock for the Nutanix solution allowed other challenges to be solved. Nutanix tackled virtualization with the release of AHV (a hardened-out-of-the-box hypervisor), automation with Calm, cost controls and security governance for multicloud deployments with Beam, and observability with Epoch.
The Nutanix solution gives IT Operations & Developers teams more time to focus on the applications that drive their businesses instead of building and maintaining infrastructure. Because big data workloads and persistent workloads like MS SQL, SAP, Oracle, and Splunk are commonplace on Nutanix, we have watched our customers across the industry start to adopt and operationalize containerization to accelerate development, testing, and application lifecycle management.
Most organizations typically run stateless workloads because they are much easier to set up, manage, and scale, but we continue to see a lot of demand for stateful workloads that need to persist when the container or pod dies or is deleted.
● SQL Databases | MySQL, PostgreSQL, SQL Server
● NoSQL databases | MongoDB, ElasticSearch, Redis, Cassandra
● Pub/Sub systems | Kafka
● Time series databases | InfluxDB, Graphite
● Big data | Splunk
● Block | iSCSI, FC
● Object | S3, Swift
● File | NFS, SMB
To keep up with customer demand, Nutanix supports the Docker volume plugin and Kubernetes External Provisioner, and now we are proud to announce our Container Storage Interface (CSI) driver for Kubernetes. Before we dive into CSI, we would like to share what got us here in the first place. Persistent storage support was introduced in Kubernetes 1.1 with (In-Tree) Volume Plugins, PersistentVolumes (PV), and PersistentVolumeClaims (PVC).
The phrase “in-tree plugins” refers to the set of included provisioners while giving the option for creating out-of-tree provisioners that would target alternative back ends. In-Tree Volume Plugins are linked, compiled, built, and shipped with the core Kubernetes code base, so development is tightly coupled with and dependent on Kubernetes releases. This means that bugs in a volume plugin can crash critical Kubernetes components, instead of just the volume plugin. For this reason, in-tree provisioners are not going to be accepted in the future.
Out-of-Tree Volume Plugins (plugins customized by storage vendors), FlexVolume drivers (introduced in Kubernetes 1.2), and CSI drivers (introduced in Kubernetes 1.9). For more details see the Stateful Workload Support Timeline in Kubernetes below.
What Is CSI?
Before we dive in, let me describe some of the abbreviations (which are also part of the CSI specification):
- CO: Container orchestration solution. These are systems such as Kubernetes, Docker Swarm, or Apache Mesos.
- Node: A host where a workload (such as a Pod in Kubernetes) is running.
- Plugin: A service that exposes gRPC endpoints.
Various community members from Kubernetes, Mesosphere, and Docker worked on this specification together. This new interface is a major benefit to the container ecosystem because it standardizes the model for integrating storage systems across container orchestrators. Specifically, it frees the storage system driver from the Kubernetes release schedule because the driver is no longer incorporated in the same code base (in-tree). With CSI, you can install storage system drivers independently of the container orchestration releases, providing faster bug fixes and features while reducing code sprawl.
With all of the major orchestration engines working toward supporting CSI, DevOps teams only have to worry about one configuration for their environments, which helps keep everything portable regardless of where you're running your compute. This new plugin model further eases development and provides peace of mind for users running persistent workloads on Nutanix, both in terms of performance and availability.
The CSI Driver for Kubernetes leverages Nutanix Volumes (formerly known as ABS) to provide scalable and persistent storage for stateful applications. More information on Nutanix Volumes can be found here.
How to Set Up the CSI Plugin
We follow the deployment model for the CSI volume driver recommended by Kubernetes. Kubernetes provides helper containers (external-attacher, external-provisioner, and driver-registrar) that help the CSI volume driver interact with the Kubernetes system.
We deploy (1) a StatefulSet containing a csi-provisioner with a volume driver ntnx-csi that performs create and delete volume operations, (2) a StatefulSet containing a csi-attacher with a volume driver ntnx-csi that is a no-op for the driver, and (3) a DaemonSet containing a driver-registrar with a volume driver that performs volume attach and detach and mount and unmount operations. See here for detailed deployment instructions.
Nutanix Volumes also offers other data services because persistent storage alone isn’t enough: data locality, erasure coding, deduplication, compression, snapshots, and replication.
1. [Video] Kubernetes Storage Lingo 101
2. CSI Specifications
3. [Blog] Container Storage Interface (CSI) for Kubernetes Goes Beta
4. Nutanix CSI Driver - Helm Chart
5. CNCF Expands Kubernetes Storage Options
Continue the conversation in our community forums on Containers
Disclaimer: This blog may contain links to external websites that are not part of Nutanix.com. Nutanix does not control these sites and disclaims all responsibility for the content or accuracy of any external site. Our decision to link to an external site should not be considered an endorsement of any content on such site.
© 2018 Nutanix, Inc. All rights reserved. Nutanix, the Nutanix logo and the other Nutanix products and features mentioned herein are registered trademarks or trademarks of Nutanix, Inc. in the United States and other countries. All other brand names mentioned herein are for identification purposes only and may be the trademarks of their respective holder(s).