Nutanix File Data Migrations and Data Management with Komprise | Nutanix Community
Skip to main content

Enterprise IT organizations across industries are facing explosive data growth and are struggling to manage data within tight budgets. Nutanix and Komprise help enable companies to do more with less—Nutanix simplifies IT infrastructure and delivers cloud-like economics, and Komprise makes it easy to identify, migrate, and manage the right data on Nutanix. The result is greater efficiency, simpler management, and lower costs.

Key use cases with Komprise and Nutanix covered in this post include:

  • Analytics and Planning: In most enterprises, over 80% of data is cold and not accessed in over a year, yet it continues to consume expensive storage and backup resources. Komprise analyzes across your NAS and cloud data to identify the right data to move to Nutanix and the savings you can expect.
  • File Data Migrations to Nutanix: Migrate file data up to 27 times faster with reliable MD5 checksums on every file. Manage hundreds of migrations with a single console.
  • Transparent Data Archiving/Tiering from Nutanix: Cut ongoing storage and backup costs by finding cold, inactive data on file storage and transparently archiving and file tiering data to Nutanix object storage or the cloud. Archived files continue to be accessed exactly as before so users see no change.
  • Native Access to Tiered Data: Komprise moves data in their native format, so files and objects moved by Komprise can be directly accessed from the destination without needing Komprise or any 3rd party software.

How it Works: Know First, Move Smart, Take Control

The Komprise software deploys in under 15 minutes, works across NFS, SMB/CIFS, and object storage without any storage agents, adapts to file system and network loads to run non-intrusively in the background, and scales out on-demand. Komprise runs as two to three main components:

  • One or more Komprise virtual machines: These run in the same location as your storage and can run on Nutanix AOS or external virtual infrastructure, both in your datacenters and in the cloud.
  • The Komprise Director console: This runs in the cloud.

These virtual machines, close to your storage, analyze your storage and perform data transfers between your storage. They also connect to the Komprise Director to request transfer configurations and to send data for the console UI. The Komprise Director UI console, shown below, enables you to view the analysis of data usage and growth, configure policies to move and manage data, and automate ongoing management across storage.

Figure 1: Adding Nutanix Files as a Source to Komprise is Easy—Simply Provide its IP Address!

Analytics Across File Stores and Planning Your Data Management

Komprise analyzes and moves data from any storage that supports NFS or SMB/CIFS mounts—including Nutanix Files, NetApp, EMC Isilon, Amazon EFS, Amazon FSX for Windows File Server, Azure Files, and Windows File Servers. Komprise profiles the data across your storage and provides analytics to answer questions such as:

  • What types of files do I have?
  • How much of the data is cold vs hot?
  • What is the distribution of file sizes in my data?
  • Who is accessing which files?
  • How fast is my file storage growing?
Figure 2: Komprise Analytics: What File Types are Most Common?
Figure 3: Komprise Analysis: How Many Large and Small Files Are There?
Figure 4: Komprise Analysis: Which Groups are Using the Data?
Figure 5: Komprise Analysis: Which Shares Have the Most Data?

The Komprise data analytics help you determine what data is hot, what data is cold, and enables you to easily search for the right data across your entire infrastructure. It fully complements the analytics Nutanix Files provides to help you detect anomalies or access violations by considering historical data and helping remediating issues like ransomware.

Along with the analytics, Komprise provides interactive ROI projections based on different data management objectives that you can set. As you set policies on what data should be moved, Komprise instantly projects the estimated capacity that will be freed up and your projected cost savings.

You can select what data to migrate into Nutanix, and you can set policies on when cold data is archived or tiered from Nutanix and to where. As you set your policies, Komprise projects your expected cost savings. The storage and backup costs used in the ROI calculator can be customized to your costs. Komprise also projects your 3-year savings based on growth rates of your data.

Figure 6: Komprise Interactively Projects Cost Savings and ROI Based on Your Own Costs

File Data Migrations to Nutanix Using Komprise

Once you have decided what data to migrate to Nutanix, you can simply set up your migrations using the Director console. With Komprise Elastic Data Migration, you can migrate entire shares or directories from any NFS or SMB source to Nutanix, and you can set up and run multiple migrations in parallel. All the data, including the metadata with full access control permissions are migrated over, and Komprise provides a report of MD5 checksums on every file.

Figure 7: Easily Manage Multiple File Migrations to Nutanix

Komprise manages data migrations by running a first iteration, and then as more changes accumulate on the source, incrementally copying these in subsequent iterations. Users can continue to use the migration source until you are ready to perform a cutover. Komprise migrates all the attributes, ACLs, and metadata as well as the file data and computes MD5 checksums on every file to ensure that file integrity is preserved.

Komprise uses an elastic architecture that can scale as needed to migrate data with the speed you require. Read the whitepaper on Komprise Elastic Data Migration to learn how Komprise runs data migrations up to 27 times faster than other approaches.

Data Archiving or Tiering Transparently from Nutanix Using Komprise

When you are ready to realize savings on cold data, you can simply activate your plan so Komprise starts moving data according to your policies. When Komprise archives or tiers data, it does so transparently at the file level—meaning that a cold file is moved entirely to the cloud or secondary storage of your choice, but it looks like the file is still in its original location. This ensures that users and applications can continue to use the file from Nutanix exactly as before, even though the file is no longer consuming expensive backup and DR resources.

Komprise delivers this transparent data archiving on Nutanix using standard protocol constructs such as symbolic links, so no special agents or proprietary interfaces are required. Furthermore, if a file archived by Komprise is accessed by a user or an application, Komprise opens the file without rehydrating the file back into Nutanix or the original storage. This ensures that you fully preserve your space savings from archiving and yet save on costs. Similarly, backups and other processes continue to operate as before, except that they are now backing up the link and not the full file. This ensures that your backup footprint shrinks to eliminate the cold data that is often 80%+. To understand in more detail how Komprise Transparent Archiving works, read the whitepaper File vs Block Tiering—What’s the Difference?

Native Access to Tiered Data on Nutanix or the Cloud

When Komprise moves data, it preserves native access to the data, meaning that the migrated and tiered files can be directly accessed at the destination without requiring Komprise or any third-party software. This ensures that you can fully leverage the native capabilities of each cloud and storage architecture without getting locked in. It also ensures that your savings are maximized since there is no rehydration required on data access.

For instance, as the screenshots below show, when Komprise archives files to an object storage, the files are still accessible from the original location as files, and they are also accessible as objects from the destination.

Figure 8: Files Archived and Tiered by Komprise Are Accessed Exactly as Before - No Agents, No Stubs Needed

Komprise for Nutanix Architecture

Komprise is designed to scale on demand to meet your requirements—you can start with two Komprise virtual machines and add more as needed. Komprise does not require any dedicated hardware and runs as a scale-out grid of virtual appliances. A typical challenge with traditional storage analytics software is that they may disrupt the performance of the storage. Komprise overcomes this issue by adaptively throttling back when the storage systems are in active use, so that Komprise runs non-disruptively in the background.

Another challenge with legacy data management solutions is their use of static stubs (pointers to the moved file). Since these stubs can be deleted or corrupted, the moved files can be orphaned. Komprise overcomes this challenge by not using static stubs, but instead using resilient dynamic links that continue to provide access to the data even if it is moved multiple times, or if the source is migrated. This fits perfectly with Nutanix as they both operate using standard SMB and NFS protocols.

Komprise is also completely out of the hot data path and does not get inline. So, access performance of the active data is unchanged and in fact may improve as the primary storage is less overloaded.

To scale effectively without centralized bottlenecks, Komprise does not use a traditional SQL database that limits scalability. Instead, the Komprise grid analyzes and aggregates metadata and the Komprise Director presents an accumulation of these aggregates. This ensures that all your data is stored in your Nutanix and cloud data stores, and Komprise simply provides an out-of-band data management pane.

To learn more about the Komprise architecture, read the Komprise Intelligent Data Management Architecture Overview whitepaper.

Benefits of Komprise and Nutanix

As your unstructured data continues to grow, managing data across storage and cloud environments is becoming a bigger challenge. Komprise and Nutanix simplify data management at scale by providing you powerful data analytics across all your storage, by making it easy to migrate the right data to Nutanix, and by reducing ongoing costs of managing data on Nutanix.

Next Steps

To try Komprise in your environment and assess how much you can save with Nutanix and Komprise, email nutanix@komprise.com or visit komprise.com/nutanix

This was a guest post from Paul Chen, Director Product Management, Komprise

©️ 2021 Nutanix, Inc.  All rights reserved. Nutanix, the Nutanix logo and all Nutanix product, feature and service names mentioned herein are registered trademarks or trademarks of Nutanix, Inc. in the United States and other countries. Other brand names mentioned herein are for identification purposes only and may be the trademarks of their respective holder(s). This post may contain links to external websites that are not part of Nutanix.com. Nutanix does not control these sites and disclaims all responsibility for the content or accuracy of any external site. Our decision to link to an external site should not be considered an endorsement of any content on such a site. This post may contain express and implied forward-looking statements, which are not historical facts and are instead based on our current expectations, estimates and beliefs. The accuracy of such statements involves risks and uncertainties and depends upon future events, including those that may be beyond our control, and actual results may differ materially and adversely from those anticipated or implied by such statements. Any forward-looking statements included herein speak only as of the date hereof and, except as required by law, we assume no obligation to update or otherwise revise any of such forward-looking statements to reflect subsequent events or circumstances.

Terms & Conditions