Knowledge is power! The key to modern IT is predicated on the source of that knowledge: data. If the data storage is not optimized, it will hinder agility across the business. There are many ways to store it, however it’s not always clear which method is best for the various use cases. In this blog we will uncover one of the most talked about options, object-based storage, and how it differs from file and block.
Data is growing at substantial rates, and according to IDC by 2020 total volumes will hit 40 Zettabytes with about 63% of it being unstructured . This growth requires scalability and flexibility all with simplified intelligent management.
Quick Recap of Block and File
Traditionally primary data is stored as blocks and files within the datacenter. In particular, block storage functions at very low levels, in bytes, and is organized in structured fixed blocks that are easily indexed and searched. Hence the name “structured” data. This is typically your SAN (Storage Attached Network) devices. File storage, however is often laid out in a hierarchy of files and folders, found on NAS (Network Attached Storage) systems. SAN devices use iSCSI and Fibre Channel protocols, and NAS filers use SMB or NFS.
These types of storage are naturally found close to compute resources. However, as data continues to grow— specifically the kind that is not ready to be migrated to cold storage and doesn’t need to be stored with compute—there should be options for storing that data efficiently, securely, and cost effectively.
Enter Object Storage
Object storage is designed for unstructured data that is highly scalable and resilient for the world of cloud computing. Unstructured data is not a fixed format and consists of varying size files. It’s storage that’s accessible over the network via simple S3-compatible HTTP REST API calls. S3 (Simple Storage Service) was developed by Amazon Web Services, which has become the de-facto standard for many of the leading object storage providers today. The reason why? They were first to market, and a well documented interface.
Here are some typical object storage characteristics:
- Data is stored as objects versus traditional blocks
- Objects can include backups, archives, videos, images, logs, HTML files, and more
- It’s unstructured by nature because there is no format to the way data is stored.
- Unlike the directory structure found in traditional file systems, it utilizes a flat list of objects stored in “buckets.”
- Objects are stored using unique IDs rather than filenames, which drastically reduces the overhead required to store data.
- Objects are stored with user-defined metadata, making it easier to find objects at scale
- Objects can be terabytes or even a few kilobytes in size and a single bucket can hold billions of objects.
- Application developers can easily access objects using simple S3-compatible API calls through “GET” and “PUT” requests, without complex directory structures.
Buckets are not nested like files, folders or data within other buckets. They are available via a single namespace and can grow from a few terabytes to petabytes in size. Interactions with the data is easier due to the single namespace, which is a single logical view of the data that can span multiple storage devices.
100MB File Example
Let’s dive into an example to better understand the differences between block and object. How is a 100MB file handled on an object-based storage solution versus a block solution?
When data is written in the block solution, the 100MB file is broken up into fixed-sized granular blocks as opposed to the entire object being written in the object storage solution. Blocks work well for highly interactive local requests, but have limited scale. In the case of updating a 100MB file, block storage can make granular updates without replacing the entire 100MB file. This is great for highly transactional database workloads.
Similarly, when the 100MB object is updated, the entire object can easily be placed into a bucket without breaking the file into blocks with fixed-sized boundaries. This complete update makes object storage ideal for data sets that are updated infrequently. Object storage tends to be the common solution enterprises choose for data that is written once but read many times (WORM). It’s highly scalable as it continually grows with your requirements, it’s built with redundancy to ensure data is always available, and it’s cost effective to reduce overall TCO.
Where should you consider object storage for your business?
Object storage isn’t for all workloads and use cases, but there are certainly many that apply. Try to identify lower I/O workloads where data is growing and your business doesn’t have budget to overprovision today, based on future conditions. Think about how you can fractionally consume storage, but know that ease of use remains the same. Here are some guidelines that might help:
- Long term retention - regulatory compliance data, WORM data
- Backups - database dumps, log files, backups from existing software (i.e. Commtrade HYCU, Commvault)
- DevOps - single global namespace and easily accessible using simple PUT and GET requests for image and artifact management
- Unstructured data - media files, documents, static images, music, and video files
- Industries with large data sets:
- Healthcare - Electronic medical records, digital images, PACS systems data
- Financial - Trading platform, email/IM systems data, regulatory archive system data
- Static web content - Large HTML files and images for websites
There are a few different storage options to consider, whether you are refreshing your datacenter, expanding it, or even just looking for newer technologies. Keep in mind how utilizing the appropriate storage solution will continue driving your business forward and keep your users happy. Legacy architecture designs typically provide file, block, and object through separate point solutions, however modern designs can provide all of these as an integrated solution leveraging the same physical storage backend.
Nutanix Buckets is currently under development and will be available in a future release.
To learn more about the Nutanix Buckets , please send a message to firstname.lastname@example.org or join our community at next.nutanix.com and start a discussion there.
Disclaimer: This blog may contain links to external websites that are not part of Nutanix.com. Nutanix does not control these sites and disclaims all responsibility for the content or accuracy of any external site. Our decision to link to an external site should not be considered an endorsement of any content on such site.
Forward-Looking Statements Disclaimer
This blog includes forward-looking statements, including but not limited to new product features and technology that are under development or in process and capabilities of such product features and technology and our plans to introduce product features in future releases. These forward-looking statements are not historical facts, and instead are based on our current expectations, estimates, opinions and beliefs. The accuracy of such forward-looking statements depends upon future events, and involves risks, uncertainties and other factors beyond our control that may cause these statements to be inaccurate and cause our actual results, performance or achievements to differ materially and adversely from those anticipated or implied by such statements, including, among others: the failure to integrate our products with our partner’s products and the possibility that we may not receive anticipated results from forming such partnerships; the introduction, or acceleration of adoption of, competing solutions, including public cloud infrastructure; a shift in industry or competitive dynamics or customer demand; and other risks detailed in our Form 10-Q for the fiscal quarter ended April 30, 2018, filed with the Securities and Exchange Commission. These forward-looking statements speak only as of the date of this presentation and, except as required by law, we assume no obligation to update forward-looking statements to reflect actual results or subsequent events or circumstances.
2018 Nutanix, Inc. All rights reserved. Nutanix, the Nutanix logo and the other Nutanix products and features mentioned herein are registered trademarks or trademarks of Nutanix, Inc. in the United States and other countries. All other brand names mentioned herein are for identification purposes only and may be the trademarks of their respective holder(s).