This blog was authored by Paul Updike, Sr. Manager, Technical Marketing Engineering at Nutanix.
We created X-Ray simply because it was needed. X-Ray solves the problem of effectively testing the performance of hyperconverged infrastructures.
The reason we had to address this problem is because traditional workload generators like iometer, vdbench, and fio are brute force speed tests designed for traditional shared storage arrays. There's a lot of baggage that accompanies that:
They just read and write data.
They were created when the upper limits of storage performance really mattered.
They are leftovers of measuring spinning disks
Ultimately, they were built in a time when storage performance was the first order problem: the most likely limit to your application’s performance was its ability to access storage.
But that changed with SSDs.
Flash changed the storage performance conversation. Just a few SSDs can easily exceed application requirements. If you take advantage of SSDs the way hyperconverged infrastructures (HCI) do, you can move most application bottlenecks to waiting for CPU, not storage. This also changed the way systems scale.
HCI allows for incremental scale with workloads colacted on the same system as your storage. It removes the guesswork needed to evaluate the maximum a storage array can handle. Instead of buying one big instance of storage and adding workloads over time, you scale your storage as your needs increase. You should still understand the performance of a system but it changes to a broader sense, one that is more applicable to true datacenter operations rather than the unsophisticated drag race of the old workload generators.
Because most application requirements are easily met with flash, storage performance becomes a control variable in a more complete test of an entire system. X-ray will run a workload characterization and for most tests it will make the throughput requirement steady state. Instead of testing the limits of storage performance, we shift the focus to that of maintaining consistent and reliable performance in a number of scenarios that are encountered in data centers.
Some of these common scenario categories include:
Noisy Neighbors - Run a workload in one VM and add a new VM with new work. The application in the first VM should be unaffected.
Additional workload - Increase the rational workload of an application, there should be little effect on performance as work is added.
Rolling upgrades - There should be little effect seen at the application as hypervisor and virtual storage controllers are upgraded.
Node failures - Fail a node, there should be a slight pause (that’s probably not noticed by application or user) and then little effect to the application after the failure.
Having a fair scenario driven tool is useful in a variety of ways. An obvious one is proof of concepts. X-ray is transparent with scenario profiles and what the tests do, we build to avoid bias in both tests chosen and test implementation. The tests are well described and defined in YAML. The exact parameters that will be used in the test are easy to see and verify. The goal is to provide a trusted tool and to do that we expose what it does. Beyond that we also see X-ray in a bigger light. A lot of effort in IT is spent in qualifying new systems, a new hypervisor and storage upgrades, etc. These efforts should include fault injection and management operations but doing this by hand is tedious and slow. A full effort can take weeks to complete with potential restarts each time a problem is encountered. This is where X-ray shines.
The ultimate realization of X-ray is in its automation and orchestration of events while a system is under load. The results you analyze are the stability, consistency and predictability of the system. This capability enables IT departments to vet changes rapidly and systematically in a lab environment before going to production. A full run of X-ray test is usually only a couple of days. For example, you can kick off testing on a Friday morning and come back to completed tests on Monday. This potentially replaces weeks of tests without automation. For shops moving to more of a devops model, this accelerates their adoption.
Back to the original question, “Why X-ray”?
Traditional workloads generators don't fit our modern needs.
X-ray moves the test focus to real-world datacenter scenarios.
X-ray is an automation engine for POCs and system, hypervisor, and OS qualification and acceptance.