on 12-03-201512:03 PM - last edited on 07-26-201608:49 AM by aluciani
Performance of HDFS/Spark/Flume is made easy with the builtin auto-tiering into the XCP platform. Using Hortonworks HDP 2.3 we take a look at running the DSFIO benchmark and see how easy it is to montior the workload.
This test was performed with a 3460 in only 2U of space!
I've had a great time working on the newest reference architecture for Hadoop running on top of the Acropolis hypervisor. It was fun to work with so many people across the company to get the testing and validation finished. Engineering, Solutions, Alliances, Marketing and SE’s alike all helped at some point.
Prasad Athawale, now Director of Business Development use to work at Yahoo and in the Nutanix engineering department saved me a ton of time. Getting MapReduce tuned for the TeraSort can be art in itself so I was happy to have Prasad to answer my questions. The Engineering team was also able to further optimize sequential traffic with the 4.5 and they were able keep more in flash with changing the heuristic on auto-teriing of SSD to HDD.
While the work was done using Hortonworks Data Platform 2.3 you could just slot in Cloudera and you’d be off to the races too. While there is no licensing cost for the Acropolis Hypervisor the real story is around ease of use.
The main take always from the work is nicely summed up below.
If you want to hear about the RA and how HDFS and the Acropolis DSF work together I would encourage you to sign up to the webinar happening next week. There are two separate times so hopefully we can make it convenient for you regardless of time zone.