Installation & Configuration

Welcome to the Nutanix NEXT community. To get started please read our short welcome post. Thanks!

cancel
Showing results for 
Search instead for 
Did you mean: 

Meltdown & Spectre Vulnerabilities

SOLVED Go to solution
Moderator Moderator
Moderator

Re: Meltdown & Spectre Vulnerabilities

quick follow up @paulw_wwf - Talked to security team. They're aware of this ask (others have asked too), and its on the plan to get it done.

 

I think, besides this, you'll find Nutanix to be an incredibly transparent company. Anything you want, unless its NDA'd, we'll give you freely

Jon Kohler | Technical Director, Engineering, Nutanix | Nutanix NPX #003, VCDX #116 | @JonKohler
Please Kudos if useful!
Apprentice

Re: Meltdown & Spectre Vulnerabilities

Thanks for the update.  If it's just down to the site structure, perhaps you could put a snapshot of the current vulnerability and patch status in this thread.  I'm particulary interested in the Intel CPU microcode updates which are required to mitigate Spectre.  In general, these are delivered via a BIOS update.  Our HP laptops got a BIOS update with the patches in mid December, our CIsco UCS Blades are due an update on 18th Feb.  Are there planned BIOS updates for Nutanix hardware, and if so, what are the timelines?

 

Thanks for listening,

 

Paul W.

 

Moderator Moderator
Moderator

Re: Meltdown & Spectre Vulnerabilities

The microcode updates depend on a mix of hypervisor and hardware. Some hypervisors (namely ESX and AHV) can load the new microcode as a side load upon boot, after being upgraded to the appropriate version.

 

Others, like Hyper-V, can not yet do that, so you need to strictly depend on a BIOS Update.

 

The BIOS update, in general, is going to be a good idea, and we're wrapping a few other goodies in there that we've been working on. We're working on that, date hasn't been set. Tenative was ~Feburary.

 

That said, you may have seen some manufacturers pulled their new BIOS updates (Dell did that with 13G yesterday) and Intel issued an advisory *yesterday* saying they are seeing some reboot issues in the field: 

https://newsroom.intel.com/news/intel-security-issue-update-addressing-reboot-issues/

 

We're taking the time to really make sure we get this right, as industry-wide these patches were incredibly rushed because of the early embargo break throwing the extra week of engineering time into a frenzy.

 

The latest copy of the Spectre/Meltdown advisory is available in PDF form here: http://download.nutanix.com/alerts/Security-Advisory_07-v5.pdf

 

You'll see the exact comment in our advisory as: 

---The availability of BIOS versions with stable CPU microcode updates for NX models is under evaluation

 

Anyhow, for your request, here's a screenshot of all advisories that are posted as of today:

Screenshot 2018-01-12 10.17.39.png

 

 

Jon Kohler | Technical Director, Engineering, Nutanix | Nutanix NPX #003, VCDX #116 | @JonKohler
Please Kudos if useful!
Apprentice

Re: Meltdown & Spectre Vulnerabilities

Thanks Jon.  I was aware Linux could push new CPU microcode, I was not aware that ESXi and AHV were doing the same.  That certainly makes the BIOS updates much less important, so I appreciate the heads up.  And I do wish to congratulate you on the excellent information provided in the PDF, it's a very good overview of the situation.  As an engineer I like it.

 

I'm not sure the data you're publishing actually helps me much with my conversations with senior managers.  These tend to be "Are we protected?" "Only partlally, it's complicated" "OK, so when will we be fully protected?".  They would much prefer some target dates for either patches, or confirmation that updates aren't required.  Things like "under evaluation" or "when it's ready" don't sit well with them.

 

Just to clarify, are you saying that the latest AHV and ESXi patches do contain fixed microcode, making BIOS updates academic?  Or just that they could do?  Again, thanks for listening, and thanks for all the technical updates you've provided.

Highlighted
Moderator Moderator
Moderator

Re: Meltdown & Spectre Vulnerabilities

RE management conversations

Understandable. You'll find that Nutanix as a company is maniacal about security, and the system is already hardened by default. AOS itself is a closed system, where you can't run 3rd party code. That doesn't remove every attack vector under the sun, but it means we do have a wee bit of time to get it right, rather than rush a patch to our core storage system.

 

 

To be clear, we are NOT taking the approach that other vendors have taken (cough cough, ryhmes with "net-lap" cough cough), where they state thing like:

"Unlike a general-purpose operating system, <other vendor system name here> does not provide mechanisms for non-administrative users to run third-party code. Due to this behavior, <other vendor system name here> is not affected by either the Spectre or Meltdown attacks." That's not fingerpointing, its a fact, those net-lappers did that in their public response.

 

In our mind, that's the "easy way out", and we don't think that's the right way to treat our customer's systems.

 

Even though that same statement is true for AOS, we're still evaluating steps to harden the AOS against this issue. I can't comment on the specifics because the patches aren't done yet, but just know that we're taking the extra time to get this right. We're not planning on punting this like those other guys.

 

RE Microcode

Yes, AHV and ESXi patches contain fixed microcode. 

 

VMware will confirm the same here: https://kb.vmware.com/s/article/52085 - See point three under the resolution. Basically, apply the BIOS update OR apply the ESXi patch.

 

We're still planning on releasing update BIOS either way, but just know that for AHV and ESXi, you get coverage in software to begin with.

Jon Kohler | Technical Director, Engineering, Nutanix | Nutanix NPX #003, VCDX #116 | @JonKohler
Please Kudos if useful!
Moderator Moderator
Moderator

Re: Meltdown & Spectre Vulnerabilities

FYI Version 6 of the update here: http://download.nutanix.com/alerts/SecurityAdvisory07-v6.pdf

Jon Kohler | Technical Director, Engineering, Nutanix | Nutanix NPX #003, VCDX #116 | @JonKohler
Please Kudos if useful!
Nutanix Employee

Re: Meltdown & Spectre Vulnerabilities

Jon has you covered on the tecnical front, but I wanted to jump in and thank you for your feedback, and others on the thread as well for the same.  As the one that typically writes the Security Advisories I wanted to thank you for the kinds words, this one took a while to write and it was quite the team effort.

 

That being said, your feedback on the "senior Manager conversation" is a great one.  It's a tough balance.  Finding that half way point between enough information to feed the Engineer while trying to avoid it becoming a tech paper.  I'll put some thought to that, see how we can better deliver the message so it's useful in more conversations.  Thank you for taking the time to provide your thoughts.  They are very valuable and most appreciated.

 

Eric Hammersley | Technical Director, Engineering (Security)
Nutanix Employee

Re: Meltdown & Spectre Vulnerabilities

Security Advisory 7, update 7 has been posted to the portal.  

 

http://download.nutanix.com/alerts/Security-Advisory_07_v7.pdf

Eric Hammersley | Technical Director, Engineering (Security)