Nutanix + AI Infrastructure

Hey Allan!

Head’s up, lots of text incoming :-)

That is definitely the $1M question, isn’t it? With supply chain issues, shifting AI tech, and upgrading your current infra, where do you start?😅

Here’s my take-

1. First understand what you’re going to do with AI. Maybe you have an OpenAI or Anthrpropic subscription for your business today?
   
   Do you have needs to run AI privately vs. with a provider like that? Are you creating agents, basic bots, or just experimenting? Maybe you already are seeing costly token bills or hitting limits? How are your AI consumers inside your company using AI right now? Are you using ‘fringe’ AI tools that could be merged and acquired by bigger companies, and/or open-sources tools?
   
   This will create a blueprint for how you need to align your tech stack.
    
2. Next, speaking of which, list out your current tech stack. Call it “waving you magic wand to fix all the things!”. Look and catalog all those things you mentioned and evaluate them around your needs that includes AI.
   
   For example, have you been putting off those network switch upgrades or need to move your AD to Azure AD? Need a way to run Kubernetes local or in the cloud? Start here to get the basics right.
   
   As a pro you probably know, core IT growth starts with core tech moderization.
    
3. Now overlay that magic wand list on top of your AI strategy or initiatives and requirements with Nutanix. You’ll immediately see peaks and valleys  on what’s ready and what’s going to need some help.
   
   Again examples: Did you just get Nutanix Flow into your environment? Awesome, now you can start to plan for container-to-container secure workflows. Did you just upgrade you nodes in your cluster with new hardware? No worries, maybe you can start with using an AI gateway for an OpenAI/Anthropic without having to go buy GPUs for now. 
   
   These will help drive where AI must mature and change and create an AI map for your needs. The cool thing is that Nutanix is built to adapt and typically doesn’t solidify it’s tech around a single concept that will leave you without options.
    
4. Finally, try and find good AI partner to help bring it all together. AI is still so new and it’s changing so fast, that having someone to help guide your decisions is critical, whether that’s just advice or actual services. 
   
   Most customers I’ve talked to have this situation internally:
   - One or two people who are really pushing and doing AI today
   - This has created some AI tech-debt. This could be a DGX spark purchase here, or coding against a certain model there. Which is fine, but it requires some strategic thinking to scale
   - Executive pressure to “are we doing this AI thing?” Usually it’s not-specific OR, it’s VERY specific on their thinking
   - This all leads to the execution falling into your lap. That’s tough because your left to juggle the actual reality of this AI thing.
   
   Having someone who has the expertise and talent is a huge win to augment your implementations.

Well, I HOPE this helps, lol. Feel free to reach out to me, I am more than happy to go deeper!

Cheers!

​@mikebarmonde Great discussion. I think one aspect that deserves more attention is that enterprise AI success is becoming less about GPUs and model selection, and more about operational consistency. Provisioning AI infrastructure is relatively straightforward; governing it at scale is the real challenge.

Organizations need repeatable lifecycle management, security, observability, and infrastructure automation just as much as raw compute. That's where a unified platform approach can make a significant difference, especially as AI workloads move from isolated pilots to production services across hybrid environments. Looking forward to seeing how others are tackling Day 2 operations for enterprise AI.

Yo!!! Great insight, Allan!

I think one aspect that deserves more attention is that enterprise AI success is becoming less about GPUs and model selection, and more about operational consistency.

AB-SO-LUTELY! :-)

You have some wisdom in here!

 Provisioning AI infrastructure is relatively straightforward; governing it at scale is the real challenge.

Funny you mention this as I saw an internal discussion today about sizing at scale and how complex it can be with all the moving parts AI demands.

Looking forward to seeing how others are tackling Day 2 operations for enterprise AI.

Completely agree. Maybe you’re (royal ‘you’re’ ) playing with an agent for a pilot of 10 people? Works awesome! NOW, how do you scale, monitor, operate, and adapt that agent across hundreds or thousands. 

Maybe it’s relatively known, but here are some things NAI does (and will do in a couple weeks pending a new version) really well for scale and operations:

• NAI removes a couples of risks associated with shadow AI and extends operational visibility:
   
  • Obfuscates your public provider keys by “vaulting” them inside NAI, then making them centrally managed and obscured through an NAI endpoint (model + NAI created API key) to apps and devs.
    
    This makes changing models and acess super easy because if a key is compromised or changed, the role-base access of that key is easily traced to an app or individual.
    
    Also, with unified endpoints now in NAI 2.7 you can now change out the models an API key uses on-the-fly without minting a new key. Running Claude today but want OpenAI tomorrow? No worries, it’s a 10 second process in NAI to change. Need run both for a while under you get some user feeback? No problem, load balance between them, then turn off one or the other when you’re ready.
     
  • Enable Open Telemetry from NAI to your SIEM or centralized logging system for instant metrics across an entire range  of things NAI captures. The new Nutanix Validated Design for NUtanix Enterprise AI does a really good job of setting up pretty verbose logging via Kubernetes
    
    The NAI dashboard is mainly used to dive into operational needs at-a-glance in my opinion. The real power lies in centralizeing that data and aligning it with your other IT systems, which this lays out.

Again, AMAZING insights!

It’s something we have be discussing as well, in my personal opinion, deploying AI at least for our usecase is we are having to look at the solution end to end and like you we have a lot of concerns when it comes to governance and ensuring the correct amount of guard rails are in place. 

Why end to end and not just implement the bulk of it and fix the rest afterwards?

This approach may work for a lot of IT implementation projects but where this approach does not work for AI and that implementation is generally only because those implementations touch a handful of systems. AI generally has a lot more moving parts and more moving parts results in more complexity. A chat bot is very different to agentic workloads.

The approach we are taking when designing our implementation is making it like a container, so each request from the business is approach from the perspective

1. What is someone else wants this?
2. If we need more of these ‘Containers’ how to we ensure it has the right network access, only access the applications it needs
3. Consistence framework for ensuring and validating it can only access what it needs

We are only in the early days of planning this and these are the sort of questions I am asking myself. 

Everyone wants AI, so designing a framework that enables us to move fast but safely is something that is getting a lot of considerations.

It’s a hard question to answer since AI is moving faster then people can understand it and that is where my concerns are, while we can have all the guard rails in place, I am still not convinced AI can be implemented safely due to how fast models are changing.

The one question I have is while we can talk about governance and guard-rails, each time a new model is released, how do we ensure those guard-rails are still effective. 

Look at the Fable5 release. It got pulled within 48 hours after someone was able to jailbreak the model, how do we prevent that same ‘jail breaking’ within our corporate environments?