Nutanix + AI Infrastructure

Hey Allan!

Head’s up, lots of text incoming :-)

That is definitely the $1M question, isn’t it? With supply chain issues, shifting AI tech, and upgrading your current infra, where do you start?😅

Here’s my take-

1. First understand what you’re going to do with AI. Maybe you have an OpenAI or Anthrpropic subscription for your business today?
   
   Do you have needs to run AI privately vs. with a provider like that? Are you creating agents, basic bots, or just experimenting? Maybe you already are seeing costly token bills or hitting limits? How are your AI consumers inside your company using AI right now? Are you using ‘fringe’ AI tools that could be merged and acquired by bigger companies, and/or open-sources tools?
   
   This will create a blueprint for how you need to align your tech stack.
    
2. Next, speaking of which, list out your current tech stack. Call it “waving you magic wand to fix all the things!”. Look and catalog all those things you mentioned and evaluate them around your needs that includes AI.
   
   For example, have you been putting off those network switch upgrades or need to move your AD to Azure AD? Need a way to run Kubernetes local or in the cloud? Start here to get the basics right.
   
   As a pro you probably know, core IT growth starts with core tech moderization.
    
3. Now overlay that magic wand list on top of your AI strategy or initiatives and requirements with Nutanix. You’ll immediately see peaks and valleys  on what’s ready and what’s going to need some help.
   
   Again examples: Did you just get Nutanix Flow into your environment? Awesome, now you can start to plan for container-to-container secure workflows. Did you just upgrade you nodes in your cluster with new hardware? No worries, maybe you can start with using an AI gateway for an OpenAI/Anthropic without having to go buy GPUs for now. 
   
   These will help drive where AI must mature and change and create an AI map for your needs. The cool thing is that Nutanix is built to adapt and typically doesn’t solidify it’s tech around a single concept that will leave you without options.
    
4. Finally, try and find good AI partner to help bring it all together. AI is still so new and it’s changing so fast, that having someone to help guide your decisions is critical, whether that’s just advice or actual services. 
   
   Most customers I’ve talked to have this situation internally:
   - One or two people who are really pushing and doing AI today
   - This has created some AI tech-debt. This could be a DGX spark purchase here, or coding against a certain model there. Which is fine, but it requires some strategic thinking to scale
   - Executive pressure to “are we doing this AI thing?” Usually it’s not-specific OR, it’s VERY specific on their thinking
   - This all leads to the execution falling into your lap. That’s tough because your left to juggle the actual reality of this AI thing.
   
   Having someone who has the expertise and talent is a huge win to augment your implementations.

Well, I HOPE this helps, lol. Feel free to reach out to me, I am more than happy to go deeper!

Cheers!

​@mikebarmonde Great discussion. I think one aspect that deserves more attention is that enterprise AI success is becoming less about GPUs and model selection, and more about operational consistency. Provisioning AI infrastructure is relatively straightforward; governing it at scale is the real challenge.

Organizations need repeatable lifecycle management, security, observability, and infrastructure automation just as much as raw compute. That's where a unified platform approach can make a significant difference, especially as AI workloads move from isolated pilots to production services across hybrid environments. Looking forward to seeing how others are tackling Day 2 operations for enterprise AI.

Yo!!! Great insight, Allan!

I think one aspect that deserves more attention is that enterprise AI success is becoming less about GPUs and model selection, and more about operational consistency.

AB-SO-LUTELY! :-)

You have some wisdom in here!

 Provisioning AI infrastructure is relatively straightforward; governing it at scale is the real challenge.

Funny you mention this as I saw an internal discussion today about sizing at scale and how complex it can be with all the moving parts AI demands.

Looking forward to seeing how others are tackling Day 2 operations for enterprise AI.

Completely agree. Maybe you’re (royal ‘you’re’ ) playing with an agent for a pilot of 10 people? Works awesome! NOW, how do you scale, monitor, operate, and adapt that agent across hundreds or thousands. 

Maybe it’s relatively known, but here are some things NAI does (and will do in a couple weeks pending a new version) really well for scale and operations:

• NAI removes a couples of risks associated with shadow AI and extends operational visibility:
   
  • Obfuscates your public provider keys by “vaulting” them inside NAI, then making them centrally managed and obscured through an NAI endpoint (model + NAI created API key) to apps and devs.
    
    This makes changing models and acess super easy because if a key is compromised or changed, the role-base access of that key is easily traced to an app or individual.
    
    Also, with unified endpoints now in NAI 2.7 you can now change out the models an API key uses on-the-fly without minting a new key. Running Claude today but want OpenAI tomorrow? No worries, it’s a 10 second process in NAI to change. Need run both for a while under you get some user feeback? No problem, load balance between them, then turn off one or the other when you’re ready.
     
  • Enable Open Telemetry from NAI to your SIEM or centralized logging system for instant metrics across an entire range  of things NAI captures. The new Nutanix Validated Design for NUtanix Enterprise AI does a really good job of setting up pretty verbose logging via Kubernetes
    
    The NAI dashboard is mainly used to dive into operational needs at-a-glance in my opinion. The real power lies in centralizeing that data and aligning it with your other IT systems, which this lays out.

Again, AMAZING insights!

It’s something we have be discussing as well, in my personal opinion, deploying AI at least for our usecase is we are having to look at the solution end to end and like you we have a lot of concerns when it comes to governance and ensuring the correct amount of guard rails are in place. 

Why end to end and not just implement the bulk of it and fix the rest afterwards?

This approach may work for a lot of IT implementation projects but where this approach does not work for AI and that implementation is generally only because those implementations touch a handful of systems. AI generally has a lot more moving parts and more moving parts results in more complexity. A chat bot is very different to agentic workloads.

The approach we are taking when designing our implementation is making it like a container, so each request from the business is approach from the perspective

1. What is someone else wants this?
2. If we need more of these ‘Containers’ how to we ensure it has the right network access, only access the applications it needs
3. Consistence framework for ensuring and validating it can only access what it needs

We are only in the early days of planning this and these are the sort of questions I am asking myself. 

Everyone wants AI, so designing a framework that enables us to move fast but safely is something that is getting a lot of considerations.

It’s a hard question to answer since AI is moving faster then people can understand it and that is where my concerns are, while we can have all the guard rails in place, I am still not convinced AI can be implemented safely due to how fast models are changing.

The one question I have is while we can talk about governance and guard-rails, each time a new model is released, how do we ensure those guard-rails are still effective. 

Look at the Fable5 release. It got pulled within 48 hours after someone was able to jailbreak the model, how do we prevent that same ‘jail breaking’ within our corporate environments?

​@jarrodl I completely agree. AI guardrails can't be treated as a one-time implementation. They need to evolve with the models they're protecting. Every model update should trigger revalidation of policies, permissions, prompts, and security controls. To me, AI governance is becoming a continuous engineering discipline rather than a compliance exercise. Moving fast is important, but maintaining trust is what makes AI sustainable in production.

Love this:

Everyone wants AI, so designing a framework that enables us to move fast but safely is something that is getting a lot of considerations.

Great question:

The one question I have is while we can talk about governance and guard-rails, each time a new model is released, how do we ensure those guard-rails are still effective. 

I feel there are two parts to the answer here:

1.  How is the industry sees this working
2. What Nutanix does to enable this

What is the best practice on making AI safety effective and consistent when switching models?

-  How is the industry sees this working:

Doing some research (do some “light reading” on OWASP if you haven’t yet lol), there seems to be a solid order of operations for this:

• One centralized guardrail service or AI gateway. (I’d say some guardrails are agnostic to model change)

• Separate policies/guardrails for chat, retrieval, and agent actions.

• Per-model evals for jailbreak resistance, hallucination rate, and tool-call behavior (I’d call this internal alignment and testing).

• Canary rollout whenever a model changes, even if the guardrail layer stays the same

The actual operations alignment could look like this:

1. The gateway authenticates the caller, applies input checks, and blocks obvious policy violations.

2. The gateway routes the request to the selected model.

3. The model response comes back through the same gateway.

4. The gateway applies output moderation, redaction, logging, and action controls before returning anything to the user.

- What Nutanix does today to enable this

Good news! The Nutanix Agent Gateway for NAI helps with a lot of this right now, and more capabilities are under development. One of the strengths of Nutanix’s approach is using partners to help augment things. In this case, leveraging something like the new integration announced at .NEXT 2026 with Palo Alto Prisma AIRS creates a holistic approach to making this real right now.

So BASED on this thread and with the help of AI, I created this whitepaper on this! 

Technical White Paper: Model-Agnostic AI Safety and Guardrails with Nutanix Enterprise AI  https://docs.google.com/document/d/1wZ9T23C_-mR7gZDiXELSrGE7aTJQlb8CcsVgC1D1WFM/edit?tab=t.0 which goes through how:

• Nutanix Enterprise AI lines up well with the modern model-agnostic safety approach: keep the real guardrails outside the model.
• Use a central gateway for auth, routing, audit logs, quotas, and tool permissions, then apply input/output checks for prompt injection, sensitive data, and unsafe actions.
• This way, you can swap models without rebuilding your safety stack.
• The Nutanix pieces that matter most here are unified endpoints, private inference, RBAC, observability, token controls, and MCP governance for agents.
• The practical starting point is a governed chat or RAG workflow, then add tool access and stronger controls only when the use case needs them.

Feedback WELCOME, but I think this should help align these challenges and what can be solved really well.

Super cool discusison!

Great insights, Mike. I agree that decoupling guardrails from the underlying model is the sustainable approach. As organizations adopt multiple LLMs, governance should remain consistent regardless of which model is behind the API. In practice, this means enforcing identity, authorization, policy validation, prompt protection, logging, and observability at the gateway layer while continuously evaluating model behavior through automated red teaming and regression testing whenever models are updated. AI governance should evolve independently of the models it protects.