Nutanix Enterprise AI 2.8 was just released. Let’s break down what’s new! 🎉
GA Features
MCP Server Support
MCP Server support is now Generally Available. Route your MCP servers through the Nutanix Agent Gateway to help your AI agents safely and securely access internal tools and data via audited connections. Configure an MCP connector that allows you to aggregate local and remote servers behind a unified endpoint, giving you precise control over which tool capabilities are exposed per API key. Reduce credential sprawl by binding a single key to both the inference endpoints and the MCP connector, and keep local servers up to date with non-disruptive rolling upgrades.
Fine Tuning
Fine-tuning for models under 8B parameters is now Generally Available, offering a streamlined workflow to train LLMs on custom, domain-specific data and persist those models on NFS storage. This capability allows organizations to achieve frontier-model accuracy using smaller, hardware-efficient footprints. By optimizing smaller models for targeted enterprise tasks, teams can maximize their existing GPU capacity to help realize potential cost savings for high-performance AI services compared to deploying much larger, more resource-intensive models.
Custom Inference Engine Parameters
Nutanix Enterprise AI enables users to leverage their preferred version of vLLM and custom parameters, allowing them to test and experiment with bleeding-edge community releases and performance optimizations immediately instead of waiting for the next officially validated release. With NAI 2.8, this capability is easier than ever to use. Parameters and environment variables can now be added in a text field as opposed to one-by-one. Additionally, updating parameters and environment variables on an existing, hibernated endpoint is now possible, instead of having to recreate the entire endpoint.
NAI Labs Updates
NAI Labs enables users to quickly test various endpoints and operational workflows such as Retrieval Augmented Generation (RAG) workflows or testing agent interactions with MCP connectors, without needing to develop custom applications. NAI 2.8 enhances the Agent application by now enabling document upload support.
Airgapped Support for NVIDIA NIM
For customers with NVIDIA AI Enterprise licenses, an internet connection to the NVIDIA NGC catalog was required for NVIDIA NIM model download. With NAI 2.8, customers can now upload NIMs from a local file share or bucket to support airgapped environments. Additionally, customers with connected environments can now import models via URL.
New Pre-Validated Model Support
Nutanix Enterprise AI provides a curated model catalog of pre-validated models from Hugging Face and the NVIDIA NGC catalog, enabling customers to centralize models across repositories while taking the guesswork out of sizing and configuration. NAI 2.8 expands the Hugging Face catalog with 4 new models:
- mistralai/Mistral-Small-4-119B-2603
- mistralai/Mistral-Large-3-675B-Instruct-2512
- nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16
- nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16
Tech Preview Features
Nutanix Enterprise AI 2.8 also brings in some new Tech Preview features.
Header-Based Rate Limiting (Tech Preview)
If your users are connected to a centralized application (e.g. via AD/SSO) that accesses the Nutanix Agent Gateway via a single API key, you can now leverage header-based rate limiting to identify users and apply token limits to that user, effectively decoupling resource consumption control from the primary authentication mechanism. Admins can now manage costs and prevent resource monopolization by setting budgets across minute, hour, day, and month intervals. This flexibility allows third-party applications to handle end-user authentication while the Nutanix Agent Gateway retains the authority to enforce granular resource quotas and token overrides for specific accounts.
Multi-Node and Multi-GPU Inference (Tech Preview)
Multi-node inference utilizes pipeline parallelism to serve frontier-class models with 100B+ parameters that exceed the physical VRAM limitations of a single node. This configuration splits computation into sequential stages across multiple worker nodes, enabling the deployment of massive models that were previously inaccessible on standard enterprise hardware. This architecture is optimized for throughput-sensitive applications, prioritizing total system capacity over single-request latency, providing a scalable solution for high-volume enterprise inference tasks.
KV Cache Offload (Tech Preview)
KV cache is a critical component of LLMs and GPU memory optimization, preventing the recalculation of previously computed context. As historical context grows, the KV cache on GPU memory can fill up, leading the inference engine to start evicting parts of the cache. To resolve this, KV cache offloading can help preserve the cache. NAI 2.8 now supports KV cache offloading (in tech preview) to host memory, enabling fast time-to-first-token (TTFT) on repeated or similar prompts.
See It In Action
NAI 2.8 is now available to all Nutanix Enterprise AI customers. Explore the latest capabilities and see how you can deliver greater governance, control, and efficiency to your enterprise AI deployments. Reach out to your Nutanix representative to learn more, or explore these self-service resources:
- Watch product demos on the Nutanix Enterprise AI YouTube Playlist
- Take a Nutanix Enterprise AI Test Drive
- Read the release blog
Note: Tech Preview features should not be deployed in a production environment. For more information on how Tech Preview features should be used, please see KB 6673.