Nutanix Kubernetes Engine (NKE)

Hello,I'm currently experiencing a problem deploying a k8s production cluster but I'm able to deploy the dev clusters.I get this error when deploying:Cluster ha-cluster:failed to deploy K8s cluster: failed to deploy master nodes: failed to deploy control plane nodes: Operation timed out: 0: 0If anyone has encountered or knows the source of this problem, I'm interested in the solutionI'm available if you have any further questions or need supplements, thank you in advance for your help.

Hi,I deployed NKP environment with default config, so I have 4 control plane nodes and 3 worker nodes. Can I reduce the nodes on NKP management cluster? or can I safely remove the cluster and re-deploy?Thanks

Good morning everyone, I'm clearing the communication rules between the nutanix clusters using the matrix provided. I came across this communication:SOURCE: HTTPS 30001 TCP - DESTINATION: Prism Central NKM How can I check what the NKM IP is or check where it is enabled. We have Nke but I can't find anything about nkm.Thank you all

I am trying to set up an NKE cluster with an external load balancer (HAProxy). When I attempt to do so, I encounter this error related to the SSL certificate:Cluster k8s-cluster: failed to deploy K8s cluster: failed to deploy master nodes: failed to deploy control plane nodes: Operation timed out: Get "https://10.38.9.165:443/api/v1/nodes/k8s-cluster-2eb745-master-0?timeout=2m0s": x509: certificate signed by unknown authority: 0: 0 Is there any documentation on how to set up an NKE cluster with an external load balancer?How can I make the certificate trusted by NKE?FYI, I am using a self-signed certificate.Here is my HAProxy configuration: #---------------------------------------------------------------------# Example configuration for a possible web application. See the# full configuration options online.## https://www.haproxy.org/download/1.8/doc/configuration.txt##---------------------------------------------------------------------#-----------------------------------------------

in my learning environmentI am trying to set up an NKE cluster with an external load balancer (HAProxy). When I attempt to do so, I encounter this error related to the SSL certificate:vbnetCluster k8s-cluster: failed to deploy K8s cluster: failed to deploy master nodes: failed to deploy control plane nodes: Operation timed out: Get "https://10.38.9.165:443/api/v1/nodes/k8s-cluster-2eb745-master-0?timeout=2m0s": x509: certificate signed by unknown authority: 0: 0 How can I make the certificate trusted by NKE?FYI i am using self signed certificateHere is my HAProxy configuration: #---------------------------------------------------------------------# Example configuration for a possible web application. See the# full configuration options online.## https://www.haproxy.org/download/1.8/doc/configuration.txt##---------------------------------------------------------------------#---------------------------------------------------------------------# Global settings#-----------------------

I am running Kubernetes management 1.28.5-1 and installed a basic production cluster. After it all installed and runs correctly I tried to use the Kubeconfig file to get cluster info via an other linux server but it fails every time.kubectl cluster-infoE0820 08:30:40.983751  237406 memcache.go:265] "Unhandled Error" err="couldn't get current server API group list: the server has asked for the client to provide credentials" the contents of the config file:cat $KUBECONFIG# -*- mode: yaml; -*-# vim: syntax=yaml#apiVersion: v1kind: Configclusters:- name: "my-kube-prd-002"  cluster:    server: https://10.249.8.2:443    certificate-authority-data: LS0<redacted>users:- name: default-user-my-kube-prd-002  user:    token: eyJ0<redacted>contexts:- context:    cluster: "my-kube-prd-002"    user: default-user-my-kube-prd-002  name: my-kube-prd-002-contextcurrent-context: my-kube-prd-002-context  How to proceed from here? this manual https://portal.nutanix.com/page/documents/details?tar

 aos 6.5.5.7prism central ：2022.6.0.10karbon version 2. 2. 3.I enabled karbon,I failed to upgrade karbon using lcm with the following error message: 2024-08-31 17:03:20,326Z ERROR 46603952 helper.py:145 (192.168.30.28, update, aff9f64a-6e19-41a4-5fda-68e7f423c861) 2024-08-31 10:03:20.231345 EXCEPT:{"err_msg":"Update failed with error: [Failed to initialize precheck helper: Failed to initialize Nutanix Kubernetes Engine interface: Failed to build Nutanix Kubernetes Engine session: Failed to get auth session cookie: Failed to make GET request: URL: https://192.168.30.28:9440/api/nutanix/v3/users/me, the error: HTTPSConnectionPool (host ='192.168.30.28', port=9440): Read timed out. (read timeout=30)]", "name": "relea.karbon.update ", "stage": 0} I executed the karbon login command successfully, but karbonctl airgap list failednutanix@NTNX-192-168-30-28-A-PCVM:~/data/logs$ /home/nutanix/karbon/karbonctl login --pc-username adminPlease enter the password for the PC user: adminLogin successf

HiI have a pod with cluster IP already. I want to expose this as load balacner or nodeport service and take one of the private IP from the subnet. However none seem to be working. I just want to know if there is some annotations or additional settings that we need to do to achieve this or exposing pods as service via standard yaml or kubectl commands should work?for example this - kubectl expose deployment sandbox-airflow-web --type=LoadBalancer --name=sandbox-airflow-web-service --namespace=sandbox-airflowdidnt generate any external IP or errors. was stuck in pending. I later patched it to get an IP but there IP didnt return the website.Kubectl patch svc sandbox-airflow-web-service -n sandbox-airflow -p'{"spec": {"type": "LoadBalancer", "externalIPs":["172.20.8.50"]}}'thanks.

We have a Nutanix nke environment and would like to activate protected domains so that in the event of a disaster we can move the PD across to our DR site. Currently I have not seen any documentation or a way to configure this. Does anyone know how we can “replicate” our NKE clusters to a DR location in an easy and simple way. My thought is to take the underlying virtual machines and just put them into a PD. However, it does not seem as simple as this. Any guidance would be apricated. We have been receiving conflicting advice from Nutanix technical support. To the point where it does not seem possible and therefore makes NKE not viable to enterprise deployment. 

We hit the below scenario today. We have a single IP pool IPAM network pool included the CVM/Host IPs When deployed a K8 cluster using this IPAM network, system generated IP list contains 2 IPs of CVM Cluster creation task proceeded without any issue After the K8 cluster creation task , nutanix cluster went down , due to the duplicate IP issueDo we have any check introduced in code for duplicate ip check in the creation task?

I'm in the process of installing MetalLB on Nutanix Kubernetes Engine (NKE) to provide LoadBalancer services. According to MetalLB's documentation, it prefers (requires) kube-proxy to run in ipvs mode for optimal performance. However, I've noticed that NKE defaults to iptables mode for kube-proxy, and my attempts to switch to ipvs mode have been unsuccessful.Has anyone successfully configured ipvs mode for kube-proxy on NKE? If so, could you share the steps or guidance on how to achieve this? Any tips or resources would be greatly appreciated as the standard documentation and methods don't seem to apply directly to NKE.I referred to this blog: Utilizing MetalLB to Provide LoadBalancer Services for Nutanix Karbon.

I was testing the NKE cluster using Test Drive and noticed that when attempting to create a cluster, the latest version available is 1.19.8. I'm curious if this limitation also applies to the regular NKE clusters, or if NKE supports newer versions beyond 1.19.8. and if so what is the latest version?

We are presently engaged with a client who possesses a Nutanix infrastructure and seeks to leverage Nutanix's NKE for deploying their application. Our intention is to utilize their infrastructure as the production setup while concurrently exploring the implementation of an NKE cluster within our VMs, provisioned on a Proxmox server. Despite searching for similar setups, we haven't found relevant precedents. Is it feasible to realize such a configuration?

I need to manage the default ownership/permissions of a mountpoint created by a PVC with ReadWriteMany (RWX) on Red Hat Openshift 4.13, currently not supported by Nutanix CSI volume driver.According to Red Hat, using the current setting of fsGroupPolicy: ReadWriteOnceWithFSType results in the fsGroup only to be respected by the driver if the volume's access mode contains ReadWriteOnce (RWO).According to this link, after changing the fsGroupPolicy parameter to value ‘File’, Kubernetes may use fsGroup to change permissions and ownership of the volume to match user requested fsGroup in the pod's SecurityPolicy regardless of fstype or access mode.https://kubernetes-csi.github.io/docs/support-fsgroup.htmlDoes anyone knows how to approach this?

We noticed that Kubernetes is still in beta version on Prism central. The latest 1.25.6 version is generating an unuseable kubeconfig file which prevents any possible management of the cluster. Also the user rights and roles to create a Kubernetes cluster seem to be mangled which limits the cluster creation to the admin user only. Are there any user who implemented their own Kubernetes cluster by VM's ? 

Hi  My cluster is 1.19.8-0 and I want it to be upgraded to 1.26. When I click on upgrade button I can only see 1.19.16-0 but not anything else. How can I get to 1.26 version

We have setup an NKE with Rancher as the Container Management and Orchestration. We have also enabled ARC Kubernetes in Azure and enabled the communication between Azure and On Prem Nutanix. We have a strict requirement for security and require Azure Defender for Containers to scan the clusters. The Defender has identified some recommendations on the Nutanix Kubernetes as medium to high in terms of security configurations.Is there any documentation on which recommendations to suppress due to the nature of NKE vs actual recommendations? We have a "clean install" (but for rancher and ARC) and the recommendations are related to nutanix pods.

Hello,I got an issue with calico pods:Warning  Unhealthy  9m53s (x3 over 10m)  kubelet, karbon  Liveness probe failed: calico/node is not ready: bird/confd is not live: Service bird is not running. Output << down: /etc/service/enabled/bird: 0s, normally up, want up >>  Warning  Unhealthy  7m3s (x12 over 10m)  kubelet, karbon-  Liveness probe failed: calico/node is not ready: bird/confd is not live: Service bird is not running. Output << down: /etc/service/enabled/bird: 1s, normally up, want up >>  Warning  Unhealthy  2m3s (x50 over 10m)  kubelet, karbon-  Readiness probe failed: calico/node is not ready: BIRD is not ready: Error querying BIRD: unable to connect to BIRDv4 socket: dial unix /var/run/calico/bird.ctl: connect: connection refused I tried different workarounds:increasing scan time from 2 to 10 then 20. Modify the “IP_AUTODETECTION_METHOD=interface” from eth.* to eth0/eth1/ens.*But no luck, Would you have any suggestions to resolve the issue.Thanks.

HelloI wanted to test Advanced Kubernetes Management but unfortunately I didn't understand how it works and I deleted my cluster on which I had activated it. Now I can't create a new cluster because I have an api error that is unreachable (this is normal since my cluster was deleted). Is it possible to reinstall properly on a new cluster or to disable it completely ? Thanks

Is there any reason or use of installing NGT Tools on k8s “vm/node”?

Hi guys,Currently working on Ansible  to deploy K8s clusters through Karbon and i got a fatal issue.Ansible is version core 2.12.9, PC is 2022.1, nutanix.ncp collection has been tested in version 1.6.0 and 1.7.0 and there’s absolutely no traffic denied between ansible and PC (to sum up, everything is opened). I use this playbook, based on example provided in ansible-doc (i also tried the example, and i got the same errors in any cases) ###EDIT : i can’t add code in this post, got a banner Something gone wrong everytime ### Here’s the logs when the playbook fails (after ~5min running, from PC, ETCD deployment get stuck at 8%) :  ###EDIT : i can’t add code in this post, got a banner Something gone wrong everytime ###Moreover, K8s cluster deployment through PC GUI is working smoothly. Does anyone got an idea to fix the “failed to deploy ntnx dvp” error ?  Thanks a lot :)  Gael

HiI’m currently working on RBAC rights for Prism Central but also for entities (VM,Apps...)In my understanding and in the documentation, i saw that you have to be User Admin on PC to get full rights on K8s cluster. In that statement, how does Karbon acts on K8s cluster rights while it deploy a new cluster ? (For instance, if you are a Viewer on PC you can’t connect to K8s cluster, if you’re User Admin, you can do whatever you want. ) Best regards Gael

error from my side, please delete this post, sorry

“Kubernetes deployments are inherently dynamic and challenging to manage at scale,” said Thomas Cornely, SVP, Product Management, Nutanix. “Running Kubernetes container platforms cost-effectively at large scale requires developer-ready infrastructure that seamlessly adapts to changing requirements. Our expertise in simplifying infrastructure management while optimizing resourcesーboth on-premises and in the public cloudーis now being applied to help enterprises adopt Kubernetes more quickly. The Nutanix Cloud Platform now supports a broad choice of Kubernetes container platforms, provides integrated data services for modern applications, and enables developers to provision Infrastructure as Code.”According to Gartner, by 2027, 25% of all enterprise applications will run in containers, an increase from fewer than 10% in 2021. This is a significant challenge for many given most Kubernetes solutions are not meant to support enterprise scale, even less can do so in a manner that is cost effe

Learn how to add GPUs to Nutanix Kubernetes Engine (NKE) node pools for running AI/ML workloads in the latest instalment of the AHV Mission Control Series.