Nutanix Kubernetes Engine (NKE) | Nutanix Community
Skip to main content

Nutanix Kubernetes Engine (NKE)

Kubernetes Management Made Simple

  • 115 Topics
  • 309 Replies

Backing Up Karbon Kubernetes clusters with Velero and Nutanix Objects

  • 4
  • 0

Latest Nutanix Karbon How-To Videos

  • 2
  • 2

Tillerless Helm on Nutanix Karbon

  • 9
  • 2
115 Topics
F
Fabio-2024Voyager
asked in Nutanix Kubernetes Engine (NKE)
Failed deployment of a production K8s cluster

Hello,I'm currently experiencing a problem deploying a k8s production cluster but I'm able to deploy the dev clusters.I get this error when deploying:Cluster ha-cluster:failed to deploy K8s cluster: failed to deploy master nodes: failed to deploy control plane nodes: Operation timed out: 0: 0If anyone has encountered or knows the source of this problem, I'm interested in the solutionI'm available if you have any further questions or need supplements, thank you in advance for your help.

340
F
G
GiuseppePinoAdventurer
asked in Nutanix Kubernetes Engine (NKE)
Reduce NKP managment cluster resource

Hi,I deployed NKP environment with default config, so I have 4 control plane nodes and 3 worker nodes. Can I reduce the nodes on NKP management cluster? or can I safely remove the cluster and re-deploy?Thanks

571
G
G
Nutanix Employee
guybrush77Nutanix Employee
asked in Nutanix Kubernetes Engine (NKE)
Info NKM

Good morning everyone, I'm clearing the communication rules between the nutanix clusters using the matrix provided. I came across this communication:SOURCE: HTTPS 30001 TCP - DESTINATION: Prism Central NKM How can I check what the NKM IP is or check where it is enabled. We have Nke but I can't find anything about nkm.Thank you all

253
G
Nutanix Employee
E
Nutanix Employee
elyacoubiNutanix Employee
asked in Nutanix Kubernetes Engine (NKE)
setting up a NKE cluster with external load balancer

I am trying to set up an NKE cluster with an external load balancer (HAProxy). When I attempt to do so, I encounter this error related to the SSL certificate:Cluster k8s-cluster: failed to deploy K8s cluster: failed to deploy master nodes: failed to deploy control plane nodes: Operation timed out: Get "https://10.38.9.165:443/api/v1/nodes/k8s-cluster-2eb745-master-0?timeout=2m0s": x509: certificate signed by unknown authority: 0: 0 Is there any documentation on how to set up an NKE cluster with an external load balancer?How can I make the certificate trusted by NKE?FYI, I am using a self-signed certificate.Here is my HAProxy configuration: #---------------------------------------------------------------------# Example configuration for a possible web application. See the# full configuration options online.## https://www.haproxy.org/download/1.8/doc/configuration.txt##---------------------------------------------------------------------#-----------------------------------------------

1222
E
Nutanix Employee
E
Nutanix Employee
elyacoubiNutanix Employee
asked in Nutanix Kubernetes Engine (NKE)
Setting up NKE cluster with external laod balancer

in my learning environmentI am trying to set up an NKE cluster with an external load balancer (HAProxy). When I attempt to do so, I encounter this error related to the SSL certificate:vbnetCluster k8s-cluster: failed to deploy K8s cluster: failed to deploy master nodes: failed to deploy control plane nodes: Operation timed out: Get "https://10.38.9.165:443/api/v1/nodes/k8s-cluster-2eb745-master-0?timeout=2m0s": x509: certificate signed by unknown authority: 0: 0 How can I make the certificate trusted by NKE?FYI i am using self signed certificateHere is my HAProxy configuration: #---------------------------------------------------------------------# Example configuration for a possible web application. See the# full configuration options online.## https://www.haproxy.org/download/1.8/doc/configuration.txt##---------------------------------------------------------------------#---------------------------------------------------------------------# Global settings#-----------------------

80
E
Nutanix Employee
R
RoelofvdWWayfarer
asked in Nutanix Kubernetes Engine (NKE)
Kubeconfig File Not Working for Cluster Info

I am running Kubernetes management 1.28.5-1 and installed a basic production cluster. After it all installed and runs correctly I tried to use the Kubeconfig file to get cluster info via an other linux server but it fails every time.kubectl cluster-infoE0820 08:30:40.983751  237406 memcache.go:265] "Unhandled Error" err="couldn't get current server API group list: the server has asked for the client to provide credentials" the contents of the config file:cat $KUBECONFIG# -*- mode: yaml; -*-# vim: syntax=yaml#apiVersion: v1kind: Configclusters:- name: "my-kube-prd-002"  cluster:    server: https://10.249.8.2:443    certificate-authority-data: LS0<redacted>users:- name: default-user-my-kube-prd-002  user:    token: eyJ0<redacted>contexts:- context:    cluster: "my-kube-prd-002"    user: default-user-my-kube-prd-002  name: my-kube-prd-002-contextcurrent-context: my-kube-prd-002-context  How to proceed from here? this manual https://portal.nutanix.com/page/documents/details?tar

35931
F
J
jianghui8-65135Explorer
asked in Nutanix Kubernetes Engine (NKE)
Failed to update karbon using lcm

 aos 6.5.5.7prism central ：2022.6.0.10karbon version 2. 2. 3.I enabled karbon,I failed to upgrade karbon using lcm with the following error message: 2024-08-31 17:03:20,326Z ERROR 46603952 helper.py:145 (192.168.30.28, update, aff9f64a-6e19-41a4-5fda-68e7f423c861) 2024-08-31 10:03:20.231345 EXCEPT:{"err_msg":"Update failed with error: [Failed to initialize precheck helper: Failed to initialize Nutanix Kubernetes Engine interface: Failed to build Nutanix Kubernetes Engine session: Failed to get auth session cookie: Failed to make GET request: URL: https://192.168.30.28:9440/api/nutanix/v3/users/me, the error: HTTPSConnectionPool (host ='192.168.30.28', port=9440): Read timed out. (read timeout=30)]", "name": "relea.karbon.update ", "stage": 0} I executed the karbon login command successfully, but karbonctl airgap list failednutanix@NTNX-192-168-30-28-A-PCVM:~/data/logs$ /home/nutanix/karbon/karbonctl login --pc-username adminPlease enter the password for the PC user: adminLogin successf

500
J
R
RajanBVoyager
asked in Nutanix Kubernetes Engine (NKE)
expose pod in Kubernetes engine as service.

HiI have a pod with cluster IP already. I want to expose this as load balacner or nodeport service and take one of the private IP from the subnet. However none seem to be working. I just want to know if there is some annotations or additional settings that we need to do to achieve this or exposing pods as service via standard yaml or kubectl commands should work?for example this - kubectl expose deployment sandbox-airflow-web --type=LoadBalancer --name=sandbox-airflow-web-service --namespace=sandbox-airflowdidnt generate any external IP or errors. was stuck in pending. I later patched it to get an IP but there IP didnt return the website.Kubectl patch svc sandbox-airflow-web-service -n sandbox-airflow -p'{"spec": {"type": "LoadBalancer", "externalIPs":["172.20.8.50"]}}'thanks.

662
R
G
GeeWillikasPathfinder
asked in Nutanix Kubernetes Engine (NKE)
Disaster recovery for Nutanix Kubernetes Environment

We have a Nutanix nke environment and would like to activate protected domains so that in the event of a disaster we can move the PD across to our DR site. Currently I have not seen any documentation or a way to configure this. Does anyone know how we can “replicate” our NKE clusters to a DR location in an easy and simple way. My thought is to take the underlying virtual machines and just put them into a PD. However, it does not seem as simple as this. Any guidance would be apricated. We have been receiving conflicting advice from Nutanix technical support. To the point where it does not seem possible and therefore makes NKE not viable to enterprise deployment. 

1721
Y
Nutanix Employee
S
Sandeep MelepunathilTrailblazer
asked in Nutanix Kubernetes Engine (NKE)
Do we have duplicate IP check in the Karbon / NKE cluster creation process?

We hit the below scenario today. We have a single IP pool IPAM network pool included the CVM/Host IPs When deployed a K8 cluster using this IPAM network, system generated IP list contains 2 IPs of CVM Cluster creation task proceeded without any issue After the K8 cluster creation task , nutanix cluster went down , due to the duplicate IP issueDo we have any check introduced in code for duplicate ip check in the creation task?

604
S
Y
YerbolWayfarer
asked in Nutanix Kubernetes Engine (NKE)
ipvs mode for kube-proxy with MetalLB on NKE

I'm in the process of installing MetalLB on Nutanix Kubernetes Engine (NKE) to provide LoadBalancer services. According to MetalLB's documentation, it prefers (requires) kube-proxy to run in ipvs mode for optimal performance. However, I've noticed that NKE defaults to iptables mode for kube-proxy, and my attempts to switch to ipvs mode have been unsuccessful.Has anyone successfully configured ipvs mode for kube-proxy on NKE? If so, could you share the steps or guidance on how to achieve this? Any tips or resources would be greatly appreciated as the standard documentation and methods don't seem to apply directly to NKE.I referred to this blog: Utilizing MetalLB to Provide LoadBalancer Services for Nutanix Karbon.

2811
JoseNutanix
Nutanix Employee
M
melbeririVoyager
asked in Nutanix Kubernetes Engine (NKE)
What are the Kubernetes versions supported in NKE as of today

I was testing the NKE cluster using Test Drive and noticed that when attempting to create a cluster, the latest version available is 1.19.8. I'm curious if this limitation also applies to the regular NKE clusters, or if NKE supports newer versions beyond 1.19.8. and if so what is the latest version?

2151
JoseNutanix
Nutanix Employee
M
melbeririVoyager
asked in Nutanix Kubernetes Engine (NKE)
Can NKE be installed in a non-nutanix infrastructure?

We are presently engaged with a client who possesses a Nutanix infrastructure and seeks to leverage Nutanix's NKE for deploying their application. Our intention is to utilize their infrastructure as the production setup while concurrently exploring the implementation of an NKE cluster within our VMs, provisioned on a Proxmox server. Despite searching for similar setups, we haven't found relevant precedents. Is it feasible to realize such a configuration?

1433
JoseNutanix
Nutanix Employee
M
Michel SchildmeijerScout
asked in Nutanix Kubernetes Engine (NKE)
Nutanix CSI volume driver.RWX support

I need to manage the default ownership/permissions of a mountpoint created by a PVC with ReadWriteMany (RWX) on Red Hat Openshift 4.13, currently not supported by Nutanix CSI volume driver.According to Red Hat, using the current setting of fsGroupPolicy: ReadWriteOnceWithFSType results in the fsGroup only to be respected by the driver if the volume's access mode contains ReadWriteOnce (RWO).According to this link, after changing the fsGroupPolicy parameter to value ‘File’, Kubernetes may use fsGroup to change permissions and ownership of the volume to match user requested fsGroup in the pod's SecurityPolicy regardless of fstype or access mode.https://kubernetes-csi.github.io/docs/support-fsgroup.htmlDoes anyone knows how to approach this?

2961
JoseNutanix
Nutanix Employee
R
RoelofvdWWayfarer
asked in Nutanix Kubernetes Engine (NKE)
Beta Kubernetes versions

We noticed that Kubernetes is still in beta version on Prism central. The latest 1.25.6 version is generating an unuseable kubeconfig file which prevents any possible management of the cluster. Also the user rights and roles to create a Kubernetes cluster seem to be mangled which limits the cluster creation to the admin user only. Are there any user who implemented their own Kubernetes cluster by VM's ? 

810
R
N
NikhileswarScout
asked in Nutanix Kubernetes Engine (NKE)
Kubernetes 1.26

Hi  My cluster is 1.19.8-0 and I want it to be upgraded to 1.26. When I click on upgrade button I can only see 1.19.16-0 but not anything else. How can I get to 1.26 version

2511
JoseNutanix
Nutanix Employee
F
FishFacePathfinder
asked in Nutanix Kubernetes Engine (NKE)
ARC Kubernetes Cluster, NKE and Azure Defender for containers.

We have setup an NKE with Rancher as the Container Management and Orchestration. We have also enabled ARC Kubernetes in Azure and enabled the communication between Azure and On Prem Nutanix. We have a strict requirement for security and require Azure Defender for Containers to scan the clusters. The Defender has identified some recommendations on the Nutanix Kubernetes as medium to high in terms of security configurations.Is there any documentation on which recommendations to suppress due to the nature of NKE vs actual recommendations? We have a "clean install" (but for rancher and ARC) and the recommendations are related to nutanix pods.

1451
JoseNutanix
Nutanix Employee
A
AmineBiExplorer
asked in Nutanix Kubernetes Engine (NKE)
calico pods does not started

Hello,I got an issue with calico pods:Warning  Unhealthy  9m53s (x3 over 10m)  kubelet, karbon  Liveness probe failed: calico/node is not ready: bird/confd is not live: Service bird is not running. Output << down: /etc/service/enabled/bird: 0s, normally up, want up >>  Warning  Unhealthy  7m3s (x12 over 10m)  kubelet, karbon-  Liveness probe failed: calico/node is not ready: bird/confd is not live: Service bird is not running. Output << down: /etc/service/enabled/bird: 1s, normally up, want up >>  Warning  Unhealthy  2m3s (x50 over 10m)  kubelet, karbon-  Readiness probe failed: calico/node is not ready: BIRD is not ready: Error querying BIRD: unable to connect to BIRDv4 socket: dial unix /var/run/calico/bird.ctl: connect: connection refused I tried different workarounds:increasing scan time from 2 to 10 then 20. Modify the “IP_AUTODETECTION_METHOD=interface” from eth.* to eth0/eth1/ens.*But no luck, Would you have any suggestions to resolve the issue.Thanks.

5150
A
R
Raph123Voyager
asked in Nutanix Kubernetes Engine (NKE)
Deactivate/reinstall Kubernetes Management on Prism Central post-cluster deletion

HelloI wanted to test Advanced Kubernetes Management but unfortunately I didn't understand how it works and I deleted my cluster on which I had activated it. Now I can't create a new cluster because I have an api error that is unreachable (this is normal since my cluster was deleted). Is it possible to reinstall properly on a new cluster or to disable it completely ? Thanks

5083
R
P
PerhVoyager
asked in Nutanix Kubernetes Engine (NKE)
NGT tools for Karbon

Is there any reason or use of installing NGT Tools on k8s “vm/node”?

862
P
G
gael01Trendsetter
asked in Nutanix Kubernetes Engine (NKE)
Ansible K8s cluster deployment failed to deploy the ntnx dvp

Hi guys,Currently working on Ansible  to deploy K8s clusters through Karbon and i got a fatal issue.Ansible is version core 2.12.9, PC is 2022.1, nutanix.ncp collection has been tested in version 1.6.0 and 1.7.0 and there’s absolutely no traffic denied between ansible and PC (to sum up, everything is opened). I use this playbook, based on example provided in ansible-doc (i also tried the example, and i got the same errors in any cases) ###EDIT : i can’t add code in this post, got a banner Something gone wrong everytime ### Here’s the logs when the playbook fails (after ~5min running, from PC, ETCD deployment get stuck at 8%) :  ###EDIT : i can’t add code in this post, got a banner Something gone wrong everytime ###Moreover, K8s cluster deployment through PC GUI is working smoothly. Does anyone got an idea to fix the “failed to deploy ntnx dvp” error ?  Thanks a lot :)  Gael

2401
G
G
gael01Trendsetter
asked in Nutanix Kubernetes Engine (NKE)
Karbon and Prism Central Management for K8s Cluster

HiI’m currently working on RBAC rights for Prism Central but also for entities (VM,Apps...)In my understanding and in the documentation, i saw that you have to be User Admin on PC to get full rights on K8s cluster. In that statement, how does Karbon acts on K8s cluster rights while it deploy a new cluster ? (For instance, if you are a Viewer on PC you can’t connect to K8s cluster, if you’re User Admin, you can do whatever you want. ) Best regards Gael

2865
mikkisse
G
gael01Trendsetter
asked in Nutanix Kubernetes Engine (NKE)
qqq

error from my side, please delete this post, sorry

450
G
aluciani
alucianiChevalier
posted in Nutanix Kubernetes Engine (NKE)
Accelerating Enterprise Adoption of Kubernetes With Nutanix

“Kubernetes deployments are inherently dynamic and challenging to manage at scale,” said Thomas Cornely, SVP, Product Management, Nutanix. “Running Kubernetes container platforms cost-effectively at large scale requires developer-ready infrastructure that seamlessly adapts to changing requirements. Our expertise in simplifying infrastructure management while optimizing resourcesーboth on-premises and in the public cloudーis now being applied to help enterprises adopt Kubernetes more quickly. The Nutanix Cloud Platform now supports a broad choice of Kubernetes container platforms, provides integrated data services for modern applications, and enables developers to provision Infrastructure as Code.”According to Gartner, by 2027, 25% of all enterprise applications will run in containers, an increase from fewer than 10% in 2021. This is a significant challenge for many given most Kubernetes solutions are not meant to support enterprise scale, even less can do so in a manner that is cost effe

1080
aluciani
aluciani
alucianiChevalier
posted in Nutanix Kubernetes Engine (NKE)
Adding GPUs to Nutanix Kubernetes Engine Node Pools

Learn how to add GPUs to Nutanix Kubernetes Engine (NKE) node pools for running AI/ML workloads in the latest instalment of the AHV Mission Control Series. 

910
aluciani

Badges

  • Courteous
    ttenglinghas earned the badge Courteous
  • Courteous
    Haroonhas earned the badge Courteous
  • Neighborly
    Maarkhas earned the badge Neighborly
  • Courteous
    MarcoThas earned the badge Courteous
  • Courteous
    niclasskovhas earned the badge Courteous
Show all badges
Terms & Conditions