Nutanix Kubernetes Engine (NKE) | Nutanix Community
Skip to main content

Nutanix Kubernetes Engine (NKE)

Kubernetes Management Made Simple

  • 115 Topics
  • 309 Replies

Backing Up Karbon Kubernetes clusters with Velero and Nutanix Objects

  • 4
  • 0

Latest Nutanix Karbon How-To Videos

  • 2
  • 2

Tillerless Helm on Nutanix Karbon

  • 9
  • 2
115 Topics
A
ASHISHREDDYVoyager
asked in Nutanix Kubernetes Engine (NKE)
Upgrade kubernetes version from 1.20.x to 1.21.x

Hello , I have a kubernetes Cluster on Nutanix with 1 Master Node and 2 Worker Nodes(Test ENV). All the Nodes have Kubernetes version of 1.20.x. I want to Update the all the Nodes kubernetes version to 1.21.x. without the Downtime of application/services running on it. I have Seen in Nutanix guide which is easy step to upgrade kubernetes version but it says application may have downtime. Any Ideas and Inputs to upgrade Kubernetes version from 1.20.x to 1.21.x withoutdown of apps.

8153
J
B
BhavenWayfarer
asked in Nutanix Kubernetes Engine (NKE)
Karbon Kubernetes version upgrade and OS image

I have Community Edition and I can only see "ntnx-0.5" image due to which I am stuck at Kubernetes v1.16. (Karbon 2.0.2). My objective is to upgrade Kubernetes to 1.17 or 1.8. LCM does not display any software to upgrade. I have uploaded one more image "1.0" under images in PC as below - which I understand is 1st step before K8S upgrade. However, I cant not get ntnx-1.0 image under Karbon download Could someone please advise on this? Thanks

6302
K
V
VaccanoAdventurer
asked in Nutanix Kubernetes Engine (NKE)
Update on Windows support for Karbon

I would like to know if support for Windows nodes has made any progress toward being prioritized for Karbon. We run a product called OutSystems and would like to move that workload to Nutanix Karbon. But OutSystems containers only work with Windows Nodes. I asked this question a year ago and was told that it was on the backlog. Could someone from the Karbon product team provide an update on the backlog item to support Windows Nodes in a Karbon based Kuberentes cluster?

2850
V
X
xiaoweivip2020Wayfarer
asked in Nutanix Kubernetes Engine (NKE)
Nutanix Volume Driver the Nutanix Docker Volume Plugin

There are many words about plugin making me confused. like the Nutanix Volume Driver 、the Nutanix Docker Volume Plugin、volume driver plugin Does anyone know the difference of this ?

1530
X
T
Theo BotVoyager
asked in Nutanix Kubernetes Engine (NKE)
Karbon Kubernetes CronJob

When I run the command “kubctl api-resources”, I get a list of all available resources types. This list includes the CronJob type in the apigroup batch. However, when I want to deploy a cronjob I get the message: no matches for kind “CronJob” in version “batch/v1”. apiVersion: batch/v1kind: CronJobmetadata: name: devel-sched namespace: devel Does anybody know, how to fix this?

2651
T
B
bandradeshipaVoyager
asked in Nutanix Kubernetes Engine (NKE)
Karbon cluster IP

Hello folks! I just signed up for a SaaS test of Nutanix Karbon. After creating the cluster and downloading the kubeconfig, I see it gives me an IP 172.31.x.x:443 After downloading the kubeconfig file and running a kubectl get nodes, it times out because it can’t reach the cluster IP What should I do in order to expose the cluster IP and try installing/running our applications on Karbon?

1893
JoseNutanix
Nutanix Employee
B
BhavenWayfarer
asked in Nutanix Kubernetes Engine (NKE)
Kubernetes Version Upgrade and OS Image

Hi All, I have Community Edition and I can only see "ntnx-0.5" image due to which I am stuck at Kubernetes v1.16. (Karbon 2.0.2). My objective is to upgrade Kubernetes to 1.17 or 1.8. LCM does not display any software to upgrade. I have uploaded one more image "1.0" under images in PC as below - which I understand is 1st step before K8S upgrade. However, I cant not get ntnx-1.0 image under Karbon download Could someone please advise on this? Thanks

1390
B
AllenLiu
AllenLiuTrailblazer
asked in Nutanix Kubernetes Engine (NKE)
Can't Download Karbon "Centos Image" on Nutanix CE

We want to test Nutanix's Karbon, when I click the download centos button, nothing happens.

4245
AllenLiu
Y
YongjuVoyager
asked in Nutanix Kubernetes Engine (NKE)
karbon install

If you check the carbon setting document on the Nutanix homepage, it is as follows. https://portal.nutanix.com/page/documents/details?targetId=Karbon-v2_2:kar-karbon-airgap-deploy-t.html There is a WEB server (192.168.100.99) and prism central (192.168.100.45) that allow you to download carbon files by following the above method. When entering the command in Prism central CLI, it was successfully confirmed, but when checking in Prism central WEB, it is confirmed as tasks fail. Please tell me another way after confirmation ASAP

4081
JoseNutanix
Nutanix Employee
Y
YongjuVoyager
asked in Nutanix Kubernetes Engine (NKE)
Karbon licensing

I understand that the Nutanix Karbon License is free to use. I have an AOS Pro License, but the OS image cannot be downloaded because of the error below. Please guide me on how to solve it. Thank you.

12685
JoseNutanix
Nutanix Employee
E
eriklundPathfinder
asked in Nutanix Kubernetes Engine (NKE)
Karbon: Customize kube-let reserved memory

Karbon always allocates 400MB for kube-let and other node resources. However, this is not always enough and kube-let can run out-of-memory when system is under load. When kube-let runs out of memory then we get pods stranded in “Terminating” state and we have to reboot the node. What I did expect was the kube-let terminated pods that got out-of-memory and that kube-let would not run out of memory. Typical 1-1.4MB are reserved at nodes with 16GB memory in other Kubernetes clusters (EKS/AKS/GKE/...) See also https://kubernetes.io/docs/tasks/administer-cluster/reserve-compute-resources/ Currently we have added extra nodes and hope that we will not get out-of-memory on any nodes. However, that is not what Kubernetes is designed. Any work-a-round for adjusting the 400MB fixed size reservation?

2071
crisj
Nutanix Employee
I
IgorStankovicAdventurer
asked in Nutanix Kubernetes Engine (NKE)
How can I add a SAN to the Kubernetes API certificate?

Having deployed a new Kubernetes cluster with Karbon and downloaded the kubeconfig, kubectl operations result in an error: Unable to connect to the server: x509: certificate is valid for cluster.local, kubernetes, kubernetes.default, kubernetes.default.svc, kubernetes.default.svc.cluster, kubernetes.default.svc.cluster.local, cluster.local, not xxxxx What is the recommended way to add a SAN to the Karbon Kubernetes API certificate?

3201
JoseNutanix
Nutanix Employee
I
IgorStankovicAdventurer
asked in Nutanix Kubernetes Engine (NKE)
Deployment issue with Karbon 2.3 and Calico CNI

Hi there, It seems that I have a failure near the very end of deployment. Here is the output from karbon_core.out 2021-10-17T11:57:31.052Z kube_prometheus.go:1016: [DEBUG] [k8s_cluster=RGS-PA-K8-CLUSTER-STAGING] expecting 5 nodes to be running calico-node daemon pod in kube-system namespace. Currently running: 42021-10-17T11:57:33.093Z kube_prometheus.go:1016: [DEBUG] [k8s_cluster=RGS-PA-K8-CLUSTER-STAGING] expecting 5 nodes to be running calico-node daemon pod in kube-system namespace. Currently running: 42021-10-17T11:57:35.135Z kube_prometheus.go:1016: [DEBUG] [k8s_cluster=RGS-PA-K8-CLUSTER-STAGING] expecting 5 nodes to be running calico-node daemon pod in kube-system namespace. Currently running: 42021-10-17T11:57:36.806Z calico.go:552: [ERROR] [k8s_cluster=RGS-PA-K8-CLUSTER-STAGING] Failed to verify calico addon2021-10-17T11:57:36.806Z k8s_deploy.go:1478: [ERROR] [k8s_cluster=RGS-PA-K8-CLUSTER-STAGING] Failed to deploy calico/flannel: Failed to deploy calico: Failed to verify cali

5875
I
M
Moises S FAdventurer
asked in Nutanix Kubernetes Engine (NKE)
Cluster Docker swarm + Volume Plugin

Hi, I have performed tests of the plugin for integration between Volume Groups in a Nutanix cluster with Dockers containers (https://next.nutanix.com/karbon-kubernetes-orchestration-30/nutanix-dvp-docker-volume-plug-in-25371) and (https://next.nutanix.com/karbon-kubernetes-orchestration-30/docker-nutanix-container-volume-plug-in-18726). While reading the Docker Volume Plugin documentation at (https://portal.nutanix.com/page/documents/solutions/details?targetId=BP-2037-Docker-Containers-on-AHV:BP-2037-Docker-Containers-on-AHV), I did not notice if such a plugin works with Docker containers in swarm mode (Docker swarm cluster). I performed some tests with Ubuntu 20.04, forming a Docker swarm cluster with 03 nodes, all VMs with Nutanix Guest Tools installed. All the tests using .yml files for the creation of stacks and services, as for the volumes the tests were made both with the command to create the volumes, as well as in the .yml files. I noticed that the volumes are created normally

5261
JoseNutanix
Nutanix Employee
J
jourosScout
asked in Nutanix Kubernetes Engine (NKE)
Airgap for K8s external components

Is it possible to use Airgap to store additional K8s components like Flux, Ingress controller etc.? I’m using terraform to deploy Karbon, so I should be able to access airgap registry from those deployment modules. How do I manage airgap registry, is it standard private container registry? How about helm charts, can airgap registry store them also?

1550
J
M
Moises S FAdventurer
asked in Nutanix Kubernetes Engine (NKE)
Karbon, Rancher and similars

Hello friends, From what I have read in the Karbon documentation, as well as from YouTube videos, through karbon in AHV I cannot manage aspects of the containers, the applications themselves, load balancing, reverse proxy, container images, etc. Is that right?Associated with this, if Karbon does not in fact have such features, is it possible to use Karbon in conjunction with tools that have the above features, like Rancher? Has anyone had this experience?

8601
JoseNutanix
Nutanix Employee
R
RanuAdventurer
asked in Nutanix Kubernetes Engine (NKE)
Kubernetes Cluster deployment failing

We have enable karbon on a 3-node nutanix cluster but kubernetes deployment is failing with below error. Any ideas? 2021-04-06T07:58:33.103Z k8s_deploy.go:167: [error] [k8s_cluster=Ranu-Demo] failed to create worker node pool and deploy worker nodes(VMs): failed to deploy K8s cluster: failed to deploy worker nodes: Operation timed out 2021-04-06T07:58:33.103Z k8s_deploy.go:882: [error] [k8s_cluster=Ranu-Demo] Error seen in configuring worker node: "Operation timed out" 2021-04-06T07:58:33.125Z k8s_lib_deploy_task.go:113: [error] [k8s_cluster=Ranu-Demo] failed to deploy K8s cluster: failed to create worker node pool and deploy worker nodes(VMs): failed to deploy K8s cluster: failed to deploy worker nodes: Operation timed out 2021-04-06T07:58:33.125Z k8s_lib_deploy_task.go:78: [info] [k8s_cluster=Ranu-Demo] token refresher received stopRefresh 2021-04-06T07:58:36.03Z deploy_k8s_task.go:365: [error] [k8s_cluster=Ranu-Demo] Cluster Ranu-Demo:failed to deploy K8s cluster: failed to create w

6241
maaz
Nutanix Employee
A
Alan CVoyager
posted in Nutanix Kubernetes Engine (NKE)
Permanent karbon kubeconfig file

If you are annoyed, as myself, about renewing the kubeconfig every 24 hours, i’ve created a simple script that generates a permanent kubeconfig file without expiration. This script makes sense, obviously, if you have more than 1 karbon cluster to manage. #!/bin/bashif [ -n "$1" ]; then echo Cluster name: $1.else echo "No cluster name has been specified. " exit 1fikubectl create serviceaccount --namespace kube-system superuserkubectl create clusterrolebinding superuser-rule --clusterrole=cluster-admin --serviceaccount=kube-system:superuser A=$(kubectl -n kube-system describe secret/$(kubectl -n kube-system get sa/superuser -o jsonpath='{.secrets[0].name}')|awk '{for(i=1;i<=NF;i++)if($i=="token:")print $(i+1)}')B=$(kubectl config view --flatten --minify|awk '{for(i=1;i<=NF;i++)if($i=="certificate-authority-data:")print $(i+1)}')C=$(kubectl config view --flatten --minify|awk '{for(i=1;i<=NF;i++)if($i=="server:")print $(i+1)}')D=$(kubectl config view --flatten --minify|awk '{for(i=1;i<=NF;

11104
A
UPX
UPXOutrider
asked in Nutanix Kubernetes Engine (NKE)
Add existing Karbon Cluster to a new Prism Central Instance

Hi everybody, the question is, as i’d to reinstall a new PC Instance (the old one crashed), is there a way to import the configuration of a previously deployed Karbon Cluster in the new PC instance? I would like to manage the old cluster with the new PC. Thank you all

2352
UPX
W
WanexaVoyager
asked in Nutanix Kubernetes Engine (NKE)
Karbon API need new stuffs

Is there a way to retrieve an image OS and k8s version in the api in the future please, need for the ci/cd and deployment.

771
JoseNutanix
Nutanix Employee
V
VaccanoAdventurer
asked in Nutanix Kubernetes Engine (NKE)
Karbon errors can't see anything wrong in Prism Element

I have a Dev Nutanix Cluster that I use to host Kubernetes clusters. Today I added a large Kuberentes cluster to it (I intended to add it to my Prod Nutanix Cluster, but accidentally added to my Dev Nutanix Cluster). I started to get a lot of alerts from 2 of the 3 Kubernetes clusters hosted on the Dev Nutanix Cluster. As soon as the install was complete, I deleted the large Kubernetes cluster from the Dev Nutanix Cluster. I figured that since the resources have all been freed up, the cluster would be fine. But I am still getting errors when using `kubectl`. The two most common are: Error from server: etcdserver: request timed out and Unable to connect to the server: dial tcp 10.62.12.91:443: connectex: No connection could be made because the target machine actively refused it. 10.62.12.91 is my control plane host. I have tried to reboot both the etcd server and my control plane host. That did not fix the issue. I loaded up the Dev Nutanix Cluster in Prim Element and it does not give a

5741
JoseNutanix
Nutanix Employee
S
senthil_pExplorer
asked in Nutanix Kubernetes Engine (NKE)
Karbon Cluster with static IP

its there any way to create karbon cluster with static ip ? if Yes then how its possible.. default worker node and master nodes are assigned with DHCP ip pools .. we cant mange IPs address. please help us .

2801
JoseNutanix
Nutanix Employee
clindevall
clindevallVoyager
posted in Nutanix Kubernetes Engine (NKE)
Renewing cluster kubeconfig from the command line

Just wanted to share how we are renewing the kubeconfig from the command line. Background: In our organization it was considered a bit clumsy to download the kubeconfig from the Karbon UI. We love the command line and wanted a quick’n’easy way to renew the authentication token. The solution: We created a docker image that contains the “karbonctl” binary and a wrapper script. The Dockerfile was also included into our CI/CD pipeline to automate the image creation. The wrapper script does two things: karbonctl login --pc-ip prism.nightingale.nu --pc-username “$1” karbonctl cluster kubeconfig --cluster-name “$2” >”$3” The commands that the end user run: docker run --rm -it -v “$(pwd)”:/tmp docker.registry.local/karbonctl:latest <login> mycluster /tmp/karbon.cfg export KUBECONFIG=”$(pwd)/karbon.cfg”

444814
mikkisse
V
VaccanoAdventurer
asked in Nutanix Kubernetes Engine (NKE)
Is Support for Windows (along side Linux) Support Planned for Karbon?

I have seen several posts that indicate that Karbon does not support Windows Containers. But I have not heard if that is on the roadmap or not. Is it on the roadmap? Being considered? Has the idea been considered and discarded? I am interested in what ever information you are able to share about future plans for Karbon in relation to mixed workloads.

2673
L
Nutanix Employee
V
VaccanoAdventurer
asked in Nutanix Kubernetes Engine (NKE)
In Karbon are all users the same user?

When I get a kubeconfig file from Karbon, the user is always default-user-MyClusterNameHere in the kubeconfig file. I am trying to setup RBAC permissions for various users as indicated on your post here: https://next.nutanix.com/community-blog-154/providing-rbac-for-your-karbon-kubernetes-clusters-33132 But the user is always the same user (default-user-MyClusterNameHere). How can I get a user logged in as the actual user that they are? And more, how can I get the groups that my users are in so I do not have to assign permissions to specific users? (We use Active Directory.) Update After a fair amount of searching, I have found that I can enable OIDC support in a Kubernetes cluster. It looks like I will have to do the following: Setup an OIDC server and make an application in there and note its ClientID and Client Secret Log into the Master Kubernetes nodes and update the file found at /var/nutanix/etc/kubernetes/manifests/kube-apiserver.yaml with the following changes under the - kube

3120
V

Most awesome this week!

Badges

Show all badges

Popular Subjects

Terms & Conditions