Skip to main content
Question

When the prism is being deployed the deployment fails at 100% where one of he pos is not coming up

  • January 30, 2026
  • 6 replies
  • 67 views

Forum|alt.badge.img

Hello All,

We are trying to deploy prism central on top of 2 node cluster but that prism central fails to deploy at 100%.

Error message that it gives - 

Encountered Exception in post_deployment step: Failed to enable micro services infrastructure on PC: deploy msp:Error deploying addons: failed to deploy monitoring addon: failed to deploy and verify kube-prometheus: failed to verify kube-prometheus: Operation timed out: failed to verify kube-prometheus: expecting 1 available replica of k8s prometheus in ntnx-system namespace. Currently running: 0

Trying to get the pods up but that isnt possible.
Checked the resources for the hardware and they are also available.

Need help to resolve this issue

Below are the versions - 
AHV - 11.0
AOS- 7.5
Prism - 7.5

Troubleshooting done so far - 

Attempted to manually bring the affected pods up and running

Deleted the problematic pod

Ensured all components are configured in the same time zone (UTC – default for Nutanix)

6 replies

selvamani
Forum|alt.badge.img+1
  • Adventurer
  • January 30, 2026

Hi Tushar , 

Prism Central is not supported on a 2-node cluster
Prism Central requires a minimum 3-node cluster to reliably deploy and run its Kubernetes-based microservices. On 2-node clusters, pods like prometheus often fail to schedule or remain pending, which matches what you’re seeing.

I hope thiis  is not a raw hardware resource issue
Even if CPU and memory look available, Kubernetes scheduling still fails due to platform constraints on 2-node clusters. That’s why the prometheus replica count stays at 0.

Can you check to confirm following :

kubectl get pods -n ntnx-system
kubectl describe pod <prometheus-pod-name> -n ntnx-system
ncli cluster info


This is the root cause of 2 node cluster 

The supported and reliable fix is to expand the cluster to 3 nodes and then redeploy Prism Central. Once the third node is added, the MSP services (including kube-prometheus) should deploy successfully and the PC deployment will complete.

If expanding the cluster is not possible, Prism Central deployment on that environment is not supported, and the failure at 100% is expected behavior.

If you want, share the pod describe output here and we can validate it, but based on the versions and symptoms,I am sure this is almost certainly a cluster sizing limitation rather than a software defect.

Thanks 
Selvamani.S


 


Forum|alt.badge.img
  • Adventurer
  • January 30, 2026

Thanks, sharing output here


Forum|alt.badge.img+3

Hi ​@selvamani ,

when you are saying PC is not supported on 2 node cluster, is there any official document for this ?is it same for single node PC and scale-out PC?

Hi ​@Rishi99 

are you trying to install a scaleout PC or single node PC on your cluster?
have you tried installing a single node PC and see if it works?


selvamani
Forum|alt.badge.img+1
  • Adventurer
  • February 2, 2026

Hi  ​@jamali.ahmad , 

You are right, and I want to clarify my earlier message. Prism Central can run on a 2-node cluster, but there is an important limitation with Microservices (MSP).

What’s happening is that Microservices require 3 nodes. The Microservices Infrastructure runs Kubernetes inside Prism Central and needs at least 3 nodes to work reliably.

On a 2-node cluster, the system often cannot start the kube-prometheus pods during post-deployment. This causes the installation to get stuck and eventually fail.

The Prism Central VM itself is supported on 2 nodes, but enabling Microservices on a 2-node cluster is unstable and commonly fails.

If you cannot add a third node, redeploy Prism Central and do not enable Microservices Infrastructure. This will allow the deployment to complete successfully and still provide standard Prism Central management features.

Sorry for the earlier confusion.

Thanks 
Selvamani.S


Forum|alt.badge.img
  • Author
  • Adventurer
  • February 2, 2026

nutanix@NTNX-172-30-30-15-A-PCVM:~$ kubectl get pods -n ntnx-system
NAME                                                    READY   STATUS    RESTARTS         AGE
alertmanager-main-0                                     2/2     Running   2 (15h ago)      16h
csi-snapshot-controller-6f9b6648bc-85hjj                1/1     Running   1 (15h ago)      16h
csi-snapshot-webhook-594df44d75-9ckq8                   1/1     Running   1 (15h ago)      16h
fluent-bit-2ptsv                                        1/1     Running   1 (15h ago)      16h
kube-state-metrics-7979976844-9lsmd                     3/3     Running   3 (15h ago)      16h
mutator-webhook-dep-54887559f5-949b7                    1/1     Running   1 (15h ago)      16h
node-exporter-th9j2                                     2/2     Running   2 (15h ago)      16h
ntnx-cluster-maintainer-6b588f4db4-jb4ht                1/1     Running   0                4h41m
ntnx-k8s-cluster-maintainer-operator-5c88ff6467-47lpc   1/1     Running   1 (15h ago)      16h
nutanix-csi-controller-868bb84888-r8sqp                 7/7     Running   14 (4h42m ago)   16h
nutanix-csi-node-mcjl8                                  3/3     Running   6 (4h42m ago)    16h
prometheus-k8s-0                                        0/2     Pending   0                16h
prometheus-operator-699f88d6dc-q2xl4                    2/2     Running   2 (15h ago)      16h


Forum|alt.badge.img
  • Author
  • Adventurer
  • February 2, 2026

Cluster Id                : 6f1f026e-16cb-47ed-ad8a-da17319a9b01::3281675071644146433
    Cluster Uuid              : 6f1f026e-16cb-47ed-ad8a-da17319a9b01
    Cluster Name              : prism
    Cluster Version           : pc.7.5.0.5
    Cluster Full Version      : el8.5-release-ganges-7.5.0.5-stable-33e266ab5c96cdbe92aa9ea64584d38bbea13cc8
    Is LTS                    : false
    External Data Services... :
    Support Verbosity Level   : BASIC_COREDUMP
    Lock Down Status          : Disabled
    Password Remote Login ... : Enabled
    Timezone                  : Atlantic/Reykjavik
    NCC Version               : ncc-5.3.0
    Degraded Node Monitoring  : Enabled