Intelligent Operations

  At Nutanix we obsess over customer experience. Form the purchase through transition to support - all things, but also your experience as an administrator is part of our obsession. We believe that you have other things to do than to spend countless hours on achieving something that can be achieved with less effort. 1-Click upgrade is a feature that is designed to streamline the upgrade process and save your time. Sadly nothing is perfect and things go out of order. If the 1-click upgrade fails with the error below some troubleshooting is due. release-api.nutanix.com could not be reached during 1-Click Upgrade Reaching a Web page is like reaching an IP address but with an extra layer of FQDN resolution by DNS hence the troubleshooting logic looks like this: Verify that the FQDN resolution is successful Inspect proxy settings Verify correct settings are applied if using Transparent Proxy Confirm that the firewall settings allow the traffic both ways Review logs and restart the

After connecting my Cluster to Prism Central I see some VMs in Prism Central that are not existing anymore in the Cluster and I cannot find a way to remove them. Here is an example: These VMs are already deleted in the clusterIn Prism these VMs are not visibleIs there a way to delete these VMs from PC? I already tried to remove the cluster from PC, run the cleanup scripts and connect to PC again (even to a new installed one), but it did not help.

So you kicked off the one-click Prism Central deployment and left your desk to get a coffee. Now you are back and see it has failed. Wonder what could have happened? Here’s a list of common issues you could take a look at before engaging our world-class Support!   The deployed Prism Central is not compatible with the underlying cluster AOS.  Check interoperability on the support portal   Network issues  - If the PC is deployed on a different subnet/VLAN and the CVM is unable to reach the PC network. You would see the PC VM has been created but since genesis service on the PC cannot be reached, the task eventually times out/fails. You could also have firewall rules in place preventing bidirectional communication on ports 80, 9440, 2100. This KB should help you with that.   If there is an existing PC binary or metadata file for deployment on a Prism Element cluster, the upgrade may fail with an error indicating the software already exists in the cluster.  The b

When managing large scale environments, it is important to have a central source of synchronised data. As is that data is not very useful. To become readable and interpretable information the data must be laid out and aggregated in a particular way. Furthermore, tracking the progression of stats overtime is, perhaps, a more valuable source of information about the environment. Reporting is one of many useful features that Prism Central provides to help you make the most of your HCI environment. You can be very granular in what you choose to include from VMs all the way to clusters. Graphs and tables. You can customise the appearance of the report with brand logos. More importantly, you can schedule it to run and be sent regularly for review via email. You can also group entities in the groups and review their performance as one. What if you wanted to look at performance or resources utilisation of VMs grouped by applications rather than one by one? You can. Remember, there are also

Prism Central allows you to configure syslog monitoring which forwards system logs(API Audit, Audit and Flow logs) from the registered clusters to an external syslog server   There are some consideration we need to be aware of:   As the logs are forwarded form Controller VM(CVM). It displays IP address of CVM We can only configure one rsyslog server  Protocols which are supported are TCP and UDP Logs can also be forwarded to a remote syslog server using Reliable Event Logging Protocol (RELP).   Configuration of the syslog server is very simple.  Login to Prim Central and go to settings       2.click syslog server   3. Add the external syslog server name, IP address and port     4. All the registered clusters will now forward logs to syslog server   For more details on syslog please refer following documents   https://portal.nutanix.com/page/documents/details/?targetId=Prism-Central-Guide-Prism-v5_16%3Amul-syslog-server-configure-pc-t.html

A few questions on logs collection and rotation today. Are the logs deleted automatically? Yes, they are! With newer AOS versions log bundles older than seven days are purged from CVM automatically.   Is there a way to check /home space usage of a CVM from the Prism Unfortunately no. Please refer to solution section KB-NCC - Health Check command fails with error "No space left on device" NCC - Health Check command fails with error "No space left on device" for the command on how to check the space usage.   Is there a difference between log collection triggered from the Prism as opposed to the same from the CLI. The end result is the same however CLI method allows for more flexibility due to additional options. From NCC 3.7.1 onwards, use of the Logbay utility for log collection is recommended instead of log_collector due to performance, usability and feature enhancements. Logbay eliminates performance issues that sometimes are observed with log_collector. If you have not yet upg

Anyone know if it is possible to create a scheduled task in AHV to migrate a VM to another hosts in an AHV Cluster?

Hi, im new in nutanix. I have 1 cluster with 2 blocks and 4 nodes here. Everything went fine until im seeing no detail in memory usage unlike other of my VMs. How can this happen? I need my memory usage detail to monitor any of my VMs with high resource usage. Any Solution to solve this off?  

You may want to unregister a cluster from Prism Central for various reasons. Currently, we don’t provide this option via the Prism GUI as to reduce the risk of accidentally unregistering a cluster. In recent releases, numerous key features such as micro-segmentation, CALM, Self-Service portal, Karbon etc require Prism Central and unregistration could result in these services being unavailable.    However, if you still wish to unregister a cluster either because the cluster is decommissioned or to deploy a new PC, these are the high-level steps you would have to follow.   SSH into any CVM in the cluster that needs to be unregistered and run the below command. ncli multicluster remove-from-multicluster external-ip-address-or-svm-ips=<PC_IP_ADDRESS> username=<PC_USERNAME> password=<PC_PASSWORD> force=true parameters: PC_IP_ADDRESS, PC_USERNAME, PC_PASSWORD   SSH into the Prism Central VM and run the unregistration script. python /home/nutanix/bin/unregistration_

Whether you are troubleshooting a particular issue or you are just curious to know how a certain component (such as a VM) performs in your cluster, understanding which tools are at your disposal will allow you to save time and effectively address issues related to your workloads in your Nutanix cluster; there are two type of charts you can use to review objects, understand how they are performing and take action accordingly, I will briefly explain each one of them below. Entity charts:   It allows you to check multiple metrics from a particular object, for example checking the latency of a VM alongside with the amount of IOPS and controller latency, which is really useful when troubleshooting performance related issues.Metric charts: You can think of it as the opposite of the option above, the metric is the main object and then you can obtain information about the metric for a particular component. For example if you need to get the latency metric, you can then filter further for th

Hey Eveyone, I recently added a new 2-node cluster to my prism central. When I try to launch Prism Element from Prism Central. I get a pop massage “This action is only supported on cluster version 4.5.2 or later. Any thoughts? Prism Central: 5.11.2.1 AOS: 5.10.9.1 Foundation: 4.5.3 Thanks  

Hi forumpeople, For some reason we see different values in Storage Summary in Prism and Sizer/Storage Capacity calculator. We are dealing with an RF2 cluster here. Storage Summary shows 47.9 TiB  Yet Sizer shows 35,53 TiB and Storage Capacity calculator shows 36,05 TiB   A small difference may be due to the actual cluster having 4x2TB NVMes per each of the 5 nodes, and only 1.92 being available in Sizer/Capacity calculator, but it should not make such a critical misalignment between Prism and Sizer/Capacity calculator. What is the actual usable capacity of the cluster (with RF2, N+1)? Also, how are “CVM Overhead” values taken into account in Prism’s Storage Summary and is it part of the actual usable capacity?

I recently added a new 2-node cluster to my prism central. When I try to launch Prism Element from Prism Central. I get a pop massage “This action is only supported on cluster version 4.5.2 or later. Any thoughts? Prism Central: 5.11.2.1 AOS: 5.10.9.1 Foundation: 4.5.3  

USEFUL COMMANDS TO TROUBLESHOOT PE-PC CONNECTIVITY ISSUES We have lots of scenarios where you might see alerts related related to PE-PC connection failure. This generally happens when network connectivity between PE and PC clusters are disrupted due to any reasons such as PC VM being rebooted, upgrading PC cluster , network issues, HTTP proxy issues(incorrect proxy whitelists) and port issues etc.   Firstly, when does PE-PC connectivity alert is raised on PE:   1)The alert is raised when PE-PC connectivity was disrupted for at least 6 minutes as of AOS 5.10.(Prios to AOS 5.10 alert is generated at a single instance of 2 minutes) Sometimes, PE-PC connectivity checks shows red heart on Prism even, if the connectivity is fine and none of the above reasons are present. In this case i.e if you verify there are no underlying PE-PC connectivity issue present, manually reset the check. Turn the check OFF and Turn it back ON from the Health page by clicking on this check like this below. 

Within the Nutanix main infrastructure there are two main components which Administrators can use to manage their environment, those are the following: Prism Element Easy to use graphical interface to manage a single Nutanix Cluster, it exists as a service within your CVMs with one of them acting as a leader.From Prism Element you can have two different authentication methods, which are local users and Active Directory accounts. Prism Central Additional component which can be deployed as a VM to manage multiple clusters which can be horizontally and vertically scaled to suit your environment depending on how many components such as VMs and Clusters it is required to manage.From PC additional to both local users and AD accounts you can also configure SAML authentication using an identity provider.If you want to know more about the authentication types here are some additional resources you can use: Prism Element Authentication Configuration Prism Central Authentication Confi

After upgrading to the latest Prism Central, AOS, and AHV versions, I wanted to check out the storage QoS options. But after looking all over, I could not find where to do this. I asked one of my fellow Nutanix adminstrators who has the same access as I do to the same resources and he showed me where the settings can be configured. However, I don’t have this option (and a few others) when logged in with my account. Not even with the Prism Admin account. I’ve tried different browsers, clearing browser cache, different computers. Nothing. Any ideas why I can’t see these options?

Upgrade Prism Central 5.11 to 5.17  through command line

We have several AHV clusters across the WANscape all managed from Prism Central. On one AHV cluster, I created a Windows Server 2016 and a Windows Server 2019 VM from CD ISOs, patched and tweaked and these are our “Golden Images”.  I clone those and run sysprep and those are what I make new VMs from. Now I have two problems: 1.) in each cluster we have an RF2 container and an RF3 container so these VMs are bound to one or the other, and 2.) how do I get these to other clusters? To address the differing containers, I end up using the image service (create image of the disks and create new VM from those disks).  But now I’m wasting storage because I’m keeping copies of the same VM on two different containers. To address the differing clusters, I was thinking about setting up Protection Domains and letting Nutanix replicate them to each cluster.  They would only get updated once or twice per month for Windows patches.  Doing it this way would allow me to control replication schedule a

Did you know that you can automatically send Nutanix Alerts to ServiceNow? Using X-Play's out of box Send Alerts to ServiceNow action along with either the Alert or new Alerts Matching Criteria trigger, makes it simple to create a Playbook to automatically send Alerts matching designated criteria to the ServiceNow events table. Here is a quick video illustrating how to send all Alerts in the system directly to ServiceNow.      

Last week, we summarized what is new in Prism 5.17 with a short demo. You may be wondering how you can create the same demo and how long would it take. Today, we will not only introduce new triggers and actions but also walk through how you can create the same demo.    

You have been asking whether they can build a workflow to detect over-provisioned VM, inform VM owner, let the owner decide whether and when to trigger the memory reduction, and then go through the ticket system to issue the execution with the approval. Many of you told me if you can build it, you may need to spend a couple of weeks to build this automation in the best situation. The cost of building operations automation forces many enterprises to continue doing manual tasks. It is not right. Prism Pro 5.17 extends its democratization of the operations automation. It adds four more trigger types, which includes scheduling, event, multiple alert policies, and the powerful webhook. It also includes a few more out-of-the-box actions. The video below shows that you can do all these and build the whole automation within 5 minutes by using these new features.    

I need to move the PC VM to a different cluster and want to change the IP address of PC. I cannot find a description for version 5.16, only a very old one for version 4 that does not work. I also tried the instructions from https://portal.nutanix.com/page/documents/details/?targetId=Advanced-Setup-Guide-AOS-v5_16:ipc-cvm-ip-address-reconfigure-t.html but it does also not work. After reboot VM ip address is still the old one. Is there a description how to do it with PC 5.16?

I have configured Blurb Text in Prism UI, but it’s not visible on Login Page, though Title Text is visible fine. AOS Version : 5.10.9 I have restarted Prism Service as well. Below is the image, i have cropped Title Text  

We have already shared a preparation sequence for an upgrade. Step 5 in the process is to upgrade Foundation.    The upgrade process for the Foundation is straightforward and does not require any downtime:  Run NCC as described in Run NCC Checks. Log on to the Prism Web Console, click the gear icon in the main menu, select Upgrade Software in the Settings page, and then click the Foundation tab. If an update is available, click Upgrade Available and then click Download. When the download is complete, click Upgrade > Upgrade Now, then click Yes to confirm.To run the pre-upgrade installation checks only on the Controller VM where you are logged on, click Upgrade > Pre-upgrade. These checks also run as part of the upgrade procedure.The Upgrade Software dialog box shows the progress of your selection, including pre-installation checks. To upgrade the Foundation of the cluster at a dark site please follow the steps provided in the Prism Web Console Guide v5.10: U

  Monitoring a cluster for memory and compute usage is an integral part of an administrator job to make sure the cluster is healthy and stable. You could face a situation in which you just went through an ESXi upgrade and now noticing that CVM Memory is displayed as 100% in Prism UI and VMware Web Client.    Is my CVM really consuming 100% memory and do I need to troubleshoot further regarding where the memory is getting utilised?   There are chances that this might just be a cosmetic issue, happened after an ESXi upgrade due to how active memory heuristics works and active sampling happens in an ESXi environment.    The following Knowledge-Base article will help you to understand the issue better thus helping you to isolate it to your environment.   CVM Memory is 100% in Prism and Web Client with ESXi 6.5 or later