5 Essential Tips for Maximizing Your Experience at Nutanix .NEXT for Bloggers
Just keeping this thread in parallel with the one on Reddit, since it’s the same issue.I don't think AHV is marking anything as bad, I think it just can't seem to locate the VM definition itself for the CVM (or it is corrupt). lspci will validate if CE can see the NVMe drives or not.The VM xml for the CVM is stored in /var/run/libvirt/qemu I believe (I'm travelling so don't have access to my CE cluster right now to doublecheck). It can be rebuilt if necessary.
For those who come here later: The xml needed to be rebuilt and he was able to get things back up and running. Here’s the reddit threadhttps://www.reddit.com/r/nutanix/comments/wiqcj2/nutanix_ce_storage_issue/
It looks like you’re just pulling the packet statistics on a per VLAN basis and just determining if they’re passing traffic.You could do something similar from AHV, but look for learned mac addresses in the VLAN you are specifically looking to see if it’s in use.From AHV: ovs-appctl fdb/show br0-up
I’m curious as to what your use case is for this? I ask because there are certain workflows in VMWare that tend to require excluding a specific disk from snapshots for performance reasons and those are usually the driving factors in using independent disks. Since these issues don’t existing in AHV, often it doesn’t make sense to exclude a particular volume.Are you looking to do this as a one-off event or as part of a schedule?High level, you could do this by creating a Volume Group with just that vdisk in it, then attach the VG to the Virtual Machine. When you take a VM level snapshot it won’t snap the VG attached disk, just the ones defined as part of the VM.There is always peril though in NOT including things in data protection schemes such as snapshots, so I’m curious as to the use case to make sure you don’t shoot yourself in the foot later when you try to do a recovery.
Absolutely. Simply place all the volumes that are a part of SQL in a Volume Group, and attach that to the VM. When a snapshot is run against the VM it will NOT snap the volume group, just the OS disk. I *highly* recommend reading our SQL Best Practices Guide, specifically the portion about Storage located here: https://portal.nutanix.com/page/documents/solutions/details?targetId=BP-2015-Microsoft-SQL-Server:BP-2015-Microsoft-SQL-Server
Sureline Systems who have developed and sell the SureEdge product is a Technology Alliance Partner, so you will have to reach out to them directly if you want a copy of their tool to test migrations.https://surelinesystems.net/products/sureedge-migrator/
Usually this alert will pop up if you’ve configured something that would normally connect over the Data Service IP address, but instead is connecting over the cluster’s VIP. Backup software and replication software are usually the culprits (like Veeam, Rubrik, Cohesity, etc.)Here’s an article on how to hunt down the offending IP address that made the connection, and confirm if you have the data services IP configured.https://portal.nutanix.com/page/documents/kbs/details?targetId=kA07V000000H3gtSAC
I would recommend taking a look at the vSphere Administration guide as it points out a few specifics that you will want to know about running vSphere 8. https://portal.nutanix.com/page/documents/details?targetId=vSphere-Admin6-AOS-v6_7:vsp-esxi-support-vsphere-c.htmlAs far as vCenter is concerned, we follow the VMWare compatibility matrix, so if ESXi 8 is supported than vCenter 8 is also supported.
Are you running Calm in this environment, and is this on Community Edition or Release edition? If you are running Calm, what release of Calm? There is a known issue where this can occur in pre-Calm 3.3 revisions, if this is on the Release edition of Nutanix and not CE, please open a case and reference KB-11594
I would definitely open up a case with support, they should be able to get this resolved quickly and will want to see why this is occurring post 3.3
You will need 3 disks in the system in order to complete the installation, one for the Hypervisor, one for the CVM and one for the Data disk. At a minimum, you will have one drive marked as “I” for the installer (this is the phoenix ISO you boot from), “C” for CVM, “D” for data, and “H” for hypervisor. It is required that you use an SSD for the CVM, and recommended for the Hypervisor. Making sure the hypervisor disk is the smallest (it doesn’t need to be larger than say 64GB) also can help. If you don’t mind posting a photo or screenshot of your disk selection screen it will definitely help.Also give us some information about the server, what make/model/generation is it, CPU, and how much memory. What kind of type and size of disks are you using, and what type of disk controller you have. All of this information will streamline us getting you to a successful install
Can you provide some more specs on the server? What CPU, how much memory, and what manufacturer network adapters are in it? Can you also detail the disk controller and what size/type of disks you’re using? You can get most of that information via iDRAC. Also, illuminating the chassis LED is a benign error message during CE installation, so I imagine something else has failed.A screenshot of the console or a photo attached of your disk select screen and also the full error will be a huge help in getting things up and running.
I recommend taking a look at the Getting Started with CE Guide and the minimum/recommended hardware requirements found here: https://portal.nutanix.com/page/documents/details?targetId=Nutanix-Community-Edition-Getting-StartedThere is a hard requirement of 1 SSD for the CVM, the others can just be spinning HDDs. You should be able to put the RAID adapter in your G9 in Pass-thru mode which will allow you to use those disks without RAID configured.
R720 is fine, I’ve run R720s many times in the past in my lab. SSD is a requirement in order for Nutanix to function for the Controller VIrtual Machine. This is where all of the initial writes go so needs to be performant. THe reason you can’t use C is because none of the drives are SSDs. Also the “I” is probably lower on the list, you just need to scroll down.
I usually create my USB stick with both, but I tend to move it around from machine to machine and some support UEFI while others don’t.Depending on which revision of the H710 you have, the best option is to flash it to IT mode, this will pass the drives through as they are and no need to create a VD, however if you have a version that can’t be flashed, create a RAID0 disk containing each disk individually (so if you have 3 disks, you have 3 RAID0 virtual devices)Next, boot the CE installer, and you’ll see all the drives detected as HDDs, note which one is actually your ssd (usually can tell by capacity or serial#) and hit Ctrl+C to break out of the installer If the SSD drive you're going to put the CVM on is sdb, run:echo 0 > /sys/block/sdb/queue/rotationalThen restart the installer and it should show your disk as an SSD./ce_installer && screen -rThe installer should now let you assign your SSD to the CVM
How is the disk attached to the system? What kind of host and what disk controller are you using? Are you using some form of RAID adapter or HBA? Typically this issue crops up when the controller between the disk and the host can’t (or won’t) let the OS know that it is an SSD.There is a workaround, but I’d recommend resolving whatever issue is causing it to show up as an HDD if possible.Workaround is to first identify the sdX name associated with the device in the disk list of the installer, then hit Ctrl+C to break out of the installer and back to the command prompt.Run: cat /sys/block/sdX/queue/rotationalRotational should be 0 for the SSD and 1 for HDD.If the drive returns 1, the system believes there is no SSDTo fix this, set rotational to:echo 0 > /sys/block/<device>/queue/rotationalThen restart the installer (do not reboot): ./ce_installer && screen -r
I would just proceed with the installation and see if it bombs, the “I” should be set automatically anyways.
Odd, can you show the output of the:cat /sys/block/sdc/queue/rotational command?Remember after setting it to 0, DO NOT REBOOT, just restart the installer.
In Nutanix we are providing an entire hyperconverged stack, even a single node is a “cluster” so comparing it to just ESXi alone doesn’t make sense. You need to compare it to ESXi + VSAN.For ESXi+ VSAN, you need a device for the Hypervisor, a Device for Cache, and a device for Storage.For Nutanix, you need a device for Hypervisor, a Device for CVM, and a device for ADDITIONAL Storage (The CVM disk actually provides some capacity for storage for Oplog and SSD capacity)So there is the same requirement for AHV on Nutanix as ESXi.Also note, there is currently no possibility for a single disk install, again due to the HCI nature of Nutanix as a platform.
I think you’re best bet would be to read through the nutanixbible.com to get a deep dive into the questions here and also reach out to a Partner or Nutanix SE that can spend the time to walk through all of these in detail for you.
I would read through http://www.nutanixbible.com It will provide a lot of details on how the architecture works and lean into sizing. We absolutely scale up and down. I cover Healthcare primarily and we have as small as two node ROBO clusters for small facilities all the way up to massive all NVMe clusters for Epic workloads. I’m an engineer so I don’t do anything with pricing. However costs are like anything else, it depends on the configuration. Compared to many of the recent pricing changes within the VMWare ecosystem I always ask what are you comparing Nutanix pricing to, and in what context?
is AHV installed on a USB disk? This may be there reason for your AHV hypervisor being stuck. Upgrading AHV when booting from USB is not supported due to some changes in the drivers for the AHV stack, and that might be the reason that you are stuck where you are at. I strongly recommend reading through the Getting Started Guide, as nearly all of your post-installation questions, along with warnings about doing LCM updates with AHV booting from USB are covered there. I’m curious as to the final command you used to create your cluster, as the “one_node_cluster” is not supported with a single CVM drive (nor in a CE cluster). That also might be impactful here as the system might be expect certain configurations that don’t exist on your clusters.https://portal.nutanix.com/page/documents/details?targetId=Nutanix-Community-Edition-Getting-Started
I would give it a try writing the image to USB with Rufus. I regularly see similar issues installing CE from iDRAC/CIMC/iLO/IPMI. It’s not a very “light” image and often times we see these I/O errors.
What error do you get when you’re deploying and what process are you following for the deployment.I have NKE deployed on one of my CE clusters now, so it does work.
@Judasani make sure the disk you’re using for the hypervisor boot disk is the smallest disk in the system. It appears that you’re running into a known issue at the end of the install where the system is not properly re-identifying the AHV boot disk to perform some AHV customization steps.I also recommend doing the install in Legacy boot mode vs. UEFI.
Already have an account? Login
Enter your username or e-mail address. We'll send you an e-mail with instructions to reset your password.
Sorry, we're still checking this file's contents to make sure it's safe to download. Please try again in a few minutes.
Sorry, our virus scanner detected that this file isn't safe to download.
