Community Suggestions

Welcome to the Nutanix NEXT community. To get started please read our short welcome post. Thanks!

cancel
Showing results for 
Search instead for 
Did you mean: 

CVM RAM Usage High after upgrade to 5.0.2

SOLVED Go to solution
Highlighted
Pathfinder

CVM RAM Usage High after upgrade to 5.0.2

Hi all.

I have just (12 hours ago) upgraded our Nutanix Cluster to version 5.0.2 (from 4.7.3). After the uppgrade the health says ”Critical” because of ”CVM RAM Usage High”

We did not have that issue before the upgrade. It seams that the new firmware version is consuming more memory? Or?

 

Interesting thing is, is it look like memory consumtion is pretty much the same as before. When a look at the CVMs in vsphere client it look like the only using 50% - 60%.

 

Any ideas?

Thanks

 

/Kenneth

 

 

1 ACCEPTED SOLUTION

Accepted Solutions
Pathfinder

Re: CVM RAM Usage High after upgrade to 5.0.2

Hi @dlink7 and @aluciani.

 

Do not think it is the hypervisor, because it is the same version as before upgrade, only thing changed was the upgrade to 5.0.2

 

My CVM:s only had 16 Gb or RAM and since v 5 the recommended amount is 32 Gb aparently. I have increased to 32 Gb so the error is not there any more.

 

But the general feeling is that the upgrade did not do any good for my cluster, only I higher memory consumtion. But it is not an big issue right now, I need to add a node soon anyway to the cluster.

 

9 REPLIES
Nutanix Employee

Re: CVM RAM Usage High after upgrade to 5.0.2

Hi @KennethH

 

Why hypervisor? I noticed that VMware had a bug out for high memory usage a couple of weeks ago. Not sure if its related.

Community Manager

Re: CVM RAM Usage High after upgrade to 5.0.2

Hi @KennethH

 

Wondering if you saw the post from @dlink7 - Thanks

Pathfinder

Re: CVM RAM Usage High after upgrade to 5.0.2

Hi @dlink7 and @aluciani.

 

Do not think it is the hypervisor, because it is the same version as before upgrade, only thing changed was the upgrade to 5.0.2

 

My CVM:s only had 16 Gb or RAM and since v 5 the recommended amount is 32 Gb aparently. I have increased to 32 Gb so the error is not there any more.

 

But the general feeling is that the upgrade did not do any good for my cluster, only I higher memory consumtion. But it is not an big issue right now, I need to add a node soon anyway to the cluster.

 

Pathfinder

Re: CVM RAM Usage High after upgrade to 5.0.2

I'm in a similar situation currently, although mine are running with 24 gb of ram. I already called support on this and they didn't have a solution other than moving resources around. Moving to 32 gb might be a solution, but it's only recommended for post-dudupe systems from what I have read. I guess I'm going to lose another 24 gb of ram. Thats quite a bit of CVM over head... 13% of my cluster memory for storage. 

Explorer

Re: CVM RAM Usage High after upgrade to 5.0.2

I've been having a similar issue starting with 5.0.1 through 5.0.2.  I'm using AHV and have always had my CVMs at 32GB.  I bumped one to 34 to see if it made a difference but after a few days, I got the alert on it as well.  Same as with high CPU usage on CVMs.

Pathfinder

Re: CVM RAM Usage High after upgrade to 5.0.2

So we have a systemic issue here then. I guess its time to get back on with support.

Trailblazer

Re: CVM RAM Usage High after upgrade to 5.0.2

Hi,

 

Did you guys also upgrade NCC when doing this?

 

In the newest NCCs, the Threshold values have been changed for High RAM Usage, which triggers the alert.

I believe this has been changed from 500MB to 1GB.

 

The same goes for the High CPU Usage.

With that said, these alerts could most probably be false positives, have you seen any services that have been killed due to low RAM?

 

Kind Regards

Pathfinder

Re: CVM RAM Usage High after upgrade to 5.0.2

Nothing has been killed so far, just a very annoying "red alert". I'm running NCC version 3.0.1.1, which I believe is the newest. My alert is triggering as follows: 

Main memory usage in Controller VM 172.28.1.63 is high. 998576 KB of memory is free.

Main memory usage in Controller VM 172.28.1.61 is high. 752012 KB of memory is free.

 

If the new alert threshold is 1gb, that would explain the alert... but not the cause. 

Trailblazer

Re: CVM RAM Usage High after upgrade to 5.0.2

The "cause" has always been there, as the RAM usually goes up and down on a normal basis, such as when curator scans are running.

Earlier, the alert got triggered at 500MB, and they have raised it a bit to aggresive to 1GB which causes false positives. Most probably the cluster is behaving just like it did before the upgrade, but now you are actually getting notified due to a more aggresive threshold value.

 

In later releases the Service Leaders will be balanced better which causes better RAM Utilization over the board. As it is right now, most leader roles are all on the same CVM, which then gets this alert when cluster tasks are running.

 

As long as the services are running fine without restart you are in the clear, but if you are uncertain open up a case with support and they can help you with moving roles around, aswell as fine-tune the alert thresholds.

 

Kind Regards