It seems that when updating AOS Nutanix is creating backups of /home/apache/www/ called PrismUI.tar.gz . Those are then stored in /home/apache/www/console/ . It further seems they tried (but failed) to exclude this console directory. (Don't have the exact tar command line any more but I guess Nutanix can find it) This means that later backups include all previous ones, recursively. I.e. backup #1 is OK. But backup #2 will include #1. Backup #3 will even include #1 and #2 (including another copy of #1). And so on. Which means the file size of subsequent backups grows FAST. For one Cluster the file grew from the usual ~125 MB to over 3GB. This in turn leads to the backup being rejected and recreated constantly. Probably bad for the SSDs.
This happened on all Versions I was able to check: ESXi: 5.20.1.1, 5.20.2 AHV: 5.18.1.2 5.19.0.5 6.0.1 6.0.1.6 6.0.1.7 6.0.2
Here the most egregious example:
$ sudo find / /home/ -xdev -iname PrismUI.tar.gz | xargs -d'\n' ls -lah
-rwxr-x---. 1 nutanix apache 118M Oct 30 2020 /home/apache/www/console/el7.3-release-euphrates-5.18.1.1-stable-4546d2908cb8495b316deb45de63b7f5e52541a1/PrismUI.tar.gz
-rwxr-x---. 1 nutanix apache 236M Jan 11 2021 /home/apache/www/console/el7.3-release-euphrates-5.18.1.2-stable-b1b096696c0c034570545912a00d39746e901f36/PrismUI.tar.gz
-rwxr-x---. 1 nutanix apache 485M Jan 14 2021 /home/apache/www/console/el7.3-release-euphrates-5.19.0.5-stable-22254494062244528e7a1ea132c23fe4005cb89c/PrismUI.tar.gz
-rwxr-x---. 1 nutanix apache 2.0G Oct 12 10:09 /home/apache/www/console/el7.3-release-fraser-6.0.1.6-stable-a922e5fdfbffa9800546950baa9072d49c590a8c/PrismUI.tar.gz
-rwxr-x---. 1 nutanix apache 1001M Aug 17 15:51 /home/apache/www/console/el7.3-release-fraser-6.0.1-stable-c5a21b612ea2da5fadb6524673cd75c0074c982f/PrismUI.tar.gz
-rw-r-----. 1 nutanix nutanix 485M Nov 12 15:55 /home/nutanix/PrismUI.tar.gz
(The last file above is the new backup currently being created)
$ tar -tvf /home/apache/www/console/el7.3-release-euphrates-5.18.1.1-stable-4546d2908cb8495b316deb45de63b7f5e52541a1/PrismUI.tar.gz --wildcards "*.gz"
tar: *.gz: Not found in archive
tar: Exiting with failure status due to previous errors
$ tar -tvf /home/apache/www/console/el7.3-release-euphrates-5.18.1.2-stable-b1b096696c0c034570545912a00d39746e901f36/PrismUI.tar.gz --wildcards '*.tar.gz'
-rwxr-xr-x nutanix/apache 123612132 2020-10-30 08:52 console/el7.3-release-euphrates-5.18.1.1-stable-4546d2908cb8495b316deb45de63b7f5e52541a1/PrismUI.tar.gz
$ tar -tvf /home/apache/www/console/el7.3-release-euphrates-5.19.0.5-stable-22254494062244528e7a1ea132c23fe4005cb89c/PrismUI.tar.gz --wildcards '*.tar.gz'
-rwxr-xr-x nutanix/apache 247074288 2021-01-11 13:49 console/el7.3-release-euphrates-5.18.1.2-stable-b1b096696c0c034570545912a00d39746e901f36/PrismUI.tar.gz
-rwxr-xr-x nutanix/apache 123612132 2020-10-30 08:52 console/el7.3-release-euphrates-5.18.1.1-stable-4546d2908cb8495b316deb45de63b7f5e52541a1/PrismUI.tar.gz
$ tar -tvf /home/apache/www/console/el7.3-release-fraser-6.0.1-stable-c5a21b612ea2da5fadb6524673cd75c0074c982f/PrismUI.tar.gz --wildcards '*.tar.gz'
-rwxr-x--- nutanix/apache 508014736 2021-01-14 14:59 console/el7.3-release-euphrates-5.19.0.5-stable-22254494062244528e7a1ea132c23fe4005cb89c/PrismUI.tar.gz
-rwxr-x--- nutanix/apache 247074288 2021-01-11 13:49 console/el7.3-release-euphrates-5.18.1.2-stable-b1b096696c0c034570545912a00d39746e901f36/PrismUI.tar.gz
-rwxr-x--- nutanix/apache 123612132 2020-10-30 08:52 console/el7.3-release-euphrates-5.18.1.1-stable-4546d2908cb8495b316deb45de63b7f5e52541a1/PrismUI.tar.gz
$ tar -tvf /home/apache/www/console/el7.3-release-fraser-6.0.1.6-stable-a922e5fdfbffa9800546950baa9072d49c590a8c/PrismUI.tar.gz --wildcards '*.tar.gz'
-rwxr-x--- nutanix/apache 508014736 2021-01-14 14:59 console/el7.3-release-euphrates-5.19.0.5-stable-22254494062244528e7a1ea132c23fe4005cb89c/PrismUI.tar.gz
-rwxr-x--- nutanix/apache 1049239743 2021-08-17 15:51 console/el7.3-release-fraser-6.0.1-stable-c5a21b612ea2da5fadb6524673cd75c0074c982f/PrismUI.tar.gz
-rwxr-x--- nutanix/apache 247074288 2021-01-11 13:49 console/el7.3-release-euphrates-5.18.1.2-stable-b1b096696c0c034570545912a00d39746e901f36/PrismUI.tar.gz
-rwxr-x--- nutanix/apache 123612132 2020-10-30 08:52 console/el7.3-release-euphrates-5.18.1.1-stable-4546d2908cb8495b316deb45de63b7f5e52541a1/PrismUI.tar.gz
Subsequent backups for 6.0.1.7 and 6.0.2 then kept continuously failing and retrying till I temporarily moved the old backups out of the way. Leading to a successful 180MB backup.
For outsourcing reasons I'm not currently allowed or able to open actual tickets. But I (and I guess pretty much everyone else including nutanix) still want to see this fixed. So I submit it to the void and hope the right person listens. Good thing I don't actually need support with this. If it's fixed in one of the next release that will be sufficient for me.
). Indeed, you are correct in this behavior and our developers have already identified, created and resolved a defect according to this specific issue. Currently and, though always subject to change, the resolution should be included within the 6.1 and later release versions.