Background
I have been using Dell Apex Backup Services (AKA Druva) for the past eleven months. Last
month we deployed our first Nutanix AHV Cluster. Provisioning a backup proxy VM on AHV to
facilitate backups to Dell Apex Backup Services should have taken a few hours at most but
instead it took nearly a week.
Provisioning Backups to Dell Apex from Nutanix AHV
What you are doing here is getting two VMDK files from a link in the Druva Management Console,
building a VM on AHV, and importing/attaching those VMDKs to your new backup proxy VM.
This VM will be the backup and restore target for all of your storage volumes and VMs on AHV.
You will need to configure networking for the VM and install a token (You get the token from the
Management Console.) on the backup proxy VM.
Next you will go back to the Management Console and (provided you made it this far) configure
one or more of the AHV VMs for backup. You will also have to enable the Nutanix iSCSI Data
Services IP Address and open several TCP ports to get this working.
Get Started!
1. Find Deploy the first Nutanix AHV backup proxy and register Prism via the Management
Console.
- Notice that they tell you to “deploy a Backup proxy in the Prism and activate it.” They do not explicitly state that this is a VM on AHV. So is this an appliance that you download and import to AHV? This is not clear.
- Notice that in step 3e where they want network details on your ‘Backup Proxy’. By now you can guess that it is a VM but they don’t tell you how you will get it from your Management Console and into AHV; so you are still having to guess what it is.
- Notice that in step 3g you are told to enter “IP Address: Enter the IP address for the Prism Element or Prism Central.” What they really want is an unused unique IP to be assigned to the backup proxy VM that you are guessing you need. They do not really want you to put the “IP address for the Prism Element or Prism Central”. Despite the names they are not referring to either.
- Notice that they link to Prerequisites for Nutanix AHV backup proxy deployment which only refers to needing TCP 443 to the Druva Cloud. NONE of the other ports that you will be needing are mentioned here.
- Notice that the Apex software on the backup proxy will create a Nutanix volume group dynamically for each operation. In order to do this it will need TCP 3205 and 3260 to the iSCSI Data Services IP and the backup proxy will also need TCP 9440 to Prism Element and (if you are using it) Prism Central. NONE of this is explained here or in the User Guide - Preparing your environment for successful Druva Nutanix AHV Backup.
2. I never got this method to work. The error I got was "Failed to validate IP address". This made no sense as I did not see any traffic coming from (I guessed it would be Druva servers on the Internet.) to my Nutanix Array. So I moved on…
Moving On: Manual Deployment
1. Next I found Manually deploy the Nutanix AHV backup proxy using Prism Console. Here
they actually tell you that you are downloading a “Backup proxy”. They don’t explicitly state that this is a VM but they do later in the article when they show you how to “Create a backup proxy VM”.
○ Notice that they link to Prerequisites for Nutanix AHV backup proxy deployment which only refers to needing TCP 443 to the Druva Cloud. NONE of the other ports that you will be needing are mentioned here.
2. Set up your iSCSI Data Services IP Address . You will need it later.
3. When you get your VM set up on AHV, and your VMDKs are imported and attached to
the VM, you will power it up. You will login via default credentials found in the knowledge
base article and then you will use a tool named ProxyConf to configure networking,
hostname, and to activate the backup proxy using the token generated on the
Management Console.
○ Notice that networking did not work for me when I used the ProxyConf tool. Instead I had to edit the network configuration manually. Here is what you will need. (The Backup Proxy VM is an Ubuntu server.)
i. Note: YAML does not like tabs. Use spaces.
- “via:” refers to your gateway address.
root@ahv:~# cat /etc/netplan/00-installer-config.yaml
network:
version: 2
renderer: networkd
ethernets:
ens3:
dhcp4: no
addresses:
- 192.168.10.18/27
routes:
- to: default
via: 192.168.10.1
nameservers:
addresses: 192.168.5.1, 192.168.6.1]
root@ahv:~# You can test your network configuration two ways:
1. Use the ProxyConf tool.
2. Use the netplan command.
root@ahv:~# netplan try
Do you want to keep these settings?
Press ENTER before the timeout to accept the new configuration
Changes will revert in 119 seconds
Configuration accepted.
root@ahv:~#
root@ahv:~# ProxyConf
====================================
Druva Phoenix Backup Proxy Setup
====================================
1. List network interfaces
2. Details of network interface
3. Change interface settings
4. Back
List of network interfaces:
- ens3 (static)
>
Enter interface name:
> ens3
IP Address: 192.168.10.18
***************************
root@ahv:~# ip a
1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
2: ens3: mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether 5 redacted ff
altname enp0s3
inet 1 redacted 7 brd 1 redacted 31 scope global ens3
valid_lft forever preferred_lft forever
inet6 fe redacted e5/64 scope link
valid_lft forever preferred_lft forever
root@ahv:~#
3. At this point you should be able to ping your VM and activate it. Here is where you will start needing open ports that the Prerequisites for Nutanix AHV backup proxy deployment didn’t bother telling you about.
○ For this step you will need your backup proxy VM to reach TCP 9440 on Prism Element and Prism Central.
○ For the next steps
i. From all of your AHV VMs you will need to get to your iSCSI Data Services IP Address on ports TCP 3205 and 3260
ii. From your backup proxy VM you will need to reach TCP 9440 on Prism Element, and Prism Central. This allows the creation of a volume group on Nutanix for each backup. (It would have helped to know about this process of creating the volume group but I did not see it explained anywhere.)
Use netcat (nc) to test your connections.
This is from the backup proxy VM.
root@ahv:~# nc -vz 192.168.5.81 3260
Connection to 192.168.5.81 3260 port ftcp/iscsi-target] succeeded!
root@ahv:~# nc -vz 192.168.5.81 3205
nc: connect to 192.168.5.81 port 3205 (tcp) failed: Connection timed out
root@ahv:~#
4. At this point I was able to set up my backups from a test VM but the backups were failing. The logs for the backup job referred failed logins to iSCSI targets. Later, when I had TCP 3260 opened the error code given was “AHV 53” which shows up as a link in the job log. You click on that link and come to a page listing every code BUT “AHV 53”. This was fixed when I opened TCP 3205. Finally, I got error code AHV 34 which was fixed by opening TCP 9440 from the backup proxy VM to the iSCSI Data Services IP.
5. Optional: Nutanix Guest Tools
○ While troubleshooting this deployment I installed the Nutanix Guest Tools (NGT).
I don’t believe that was necessary but it is recommended. For this you will need
your VM to reach Prism Element and the CVMs on TCP 2074.
Conclusion
The effort to provision backups from AHV to Dell Apex/Druva started six days ago and easily
took sixteen hours. The documentation was missing important information and was in places
misleading. A short paragraph providing an overview of the process, and reviewed by someone
who is not an experienced Nutanix customer, would greatly benefit future Nutanix users. I hope
the following documentation is updated soon:
● Deploy the first Nutanix AHV backup proxy and register Prism
● Manually deploy the Nutanix AHV backup proxy using Prism Console
● Prerequisites for Nutanix AHV backup proxy deployment
In addition, adding links to the User Guide - Preparing your environment for successful Druva
Nutanix AHV Backup to all three of the knowledge base articles above would be very helpful.