Question

Nutanix DVP

  • 20 October 2020
  • 1 reply
  • 112 views

Dear Community

Scenario

I’m currently testing Nutanix DVP (Link) on Centos 7 with Netbox Docker (Link).

I’m overriding the original volume definition in the Docker Compose file. Instead of the local driver I use nutanix:latest.

 volumes:   netbox-static-files:     driver: nutanix:latest   netbox-nginx-config:     driver: nutanix:latest   netbox-media-files:     driver: nutanix:latest   netbox-postgres-data:     driver: nutanix:latest   netbox-redis-data:     driver: nutanix:latest

I’m then able to start and use Netbox Docker (docker compose up -d).

Problem/Symptoms

When I’m stopping Netbox Docker with docker compose down usually I’m able to bring up Netbox Docker again. Sometimes this works up to three times. But after a few times of up and down it isn’t possible anymore. If it doesn’t work anymore, following error is shown while starting:

root@b2cntr-dckwor02[13:06:08]/var/projects/netbox-docker$ docker-compose up -d Creating network "netbox-docker_default" with the default driver Creating netbox-docker_redis-cache_1 ... done Creating netbox-docker_redis_1       ... done Creating netbox-docker_postgres_1      ... done Creating netbox-docker_netbox-worker_1 ... error

ERROR: for netbox-docker_netbox-worker_1  Cannot start service netbox-worker: error evaluating symlinks from mount source "Not mounted": lstat Not mounted: no such file or directory

ERROR: for netbox-worker  Cannot start service netbox-worker: error evaluating symlinks from mount source "Not mounted": lstat Not mounted: no such file or directory ERROR: Encountered errors while bringing up the project.


After that, nothing I do helps to run Netbox Docker again. Only docker-compose down -v helps, which obviously destroys all data.

Analysis so far

I can see, that /var/log/messages notes a lot while bringing down and up Netbox Docker and I’m not sure, what messages could be important for further debugging. (See https://drive.unicocloud.ch/s/ooxjLwWB6GK6NoR pw: dckrf4il).

I’m pretty sure it has something to do with the mapping and unmapping of the volumes because with local driver chosen everything seems fast and reliable.

My Question

Is there anybody who has seen this behaviour and knows an instant advice? If not: Is there anybody who knows further steps for debugging?

Thanks in advance for your help!

 

EDIT: Sorry for the title of this thread. I’m not able to edit it after posting.


1 reply

@Daniel-Unico Based on the logs attached. It looks like iscsi.service is not loaded which required for plugin for it to talks to remote storage via iscsi.

20T13:10:23+02:00" level=error msg="/ - - [20/Oct/2020 11:10:23] \"POST /VolumeDriver.Capabilities HTTP/1.1\" 200 -" plugin=90ce0da80e7bd187290ddace2f7b49c9ad63742d59cd161f0abde8007d8adbab
Oct 20 13:10:23 b2cntr-dckwor02 systemd: Unit iscsi.service cannot be reloaded because it is inactive.

From what it appears:  iscsi down --> plugin install fail --> no volume mounted---> and hence the error. Can you please provide the nvp logs ?
Location: 

/var/lib/docker/plugins/rootfs/nvp.log

I would highly recommend to open a support case and engage someone to take a look at the setup.

Best,

Ateiv Jain

Reply