I was wondering if anyone could assist us in some proof of concept work we are presently undertaking. We are looking at Acropolis block services for some of our systems which will remain as physical devices. In our initial PoC work we have noticed a quirk with how the system reacts to node failure and we are wondering how others get around it.
Our PoC is three OEM nodes with a volume group containing three vdisks. This works out nicely as each CVM ends up assigned with one vdisk. We then have external clients connecting to the ABS ISCSI data services IP address to get storage from the PoC cluster.
As part of our operational acceptance testing, we have been disconnecting a node from the network to replicate a failure and then observing the ABS and client OS behaviour.
Coming from a 3 tier solution back ground I would normally expect to see a redundant path kick in pretty much instantly and applications being completely unaware of any issue. The OS would normally registered path down in event viewer, but the MPIO tools allow seamless recovery.
In our PoC the behaviour is different. The OS figures it doesn’t have connectivity to the vdisk anymore, waits the default OS timeout of 60 seconds, then forces the ISCSI initiator to re-initialise, this takes an additional 15-20 seconds (in our testing 19 seconds consistently)
It almost seems like there is no multi path / resilience on the storage / ABS level in the same way, it is completely reliant on switching CVM targets which takes time.
This is a worry for us, certain applications we have cannot cope with this level of disk interrupt so I was wondering what other people do? Reduce the OS time out to a few seconds to force the re-initialisation earlier? I suspect the 19 seconds would also give us problems on one application we have L
When we plug the node back in, the fail back works seamlessly after a couple minutes, taking the 15-20 seconds once more for re-initialisation to favoured node.
Does anyone use ABS extensively? We are presently quite concerned about recommending it internally..
As an aside, previously in version 4.7 full multi path functionality seemed to be available but this seems to have been deprecated now, is there a specific reason for this? Was it too complicated to maintain a path map for the data segments in RF2 and 3?