My question is about the network connectivity of Nutanix nodes across two campus buildings. The objective is to create a four node cluster in building A and a four node cluster in building B. The buildings are about 2000 ft apart with dual fiber at 10Gbps Then with VMware ESXi create a stretch clsuter across all 8 nodes. That's part seems rather straight forward.
The questions are around network connectivity. In building A there is a core switch Cisco 6506 and an access layer Cisco Nexus 5548. In building B there is the same thing. The two core switches have 20 Gbps interconnect links and VSS. The Nexus 5548s are cross connected to the core and to each other, all with 20 Gbps each. The VSIs are on the Core.
Normally I would have two switches in the same building to redundantly connect each node to, but in this case I only have one 5548 available in each building. If I connect each node with one link locally to the 5548 then using a long range SFP in the server I connect the redundant link to the 5548 in the other building.
What problems will I have if I do this?
Is there a way to keep all traffic between all of the nodes in a particular building so that it does not involve the switch in the other building except when it is writing the data to the other four node cluster?