Solved

Networking Assistance - Adding a firewall VM


Howdy,
We currently have a fairly stock Nutanix AHV (5.5.2) cluster setup with 3 nodes. Currently all VMs on here are using the same VLAN. Nutanix is plugged into a single physical Cisco switch with the following configurations:

Switch (all the ports are configured the same):
code:
interface GigabitEthernet1/0/3
switchport access vlan 37
switchport mode access


CVMs:
code:
nutanix@NTNX-18FM6G090043-A-CVM:10.110.37.51:~$ manage_ovs show_interfaces
name mode link speed
eth0 1000 True 1000
eth1 1000 True 1000
eth2 10000 False None
eth3 10000 False None
nutanix@NTNX-18FM6G090043-A-CVM:10.110.37.51:~$ manage_ovs --bridge_name br0 show_uplinks
Bridge br0:
Uplink ports: br0-up
Uplink ifaces: eth1 eth0


Hosts:
code:
[root@NTXA ~]# ovs-vsctl show
0b0ac217-d903-43f9-8a1b-fee797db3167
Bridge br.mx
Port br.mx
Interface br.mx
type: internal
Port "br.mx.u.br0"
Interface "br.mx.u.br0"
type: patch
options: {peer="br0.local.d"}
Port br.mx.d
Interface br.mx.d
type: patch
options: {peer=br.microseg.u}
Bridge "br0.local"
Port "br0.local"
Interface "br0.local"
type: internal
Port "tap0"
tag: 0
Interface "tap0"
Port "br0.local.d"
Interface "br0.local.d"
type: patch
options: {peer="br.mx.u.br0"}
Bridge br.microseg
Port br.microseg.u
Interface br.microseg.u
type: patch
options: {peer=br.mx.d}
Port br.microseg
Interface br.microseg
type: internal
Port br.microseg.d
Interface br.microseg.d
type: patch
options: {peer=br.nf.u}
Bridge br.dmx
Port "br.dmx.d.br0"
Interface "br.dmx.d.br0"
type: patch
options: {peer="br0.u"}
Port br.dmx.u
Interface br.dmx.u
type: patch
options: {peer=br.nf.d}
Port br.dmx
Interface br.dmx
type: internal
Bridge "br0"
Port "br0.u"
Interface "br0.u"
type: patch
options: {peer="br.dmx.d.br0"}
Port "vnet0"
Interface "vnet0"
Port "br0"
Interface "br0"
type: internal
Port "br0-up"
Interface "eth1"
Interface "eth0"
Port "br0-dhcp"
Interface "br0-dhcp"
type: vxlan
options: {key="1", remote_ip="10.110.37.52"}
Port "br0-arp"
Interface "br0-arp"
type: vxlan
options: {key="1", remote_ip="192.168.5.2"}
Port "vnet2"
Interface "vnet2"
Bridge br.nf
Port br.nf
Interface br.nf
type: internal
Port br.nf.d
Interface br.nf.d
type: patch
options: {peer=br.dmx.u}
Port br.nf.u
Interface br.nf.u
type: patch
options: {peer=br.microseg.d}
ovs_version: "2.5.2"
[root@NTXA ~]#
[root@NTXA ~]#
[root@NTXA ~]# ovs-appctl bond/show
---- br0-up ----
bond_mode: active-backup
bond may use recirculation: no, Recirc-ID : -1
bond-hash-basis: 0
updelay: 0 ms
downdelay: 0 ms
lacp_status: off
active slave mac: ac:1f:6b:62:03:b9(eth1)

slave eth0: enabled
may_enable: true

slave eth1: enabled
active slave
may_enable: true


Now, I have setup a second VLAN within AHV (2525). I would like to attach it to a collection of VMs to keep them completely separated from the other VMs network wise, while still allowing them to talk between themselves. To accomplish I was thinking that I could install a virtual firewall/router (like pfsense). I would give it two NICs, one for each VLAN. It would act as the gateway for these isolated VMs. Its "internal" NIC would use VLAN 2525 and its "outside" would be vlan 37 so that it can talk to the rest of the network (if we decide to allow certain traffic to pass).

I have gone ahead and set this up, but I have been unable to get this to work yet though. Does it require any changes within AHV/OVS to make this work?

Thanks!
icon

Best answer by bbbburns 18 June 2018, 17:52

@AlanS I should be able to help out.

The configuration you're proposing creates an isolated network on VLAN 2525 that's unique to each AHV host. VLAN 2525 is not present on the physical switch, so any traffic from one VM to another on VLAN 2525 will only extend as far as a single AHV host. This creates 3 separate networks (one on each host), so you're also going to need 3 separate firewalls, one on each host.

This probably isn't what you want, since a VM in VLAN 2525 on host 1 will not be able to communicate at all with a VM on host 2 in VLAN 2525. If this IS what you want, let me know and we can work through the setup.

Otherwise I would recommend turning your switch port configuration into a trunk, with VLAN 37 as the untagged, or native VLAN and 2525 as an allowed trunked VLAN. With this configuration you're still separating VMs from each other, and you can still create a firewall VM that has interfaces in both VLANs. The configuration in AHV would remain the same.

In this second proposed configuration all of the VMs in 2525 would be able to communicate with each other over the physical network, and you could have just a single firewall VM on one host for all of the VMs.

View original

4 replies

Userlevel 7
Badge +35
Hi @AlanS

Thanks for sharing, I'll see if I can get some folks to review this 👍
Userlevel 3
Badge +14
@AlanS I should be able to help out.

The configuration you're proposing creates an isolated network on VLAN 2525 that's unique to each AHV host. VLAN 2525 is not present on the physical switch, so any traffic from one VM to another on VLAN 2525 will only extend as far as a single AHV host. This creates 3 separate networks (one on each host), so you're also going to need 3 separate firewalls, one on each host.

This probably isn't what you want, since a VM in VLAN 2525 on host 1 will not be able to communicate at all with a VM on host 2 in VLAN 2525. If this IS what you want, let me know and we can work through the setup.

Otherwise I would recommend turning your switch port configuration into a trunk, with VLAN 37 as the untagged, or native VLAN and 2525 as an allowed trunked VLAN. With this configuration you're still separating VMs from each other, and you can still create a firewall VM that has interfaces in both VLANs. The configuration in AHV would remain the same.

In this second proposed configuration all of the VMs in 2525 would be able to communicate with each other over the physical network, and you could have just a single firewall VM on one host for all of the VMs.
Userlevel 7
Badge +35
Hi @AlanS

Did you see the reply from @bbbburns - let me know how things went.
Howdy!
Sorry about the late reply. I continued to work with support and we were able to make it work eventually with how you described using trunk instead of access switch mode. Thanks for the help!

Reply