Setting up a Virtual Router on KVM

Hello everyone! Not sure how helpful this article will be, but I found it quite helpful for myself, and I just want to really just write down what I did so that if I have to do this in the future, which I have now done this about 16 times in the last 4 years, I have a reference.

The premise of this article is mainly how to create a Linux router in a Virtual machine so that you have direct access to your VM network from any machine on your network.

Many of us that have virtual home labs, usually will use network segmentation to separate our VM’s. For example, you may want to build an OpenStack lab, but not want it to be impacted by your home DHCP server or impact that network so that your kids or guests don’t mess around with it, so you’ll put it on a private network that only those VM’s can access, and perhaps use NATing for Internet access. While this does work, sometimes if you want to work on the systems, like if you spin up a Horizon server, you need a jump box on both your regular network and your internal network, which can be a hassle. Or, if you want to have some people have access to your environment, but don’t want them on the full network, this method works really well.

Basically, I came up with this need about 4 years ago when I worked for a company that had very strict networking policies. I was testing OpenStack in our Hyper-V environment, but it had no access to the Internet. To get around this, I created a VM on the Hyper-V host that had 3 NIC’s, one that used the Hosts Network adapter that had access to the Internet for updates as the main egress port, another NIC that was used to manage VM’s from my workstation, and the last was an internal network that was going to be used for the intercommunications of the OpenStack nodes. ‘

This VM I decided was going to run CentOS, since the company was a Red Hat shop, and I am quite a bit more familiar with Red Hat (even though as I write this, I found that as I have worked with Canonical for over a year now, I have forgotten some of the slight differences between the two). I managed to build a CentOS router, and it worked. I was able to get my machines in the private network out to the internet without having to NAT each one out the internet port that would have caused bottlenecks with the other VM’s, and the best part, I was able to connect directly to the VM’s from my workstation without needing a jump box, so I could share the OpenStack environment with my co-workers and they could test it.

So, in my house, I am doing something quite similar. I have a KVM host that has 4 networks, my external network with my private IP addresses, my internal network on the 10.1.10.0/24 subnet with its own DHCP and DNS servers, and my private internal KVM network that is not NAT’d (192.168..2.0/24) and my KVM NAT’d network (192.168.122.0/24).

Now, I know what your thinking, why didn’t I just use the NAT’d address range and all my machines would have access to the internet and I could download files and not have to do all this. Your correct on one part. The machines would have access to the Internet, and they have access to everything on my internal network, however, its one way only. I cannot on my workstation connect to those servers unless I use a jump box, which i do not want to do. Of course, I could have adjusted the settings in KVM network or even added the NAT’d routers IP address to my Routing table on my core router point the KVM host as the next hop for resolution. That I can do in my home lab, but what if I’m not running KVM? What if I’m running Hyper-V or VMware ESXi? While it is possible to do the same thing on the other Hypervisors, if you are not familiar with Powershell or the esx-cli command, you could spend hours on this, and potentially break the core networking on those hosts. This method is quick and somewhat painless.

First thing you need to do is build a VM, with NIC’s on each network segment you want it to manage. In this example, I just put two, one on my Internal 10 network, and one on the non-NAT’d network. I installed CentOS 7 on this, minimal install, and I gave it a static IP on my 10 network, the gateway and DNS servers on that network, as well as the 192.168.2.1/24 IP address on the other interface but no gateway or DNS. After it was installed, I ran yum update to update the server and rebooted it. After the reboot, I enabled IPv4 forwarding in the /etc/sysctl.conf file by adding net.ipv4.ip_forward = 1 to it. Then run sysctl -p to make the changes take effect. Now we are ready to setup the firewall rules to allow IP masquerading and forwarding. Run ip a to see the devices and what networks they are connected to. Then, run:

firewall-cmd --direct --add-rule ipv4 nat POSTROUTING 0 -o ext-eth -j MASQUERADE
firewall-cmd --direct --add-rule ipv4 filter FORWARD 0 -i int-eth -o ext-eth -j ACCEPT
firewall-cmd --direct --add-rule ipv4 filter FORWARD 0 -i ext-eth -o int-eth -m state --state RELATED,ESTABLISHED -j ACCEPT
firewall-cmd --zone=trusted --add-source=192.168.2.0/24

That is it on the server. Now, on your router, the main one, you need to add the static route to it so that it knows how to forward packets to your 192.168.2.0/24 network to it. Most home routers have this capability in the Advanced section usually labeled “Static Routes.” Here, enter in the network, 192.168.2.0 and the netmask or 255.255.255.0, and the next hop or source IP depending on how its labeled will be the IP address of your Virtual Router you just built, on the 10.1.10.0 network. Give it the static IP address you gave the router, and save the configuration.

Now test that you can get to a Virtual Machine that is attached to the 192.168.2.0 network and is using your virtual router as its gateway.

ping 192.168.2.2

You should get a reply. Try to SSH to that machine and if you get it, your done. Last thing you need to do if everything test right, is make the firewall rules permanent, but typing the following:

firewall-cmd --permanent --direct --add-rule ipv4 nat POSTROUTING 0 -o eno16777984 -j MASQUERADE
firewall-cmd --permanent --direct --add-rule ipv4 filter FORWARD 0 -i eno33557248 -o eno16777984 -j ACCEPT
firewall-cmd --permanent --direct --add-rule ipv4 filter FORWARD 0 -i eno16777984 -o eno33557248 -m state --state RELATED,ESTABLISHED -j ACCEPT
firewall-cmd --permanent --zone=trusted --add-source=192.168.2.0/24

And thats it. You can do this for any other network you build in your VM environment if you want to be able to access those machines from any other client.

If you have any questions, or just want to leave a comment on if this helped you, leave ’em on the bottom.

Thanks!

[ayssocial_buttons id=”2″]