OpenBSD: Gateway outside subnet (works in Linux)

This is an old thread, but here goes.

As it happens, I run a number of OpenBSD VMs on an ESXi 6.0 running at SoYouStart, a daughter company of OVH. The network setup there is the same as with OVH and I think, although strange, its main purpose is to eliminate ARP traffic as much as possible by artificially limiting the broadcast domains, and without the need for using VLANs for example.

In my case, I've requested extra IP addresses from OVH and they come from a completely different range. For the discussion here, let's assume these are my settings:

• my main IP address (which the ESXi Host is using): 213.0.113.78/32
• the extra IP address range for VM guests: 192.0.2.64/30
• the default gateway for ALL of the above: 213.0.113.254
• please note - all hosts need to use a host netmask (255.255.255.255) due to the way the OVH network is configured

To configure the routing on the OpenBSD host, this is what I need to do:

ifconfig vmx0 inet 192.0.2.64 255.255.255.255 
route add -inet 213.0.113.254 -llinfo -link -static -iface vmx0 
route add -inet default 213.0.113.254

To have all this done during the start, I ignore the /etc/mygate file and put the following in the /etc/hostname.vmx0:

inet 192.0.2.64 255.255.255.255
!sleep 2
!route add -inet 213.0.113.254 -llinfo -link -static -iface vmx0
!route add -inet default 213.0.113.254

You will notice the sleep command - for some reason this is required on OpenBSD 5.9 but wasn't before. Without the sleep, the first of the to route commands will not be executed and therefore your routing will not be configured correctly.

This works using the following trick:

• we configure the IP address on the interface
• with the first route command we translate the IP address of the gateway (213.0.113.254) to a link address (MAC address); this is done by the -llinfo option;
• in the same route command, using the -link option, we install the link address to the routing table and using the -iface vmx0 we tell the kernel which network interface that link address is reachable via; the -static switch marks it as a manually inserted entry into the routing table
• the second route command can now succeed, as the route to the default gateway is now known

One thing I did come across in the setup above, which I have no solution for yet, is that one of four OpenBSD hosts, configured in the exact same way, running the exact same version of the OS and the exact same kernel build, and even running on the same VMware host, every 24 hours or so, seems to be "losing" that magic route from its routing table...