Routing issues with single Azure vNet with a VPN Virtual Network Gateway and another Express Route Virtual Network Gateway

VPN
1

We originally setup a VPN Gateway in Azure on a single vNet that connected two separate data centers to Azure. Azure’s vnet has an ip address range of 10.10.0.0/16. DC-A has an ip address range of 10.1.0.0/16 and DC-B has an ip address range of 10.2.0.0/16. Recently Express route was ordered. We setup another virtual network gateway for express route that has bgp configured on it in the same vNet up in Azure. Locally on our data center firewalls, we setup a route to use the express route connection with a metric of 20. We setup another route to use the VPN tunnel with a metric of 25 as backup in case express route failed, the VPN would take over.

When local traffic from one of the DCs is sent to Azure it would go via the Express route connection to Azure. When the endpoint in Azure responds…

  1. Where does the return traffic get sent? Does it get sent to the VPNGW or the EXPRSSGW? Is there a way to weight the gateways so that all traffic goes to the EXPRSSGW if that is up and if it is not, it would then send it to the VPNGW?

  2. If I created a custom route table for my vNet in Azure to override the default routes, I could put in the local IP addresses for each data center, but how can I choose which virtual network gateway to use? I can create a route in a custom table but for next hop, I choose virtual network gateway…but it won’t let me specify which one to use. How can I do that? I could also choose virtual appliance and specify an IP address…but for one I don’t know how to find the internal IP address of each gateway secondly, there is no weight or metric on the routes that I create in Azures custom route table.

  3. Even more basic, that single VPNGW connects to both data centers…If that return traffic from Azure wants to go to 10.1.0.0/16 or 10.2.0.0/16, the default routes will take it to the same VPNGW but how does it know which VPN tunnel to take? Is there somewhere in the VPNGW configuration where you specify which subnet belongs to which tunnel or does it learn from the local data center connecting VPN endpoint.

2

I was able to figure out the answer to #3. The IP subnet for each local data center is defined in the “connection” to the local VPN endpoint. But still no answer to 1 & 2.

3
  1. Return traffic goes to the GatewaySubnet, but which gateway is used depends on where the ExpressRoute and VPN are connected to, and what routes have been configured?

Since both ExpressRoute and VPN are connected to each of your DCs, VPN as backup, the ExpressRoute is used. ExpressRoute, once implemented, is always preferred, unless a VPN connection is configured with a more direct route. So my assumption is that the routes propagated to your ExpressRoute are the same as configured on your VPN.

If, however, the routes for your ExpressRoute and VPN are not the same, say your ExpressRoute has 10.1.0.0/16 but you have a route of 10.1.1.0/24 on your VPN, and the destination address is 10.1.1.10, the VPN will be used.

  1. Since you have one vnet, with both gateways deployed, you are limited in what custom routes you can configure. The GatewaySubnet, once you configure VPN and ExpressRoute connections, will add routes to the vnet route table for the routes propagated from the ExpressRoute via BGP, and are configured for the VPN Local Network Gateway, next hop being Virtual Network Gateway.

You cannot configure weights on specific routes or gateways in Azure. ExpressRoute is always preferred, once implemented, and only a more direct route to any route on the ExpressRoute, will be preferred.

Like my example earlier, if you wanted to preference a VPN connection over ExpressRoute, you would configure the VPN Local Network Gateway with a more direct route. Any destination address failing into that more direct route would go via the VPN, everything else would go via the ExpressRoute. But with the weighting you have in each DC, doing this would create asymmetric routing, since traffic from a DC would use the ExpressRoute, and if the address matched the more direct route, it would use the VPN going back.

4

thank you! much appreciated