Need help setting up a split tunnel on Raspberry Pi

VPN
1

So I have my RPi set up with openvpn (privateinternetaccess) and it’s working well. However I’d like to exclude at least one program from running through the VPN. From what I’ve read I’d need to do this via split tunneling. Is that correct or can I redirect a specific programs traffic before it even gets to openvpn? If I need to split the tunnel, can someone tell me how I’d set that up or point me to a guide? Afaik the openvpn service doesn’t come with the openvpn web ui which is what the openvpn website points to to set up split tunneling. So I’d have to manually edit the config files.

Would be great if someone could help me with this :slight_smile:

Edit:

SOLVED. ip routing works when accessing specific IPs.

Another method that works is using docker. Creating a container automatically bridged the direct ethernet connection so it bypassed the VPN by default.

2

iptables has a match on owner and you can use together with mark to a different routing table.

3

So you’ve got a couple of bits of info that are muddying things for you.

With VPN apps running on phones/desktop windows and Mac(I think), they are able to do split tunneling by app. When you move to a general client, like just the normal VPN client, you can still do a split tunnel, but you do it the “normal” way, which is at the network level. You can’t do it per app.

Split tunnel is really just a phrase to encompass routing traffic to specific gateways. When you have a VPN active, somewhere there is an additional gateway for outbound non-local (i.e. Internet, generally) traffic. Those consumer VPN clients typically will just set all your outbound traffic to use the VPN gateway, but your normal gateway to the Internet is still there and active, it’s just a matter of selecting what traffic goes out it.

Ok, so that said, specifically how you do this with a general VPN client is with route rules. Unfortunately, this means you do have to know/learn some TCP/IP networking to make this happen. What you are doing is inserting a route, that says for a specific destination(IP address), always use a specific gateway, in your case, your normal Internet gateway instead of the one created by the VPN connection.

Now, you’ll notice that this is by destination, not app. This is the typical way split tunnels happen at a network level- those client apps can only do what they do by poking around system internals and being able to pick traffic from other running processes. I don’t know if any of the client apps can do that on Linux. So what you need to do to do this is know the IP address(or address range) that you want to have on the split, and what your local gateway is.

So a trivial example. We’ll assume your local gateway, your router, is sitting at 192.168.1.1, which is does for a lot of people. And lets say you want Google DNS to not go over the VPN, which is 8.8.8.8. In that case, the command(as root) should be:

ip route add 8.8.8.8/32 via 192.168.1.1

That should cause the Google IP to always use your gateway. There’s some additional complexity with route weighting, but I think the more specific the affected IP destination, the more preferential it is. The /32 means to only apply to that IP, specifying something else would get you a larger set of IPs. And the “via 192.168.1.1” is just like it sounds, it’s saying use that as the gateway for that destination. This is setting up a split tunnel for that IP address. So, you should now be asking “so for an app, I need to add a rule for all the IPs it uses that I don’t want to go over the VPN?” and the answer is, unfortunately, yes. On the plus side, you don’t need to do it more than the one time, so if you have multiple apps using the one IP(Like the DNS example) the one rule covers the destination for everything on that system.

So as you can see, you do end up needing to know a little TCP/IP networking, and a little about your local network config, to do split tunnels manually. Hopefully that’s enough to get you going. I don’t remember if those rules are persistant by default, I don’t think they are, you may have to add another switch, (-p?) to make it stay between reboots. Incidentally, being there all the time shouldn’t hurt, like, if your VPN is off, it won’t stop working, as long as your router’s address doesn’t change.

4

Hmm…that’s a neat one I didn’t know about. I’ll have to look into that, though that gets even more complicated than my solution since it involves adding another route table too. :smiley:

5

Thanks a lot for the detailed answer! Sounds fairly doable. Just need to filter out which IP(s) my program is talking to, that should be doable, I think. Just to be sure, the local gateway I’m routing to is the local IP of my router, not of the device itself, right?

Thanks again!

6

Great response thank you!

I want to achieve kinda the opposite of what OP wants here. I want all traffic to go to the router as normal (public internet), EXCEPT a few docker containers and maybe other programs outside of docker in the future.

So it’s a Rasberry Pi with docker. The nice thing with docker is that it creates it’s own networks. I have been reading about split tunneling for a while and most methods are either too complicated for me or methods that want to route based on destination.

My thought is; can’t I route to the tun21 interface based on source? Since each docker container has it’s unique IP (or even better have the containers on one network let’s say 172.17.1.0/24), can’t I route traffic originating from that network to the tunnel while still having access to it on my home LAN?

7

Just to be sure, the local gateway I’m routing to is the local IP of my router, not of the device itself, right?

Correct, your router is acting as your gateway, in plain english, the route rule you are adding is saying “for this destination, always use this address as the gateway”. When the VPN is not on, the router is your gateway anyway; when your VPN is enabled, it changes the default gateway to be its settings. This is how it captures all your network traffic and gets it to use the VPN instead. The “split” in split tunnel is really referring to the fact that not all traffic uses the same gateway anymore- it’s split between multiple gateways.

The purpose of the gateway is that of being the…gateway to non-local destinations. Traffic that’s intended for local destinations can arrive at the gateway, that’s fine, it’ll put it back on the local network. If you never sent anything to the Internet, only shared things on your local network, you wouldn’t even need a router, really. But traffic that needs to go elsewhere specifically goes to the gateway, which is a router. The gateway has the ability and connections to send traffic onward to destinations outside the local network.

There are situations where you could just use the IP of your device and it’d be fine(split tunneling just your local network- like say you want your network printer or NAS file server to still be accessible while your VPN is on), but it’s probably easiest to just use the router and let it deal with figuring out where to send things- that’s it’s purpose, after all.

8

Good luck if that app uses aws and the IPs change frequently (like Netflix).

9

It’s not actually any different. In the OP case, the VPN is the default gateway, with the router being the target for the exceptions. In your case, the router is the default gateway with some exceptions for a target address or address space. The key thing to understand here is that the VPN and the router are both valid gateways on your local network. So your approach isn’t any different- you still use route rules to change routing behavior, it’s just that instead of the VPN gateway being the thing with exceptions, the default router is where the rules are applying.

Another thing to be aware of is that often times, traffic to a local network isn’t sent to a gateway at all, it’s just put on the network. When you introduce a VPN, this isn’t always true, it depends on how the VPN re-writes the route table when it’s active.

Now, Docker introduces another layer of complexity, and unfortunately, the answer here has a certain amount of “it depends on your Docker configuration”. Specifically, it depends on if you are exposing your containers to the network directly in a macvlan type network, or if you are just using Docker in a basic configuration where the Docker host is passing traffic to containers by port, but all traffic is targetted to the host IP.

In the case where you are working off just ports, that’s very straightfowrad- the internal IPs of the containers don’t matter. Only the IP of the host matters, and then you are back to what I mentioned earlier about the LAN addresses may always just show as local in which case you don’t need to do anything. If a VPN gets involved and local traffic is involved, in that case you just have a route rule for just that IP, and use the local IP of the machine you are putting the rule on, or the IP of the local router as the gateway address. Having to only work with a local subnet address simplifies things.

If you are exposing the containers to the local network, it starts to then depend on how you’ve configured their networks etc. And we’re back to the complexity- you’d need to provide more information about how you are setup. But if you map out the network and the IPs, it might help you see that in fact, it’s not actually as different as you think when you view everything from the network layer. From the network layer, abstractions that you are working with in the container side may not exist, or don’t exist in the same way. From the network side, the network is the network, it doesn’t care about if a thing is in a container or a VM or is a physical machine, just the network configuration, the routers, switches, IPs, and VLANs that are in use(and thus the broadcast domains, which is what VLANs/LAN networks are really defining) matter.

10

It’s more work, but you can add routes for those address spaces. That’s one of the reasons why AWS and Azure publish their IP blocks. You can also find the Netflix blocks as well.

There are other ways to restrict the addresses that come back as well, at the DNS/hosts file level, depending on how things are setup, so that you can limit the addresses you have to cope with. There can be side effects for this, but much of the time it probably would work fine.

11

Great thank you. Tell me how to get you the information you need. I’ll tell you here what I know.

  1. Everything I want to work on is on one Raspberry Pi, even the vpn client will be working from there (in the future I want to get an openvpn server working there too).

  2. The Raspi is behind one router (Netgear R7000). The Raspi is the DNS server with pihole, it hosts a reverse proxy too. There are services hosted on another PC in the same local network.

  3. The local network is 192.168.1.0/24 which the Raspi is part of (192.168.1.10).

  4. Docker; I installed Docker then Portainer for the GUI. I didn’t change any settings at all in regards to Docker networks. So most containers are part of the “bridge” network Docker created. That network is 172. 17.1.0/24.

  5. I access the containers via 192.168.1.10:port. They’re also reachable via the docker network addresses after I added a static route in the router.

Now, if having this Docker network complicates things I think I can delete the “bridge” network and have the containers running on the “host” network, I’ve seen an option for that in Portainer. If having the Docker “bridge” network is easier I can do that too.

12

Correct but if you want to route Netflix through vpn but not another AWS domain then the problem becomes bigger.

13

Hello /u/MasterChiefmas

Would you mind taking a look at this again. To recap, I need to route one docker network (172.20.0.0/16) through a vpn interface (tun21).

14

Heya…well start with how the containers are sitting on the network. When you connect to a service on the container, do you go to the Portainer address, or does each container have it’s own address?

That was the long way of asking if you are using a macvlan network for your containers, i.e. they share the nic directly and look like they are directly connected to the network.

15

No I do not use a macvlan, if it’s easier I will switch to it. But right now I’m using the bridge driver, each container has its own IP address, my plan is to have the containers I’m interested in routing through a VPN in one docker network, thinking it would be easier.

I can access the containers from the local network using:

# using the Raspberry Pi IP address
192.168.1.10:9000
# using the container's docker IP address, after I added static routes in my router
172.20.0.2

I can ping and access their web UI’s just fine using both methods above.

Here’s a screenshot of the docker networks in my portainer:

1849×863 133 KB

16

I think I’m halfway there. I think I need to do something like this:

ip rule add from 172.20.0.1 lookup vpn
ip route add default via 10.13.112.1 dev tun2 table vpn

Where 10.16.112.1 should be the gateway for tun2.

My problem now is when I start the openvpn client:

sudo openvpn 123.ovpn

It changes the routing table, nuking my new rules.

$ ip route
0.0.0.0/1 via 10.16.112.1 dev tun2
default via 192.168.1.1 dev eth0 proto dhcp src 192.168.1.10 metric 202
10.16.112.0/24 dev tun2 proto kernel scope link src 10.16.112.203
128.0.0.0/1 via 10.16.112.1 dev tun2

I need openvpn to run silently in the background without interfering.

Also see my post:

https://www.reddit.com/r/OpenVPN/comments/tgqx9m/how_to_stop_openvpn_from_changing_the_routing/

17

bridge driver, each container has its own IP address

Ok, got it. This part was what I was trying to clarify. Containers _always_ have their own IP address. It’s a question of whether said IP is visible to the external world or not.

18

I suggest before we work on ignoring the openvpn route table changes, making sure your route rules work, if possible. Otherwise we’re muddying the water with 2 things happening at once.

So, start with having openvpn connected, and then add your route rules. You want to know if they are sending traffic over the correct interface. If you don’t have any other mechanism to do so, you might want to set up a small container with a shell or a tool that you can use to easily have it report what the outside world sees as it’s IP address.

Once you’re sure the rules are working as expected, then you can start looking into getting openvpn working as you want it to.

19

The answer to you question is that when I had the vpn connected I would see the VPN’s public IP, if not I would my my own IP.

I think I achieved what I want, at least temporarily. Right now, the openvpn client config includes:

pull-filter ignore "redirect-gateway"

connect to openvpn:

sudo openvpn --config 123.ovpn --daemon

then add rules:

ip rule add from 172.20.0.0/16 lookup vpn
ip route add default dev tun0 table vpn

This way, on the rasberrypi I see my own IP. But from within the container I see the VPN’s IP.

curl icanhazip.com

The trick seemed to be adding the rules AFTER establishing the VPN connection.

Now, how do I make this persistent after reboot and in that order?

EDIT:

This command is needed after each reconnect/disconnect to the VPN.

sudo ip route add default dev tun0 table vpn

Why doesn’t this show on ip route? where can I see it?

20

Glad to see you got it all working!

So yeah, does seem like the VPN was overwriting your route table rules.

As for the ip route…try ip route list table vpn.

You aren’t seeing it because ip route with no switches is showing you the default route table, and you are adding rules to a table named ‘vpn’.

If it’s not there, try ip route list table all it should be in there somewhere!