L2TP VPN Server Setup Issues

VPN
1

EDIT: I’ve decided to try a different route and give OpenVPN a shot. I’ll leave this up in case someone who has this issue finds the comments helpful.

I followed this guide and when I try to connect to my VPN in Windows 7 I get:

Error 789: The L2TP connection attempt failed because the security layer encountered a processing error during initial negotiations with the remote computer.

All the ports are forwarded, I followed the tutorial’s instructions to the T, all the while substituting my IP addresses with the tutorials, and I don’t think I forgot any…I’ve triple-checked.

Any idea what could be happening? I’m pretty tech-savvy but in a self-taught kind of way, so the error message doesn’t mean anything to me.

2

Setting up L2TP and IPsec correctly can be rather difficult on Linux. If I may make an alternate recommendation, consider using SoftEther instead. It’s open source, compatible with the Pi, and you can use it to set up SSTP, which is secure and compatible with Vista and above.

3

Where are you trying to connect from? If you’re trying to connect to the VPN server on the LAN from another machine on the LAN (for testing purposes) you may run into issues.

4

For one, make sure your IPSec service is running. I know XP notoriously doesn’t have it on, and I’ve seen it happen once or twice with 7.

Do you see any logs on the rpi’s end?

5

hello, ssl/vpn barracuda networks technician here. A couple of things would help troubleshooting if you’re still having the issue:

here are some commands that will help out:
netstat -tulpan: should see 500 and 4500 udp and 1701 tcp all on 0.0.0.0/0

if you’re using arch-arm:
sudo systemctl status -l xl2ptd
sudo systemctl status -l openswan.service (assuming you’re using it)

these will give you tail’d log output from the services (make sure they’re up)

or if you’re on rasbian look at the init script to see where you’re logging to.

IPsec goes through 2 phases, this one looks like the initial connection is failing. (phase1 = IKE, setting up a diffie-hellman for your key exchange)

basically, we can’t even exchange keys because you’re not getting to the point where we even have an established connection

if you’re NAT’ing over a firewall, take a look at what type of NAT you have set up. If we’re changing the IP of the incoming packets to an internal address, can the pi find it’s way back out?

tcpdump -ni any -w dump -s 65535 udp port 500 or udp port 4500 or udp port 1701 or tcp port 1812 or tcp port 1813

this tcpdump command will help with IPsec, you can use -w /path/to/file.pcap and take the output and look at it in wireshark

also it looks like this ipsec.conf is for passthrough, you might want to set up a no-NAT configuration. (especially if you’re using a NAT firewall and it’s fucking things up)

edit: this gentleman has another tutorial that might be more suited to what you’re looking for.

this is assuming that you want to use l2tp in conjunction with SWAN (which you should be doing. neither protocol is perfect hence why they are implemented together)

6

I second using SoftEther VPN. Very simple to set up and works natively with OS X and iOS (huge plus for me).

7

Thanks for this suggestion, I had connection troubles setting up a VPN server (first I tried OpenVPN, then L2TP) and I couldn’t figure it out. Today I tried setting it up with SoftEther using this tutorial: http://tomearp.blogspot.nl/2013/11/setting-up-l2tpipsec-vpn-with-softether.html and it worked!

8

As a matter of fact I am trying to do exactly that, I’m attempting to connect from my desktop (192.x.x.51), my Pi is 192.x.x.21. Am I having issues because my VPN server “host” is on the same network as the desktop “client”, and the ports are somehow being overworked, or some other seemingly obvious hiccup?

Actually I have a sub-problem as well if you don’t mind weighing in on it…I assigned my Pi a static IP in my router (192.x.x.100) but every time I connect to my home network it keeps getting 192.x.x.21. I was about to restart my router and figured I’d ask here first, as it’s related to the successful setup of a VPN.

9

Please note I’ve given up on the old issue and tried OpenVPN, though I still can’t connect haha. Perhaps you can still help me?

Whenever I reboot it informs me at startup before I log in:
starting virtual private network daemon: server failed!
startpar: service(s) returned failure: openvpn … failure!

Obviously something’s wrong, I just have no idea what. Everything looks the way it should, all the fields are correct.

10

Maine the pi is not using dhcp? Or maybe something else is also giving out ips?

11

You’re trying to VPN from, say, 192.168.1.51 to 192.168.1.21? Why? And, that is likely not going to work. You would want to be on different subnets. You want to do this from an external location.

12

I run an openvpn server so maybe. Show your client and server configs

13

I’m not sure, I’m using Comcast if it makes a difference. I have the router’s DHCP range set to 192.x.x.1 - 192.x.x.20, and have assigned my desktop, laptop, and supposedly Pi static IPs above x.20, with ports 500 + 4500 UDP and 1701 TCP forwarded to the Pi’s static IP…it’s just getting the Pi to accept its IP address that’s giving me a problem. When I plug the Pi directly into the router via network cable, ifconfig yields no IP address, so I changed the static IP in the router to point to the new eth0 MAC address instead of the old wlan0 MAC address.

14

Yea that makes sense, I was hoping to test it at home but it sounds like that isn’t going to happen unless it’s with an cell phone. I’m going on a camping trip this weekend so I’ll post back when I’m back

15

Wikipedia states that udp port 1701 is used, not tcp. As far as I can remember I’m not forwarding top 1701 in my firewall, only udp.

16

Yeah you cannot VPN into a subnet from the same subnet without some very awkward config. Also, make sure the network you are VPN’ing from is not identical in IP to the one you’re VPN’ing to. For instance, if you go to work and VPN from 192.168.20.25 and are connecting to your home at 192.168.20.1 you will have issues.