My company has this issue where sometimes our clients need support for industrial automation systems based on PLCs (Programmable Logic Controllers) on networks that are not accessible remotely through the internet, and they are 300+ miles away, have no network implementation for remote support, etc.
What we wind up doing is sending a laptop loaded with software and with the help of someone on sight connect via remote desktop and do what we need to do. Depending on network speed and the knowledge level of the on site support this can be painfully slow. I mentioned the possibility that we could skip the laptop and send a cellular router configured for VPN and just connect with a local computer. They liked the idea enough for me to purchase some hardware, I picked out a PC Engines APU3C2 and Sierra MC 7455 for Verizon. I have everything configured and get cellular data to the LAN but the issue I didn’t foresee is how cellular devices handle WAN IP Addresses. I was intending on using Dynamic DNS to address the VPN server and simply tunnel to the local network but it looks like the WAN IP isn’t actually addressable through the internet due to “Carrier Grade NAT” getting crafty with IPv4 and I do not appear to be getting an IPv6 address. Is there any other way to resolve an IP to connect to the Cellular connected “server”?
My next thought was set up a cloud based private VPN server and set up the box and administrating computer both as clients, that would put the PFSense box on the correct subnet but the remote PLC is going to be different based on location. So on initial connection I would log into the PFSense box and change a secondary interface IP to match the subnet of the remote PLC and let PFSense handle routing between the two.
This is unfortunately a bit more complicated than the initial idea and would require a subscription service. Recommendation for subscription service would also be nice.
Does anyone have any other suggestions on how to implement this without forking out $500 for a Verizon static IP? Moral of the story is that I need to have access to a remote device via the internet without exposing either side to security vulnerabilities.
Yeah the mobile CG-NAT problem sucks. The most reliable configuration I found was to have the remote units dial back in with OpenVPN - it’s pretty immune to middleboxes and NAT layers. Basically just enough connectivity so I can remote in and configure for the situation.
Be sure to point your remote firewalls to a DNS name not an IP so you can change it if you have to.
OpenVPN with subnets behind dialups can be a bit squirrely - at least in TUN mode you might have to define a CCD for each remote unit so you can add iroute directives.
My go-to solution is a Ubiquiti ERLite-3 and Netgear LB1120 combo, mainly because I can’t find a decent pfSense box remotely close to $99. The ER-X for $57 also works in testing but I don’t have any in the field yet so I can’t vouch.
If you have a physical office with a static you can probably have them dial straight back there, but if not you can rent a VPS. I’ve loaded pfSense on a Vultr $3.50/mo node and it runs well - their how-to’s are pretty good.
Just a note: I can see you running afoul of security controls here. Just because you can talk a guy from maintenance through hooking up your cell-connected blivet to their plant bus, doesn’t mean you’re not breaking their internal controls by doing that.
Our office does have a static IP but our server/network is handled by an outside company as IT is not our specialty. My access to do anything is pretty minimal, rightfully so as security concerns on our end are pretty stringent. Considering just adding a VPN to my home network, it’s not static IP but I have an NVS server that works fine remotely with Dynu.
Vultr looks like they have a “one click” openVPN Which sounds promising.
In regards to security I agree with you entirely, the only thing that separates this from a network infiltration device is the lack of nefarious intent. There are places where implementing this is “A OK” because they don’t have any hard-fast rules and we don’t intend on using it elsewhere. We would have a purchase order for service before this would get sent out and it would likely be sent to a plant engineer to plug in. We assume liability for any communications that happen through that device but the same applies if we are there in person.
I came to say exactly that. I’ve also used A couple VPNs to Azure to get some inbound ports. Soft Ether is another one that might work, they offer a free vpn that has Azure broker the connection, then they go direct.
Sounds like you know your way around the security pitfalls then.
I’m kind of curious about is whether it’s even plausible to get cell signal in a factory building. The situation I’m working on now is zebra printers in contract shipping warehouses, and I’m afraid that “butler building” construction is going to block everything.
It’s definitely going to be a challenge, hopefully one we can overcome in most cases. Part of why I went the PCIe modem route instead of USB modem was because of the ability to put on 2 high gain antennas. The smaller jobs we work on tend to be low density buildings so we can generally get decent cell service. Definitely won’t work on the bottom floor of a 3 story automotive plant but we wouldn’t use this there anyway due to them actually have a developed network security protocol.
This idea may develop further to include a wireless card for internet depending on the plant. Could still be configured via web if they can initially plug the router in somewhere that they get cellular. If we stay with the client-server-client configuration fail-over wouldn’t be a bad idea but I got a “if it works $500 is no big deal” response in regards to the static IP.
So right now I have this set up with a Vultr server and both devices as clients with static IPs, on the client laptop I have the gateway set to the PFSense box since the PLC is on a different subnet. This seems to work ok, unless I change the LAN interface IP for the PLC subnet to something other than the IP listed as the gateway in the PLC. The PLC is still reachable for “awhile”, then it can no longer be ping’d outside the subnet, despite the fact that the PLC is still updating in the ARP table. Any hint? I would guess that the PLC insists on returning pings through the gateway but can’t see why that wouldn’t be instantaneous.
I’m considering the possibility of going to a layer 2 VPN in the event that we get a static IP. It looks like this would give us the most reliable setup in regards to application compatibility and not have to deal with routing. We could always switch back to L3 Client-Server-Client if wifi fail-over was necessary. Any words of wisdom?
I’ve got around your routing problem a few times by doing source NAT (outbound NAT) on the LAN interface, so that the connections appear to be coming from the same subnet as what you’re connecting to. Then it doesn’t need a gateway to get back and bob’s your uncle.
The hitch is if the PLC protocol won’t play with NAT. The only one I’ve dug into at that level is modbus over TCP, and it’s pretty much “jam the old serial protocol over a TCP socket”. It wouldn’t surprise me if they’re all like that, but I don’t know.
I used TAP once so I could run OSPF - it worked but I lost another 38 bytes off MTU. That just turns the link into a pseudo-wire though - it doesn’t bridge you on the same network. I’ve read a howto about creating a bridge device and adding your TAP and LAN segment, which I think would do what you’re talking about.
I can’t say how well that would work though - maybe it just flings every ARP packet over the wire. The guys in /r/networking seem to flip tables when people talk about bridging L2, but RAS/PPTP used to do it all the time.
Bridging L2 is basically adding in another (virtual) switch, right? The Linux bridge device runs spanning tree, but I don’t know about the BSD one.
You might cause a disruption just by attaching it, if it takes over spanning tree root. And I have no idea how a new switch would interact with that double-redundaloop thing Scalence switches run.
Thanks a ton for the help, I’ve got to back burner this for awhile and get back to some other work but looks like it’s heading in the right direction. Going to do some experimentation with L2 once we get a job that we can bill the static IP to.