UDM SE: Extremely slow VPN speeds compared to Asus router. What am I doing wrong?

VPN
1

Hi!

Last weekend, I replaced my Asus AX88U router with a new UDM SE (UniFi OS 3.2.7). Inside my network I have an unRAID server running a WireGuard server which has been working really nice. I use the WireGuard server daily to access my home network from work. But since replacing the router with the UDM SE the speeds over WireGuard are just terrible. I’ve tried the internal WireGuard server of the router and the internal OpenVPN server as well. All three options have exactly the same slow speeds.

I switched back to the Asus router to test, and the WireGuard speeds went back up immediately.

Both sites have fiber connection 1000/1000 Mbps. I’ve connected my fiber at home directly into a Fiber SFP in port 10.

Speedtest at home:

󰒍 root on unRAID at 󰋜 ~ ./bbk_cli
Start: 2023-12-13 08:03:11
Network operator: Tele2 Sweden
Latency: 5.942 ms
Download: 951.093 Mbit/s
Upload: 952.926 Mbit/s
Service provider: TELE2

Speedtest at work:

 Administrator on patrik at …\Documents .\bbk_cli.exe
Start: 2023-12-13 08:27:46
Network operator: Bahnhof AB
Support ID: gbg10351d98b
Latency: 1,970 ms
Download: 915,665 Mbit/s
Upload: 928,414 Mbit/s

Iperf over WireGuard (server on unRAID) with Asus router. Iperf server running at home and client at work:

 Administrator on patrik at …\Documents iperf3 -c 10.13.37.10
Connecting to host 10.13.37.10, port 5201
[ 5] local 10.13.38.2 port 60751 connected to 10.13.37.10 port 5201
[ ID] Interval Transfer Bitrate
[ 5] 0.00-1.00 sec 105 MBytes 877 Mbits/sec
[ 5] 1.00-2.00 sec 103 MBytes 861 Mbits/sec
[ 5] 2.00-3.00 sec 103 MBytes 867 Mbits/sec
[ 5] 3.00-4.00 sec 103 MBytes 862 Mbits/sec
[ 5] 4.00-5.00 sec 103 MBytes 866 Mbits/sec
[ 5] 5.00-6.00 sec 104 MBytes 869 Mbits/sec
[ 5] 6.00-7.00 sec 104 MBytes 878 Mbits/sec
[ 5] 7.00-8.00 sec 104 MBytes 868 Mbits/sec
[ 5] 8.00-9.00 sec 104 MBytes 875 Mbits/sec
[ 5] 9.00-10.00 sec 104 MBytes 871 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate
[ 5] 0.00-10.00 sec 1.01 GBytes 869 Mbits/sec sender
[ 5] 0.00-10.04 sec 1.01 GBytes 863 Mbits/sec receiver

Iperf over WireGuard (server on unRAID) with UDM SE. Iperf server running at home and client at work:

 Administrator on patrik at …\Documents iperf3 -c 10.13.37.10
Connecting to host 10.13.37.10, port 5201
[ 5] local 192.168.2.2 port 60917 connected to 10.13.37.10 port 5201
[ ID] Interval Transfer Bitrate
[ 5] 0.00-1.01 sec 2.00 MBytes 16.5 Mbits/sec
[ 5] 1.01-2.01 sec 1.75 MBytes 14.8 Mbits/sec
[ 5] 2.01-3.00 sec 1.75 MBytes 14.7 Mbits/sec
[ 5] 3.00-4.01 sec 1.88 MBytes 15.6 Mbits/sec
[ 5] 4.01-5.03 sec 1.75 MBytes 14.4 Mbits/sec
[ 5] 5.03-6.01 sec 1.75 MBytes 15.0 Mbits/sec
[ 5] 6.01-7.01 sec 1.75 MBytes 14.6 Mbits/sec
[ 5] 7.01-8.01 sec 1.88 MBytes 15.8 Mbits/sec
[ 5] 8.01-9.00 sec 1.75 MBytes 14.7 Mbits/sec
[ 5] 9.00-10.01 sec 1.88 MBytes 15.6 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate
[ 5] 0.00-10.01 sec 18.1 MBytes 15.2 Mbits/sec sender
[ 5] 0.00-10.02 sec 17.9 MBytes 15.0 Mbits/sec receiver

Speedtest at work with WireGuard enabled (when UDM SE is serving as router at home):

 Administrator on patrik at …\Documents .\bbk_cli.exe
Start: 2023-12-13 08:40:16
Network operator: Tele2 Sweden
Support ID: gbg10b2a9936
Latency: 11,344 ms
Download: 562,764 Mbit/s
Upload: 32,447 Mbit/s

So it seems that its upload from client to server through the UDM SE that is extremely slow compared to my old Asus router under the same conditions. And as previously said, the results are the same with WireGuard on unRAID, WireGuard on the UDM SE and OpenVPN on the UDM SE.

What am I missing? I have I misconfigured something in the router that is slowing down the traffic, or have I missed configuring something? The UDM SE should be at least comparable to a consumer grade Asus router, right?

2

Hello! Thanks for posting on r/Ubiquiti!

This subreddit is here to provide unofficial technical support to people who use or want to dive into the world of Ubiquiti products. If you haven’t already been descriptive in your post, please take the time to edit it and add as many useful details as you can.

Please read and understand the rules in the sidebar, as posts and comments that violate them will be removed. Please put all off topic posts in the weekly off topic thread that is stickied to the top of the subreddit.

If you see people spreading misinformation, trying to mislead others, or other inappropriate behavior, please report it!

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

3

The UDM SE should be capable of around 9Gbit routing WAN to LAN when you don’t have IDS/IPS enabled, around 3.5Gbit with, and about 300mbit with Smart Queues enabled regardless or IDS/IPS setting. That’s all regarding internet-bound traffic though, if you’re having issues between local devices then it’s probably something else.

I get about 700Mbit between my UDM Pro and laptop at work through WireGuard (running on the UDM Pro with IDS enabled). Actually a bit surprised I don’t get more, might have to look into that myself.

Are you using identical WAN and LAN cabling when using either router or could the issue be the cables you’re using when plugging in the UDM SE?

The UDM SE (as well as the UDM Pro) have a flaw in their design regarding traffic between the internal switch and the RJ45 WAN and SFP+ ports, it’s capable of switching full speed across all the internal switch ports, but traffic going to and from the other ports is limited to a total of 1Gbit. That doesn’t seem to be the issue here either, but it’s good to know as a user of such a gateway.

Is there other network equipment involved between the test source and the target, or is it “only” the internet and the router?

If you just run iperf3 on a computer on the LAN and test speed between that and the unRAID machine, without WireGuard, do you get the full expected 1Gbit in both directions?

If you run speedtest from LAN to the internet, without WireGuard, do you get the full expected 1Gbit in both directions?

In settings, under Security > Traffic Rules, do you have any Speed Limit traffic rules configured?

It might help debug the problem if you could list any settings you’ve played with.

4

Do you have jumbo packets turned on? Perhaps some weird fragmentation is occurring

If possible take packet captures of your wireguard traffic with Asus router and Ubiquiti one and compare those in wireshark

You could also try and tune the wireguard packet sizes to see if it makes a difference

5

Downgrade your UniFi OS 3.2.7 to 3.1 or 3.0.x and all will be fine.

6

Thank you for your reply!

Yeah, from what I have gathered before my purchase it should be a very competent machine, so this shouldn’t be an issue, so it feels like a misconfiguration or a bug.

Same identical cabling, tried a few other cables as well. With the Asus router, I plugged in my fiber to my fiber converter from my ISP and Ethernet to WAN on the Asus router. I tried to use the Ethernet WAN instead of SFP on the UDM SE as well, but same results.

Thanks for the heads-up on the flaw, but since I only have Gigabit fiber it shouldn’t be an issue for now, right? Or would it be beneficial to use Port 8 as WAN? If I were to get higher tier fiber in the future, I guess I would have to use the LAN SFP 10G to an 10G port on a new switch, that would work around this flaw?

No other network equipment between, except for the Fiber Converter, which I have tried with and without. Here is the network topology without the clients:

https://imgur.com/36LWq1G

Fiber Internet is currently connected directly into SFP port 10. unRAID is connected via Ethernet on port 6 (1m cat 6).

Internal LAN speeds seems to be fine from the tests I have done. I copied a file from unRAID over Samba to my Desktop which is connected to my Flex Mini “Switch Kontor” in the above topology, and it was steady at around 110 MB/s.

However, one other small thing I noticed when running Ookla Speedtest Windows app on my Desktop connected to my Flex Mini as above, was that I was only getting around 600–700 Mbps download and full upload at around 940 Mbps. When I connect everything to my Asus router and running the same switches (I guess they run as dumb switches without a controller) the speeds immediately went up to ~950/950 Mbps in the Speedtest app on the same server. I don’t know if this is related or if this is a separate issue. LAN speeds seem ok, so it sounds like something with the UDM SE when routing the traffic to and from the internet.

Running Bredbandskollen CLI (Swedish equivalent of Speedtest) in the command line on unRAID I get the expected speed without WireGuard yes. I did get marginally better speeds with my Asus router, but it is only around 10 Mbps difference. First 24 hours the speed was fluctuating a bit. Seems pretty stable since last night though:

https://imgur.com/sVWV3aZ

(Marker is at when I switched routers, running speedtest to Grafana every hour)

Iperf3 between Desktop and unRAID on LAN works as expected, I would say, 910-940 Mbps both directions.

No traffic rules configured.

I’ll try to specify what I’ve changed in my initial config:

  • Changed LAN subnet to 10.13.37.0/24.
  • Added WiFi
  • Added port forwardings for my internal applications including WireGuard to unRAID
  • Added static route to WireGuard on unRAID (tried to remove it but no difference): https://imgur.com/j4zuCC2
  • Tried disabling these but without effect: https://imgur.com/1eTP2fz
  • Tried enabling flow control on all switches (no effect, disabled again)
  • Enabled WireGuard and OpenVPN servers on the UDM to test, same bad speeds.
  • Enabled autoupdates
  • Changed primary WAN to SFP port 10
  • Added expected ISP Speeds (1000/1000) under Internet settings

Other than that, I think it’s mostly default settings.

I did try to run iperf3 with multiple streams and the speed seems locked per stream?:

Connecting to host 10.13.37.10, port 5201
[ 5] local 10.13.38.2 port 63821 connected to 10.13.37.10 port 5201
[ 7] local 10.13.38.2 port 63822 connected to 10.13.37.10 port 5201
[ 9] local 10.13.38.2 port 63823 connected to 10.13.37.10 port 5201
[ 11] local 10.13.38.2 port 63824 connected to 10.13.37.10 port 5201
[ ID] Interval Transfer Bitrate
[ 5] 0.00-1.01 sec 1.62 MBytes 13.5 Mbits/sec
[ 7] 0.00-1.01 sec 1.62 MBytes 13.5 Mbits/sec
[ 9] 0.00-1.01 sec 1.62 MBytes 13.5 Mbits/sec
[ 11] 0.00-1.01 sec 1.62 MBytes 13.5 Mbits/sec
[SUM] 0.00-1.01 sec 6.50 MBytes 54.0 Mbits/sec

7

Jumbo frames are disabled.

I’ve tried to tune WireGuard MTU without success.

After a factory reset, I have concluded that it must be a faulty unit, so I will be returning it.

8

Is this a known issue in later releases? I upgrade from 3.1.16 up until yesterday with exactly the same issue. It autoupdated to 3.2.7 last night and tried with the new version this morning but same results.

9

Down-vote as much as you want…i downgraded and my speed issue was fixed…No clue if your case is different…

10

Tried disabling these but without effect: https://imgur.com/1eTP2fz

What about these?

https://i.imgur.com/NB17O0g.jpg

GeoIP filtering?

11

Well my wireguard performance is also bad so it’s not your unit

I lost hope long time ago

12

Don’t listen to them, there are no performance issues with 3.2.7 (at least none that would have the severe effects you describe).

13

Is it these settings you mean?

https://imgur.com/Vesb1VH

All seem off if I understand it correctly.

14

Ok, thanks for the heads up!

15

Yep, that’s all IDS/IPS settings disabled. I’d probably open an issue with UI at this point. You may have a måndagsexemplar on your hands unfortunately.

16

Ok, thanks. No misconfiguration then. Good to know that it at least is supposed to work the way I want.

I’ll try a factory reset of all Unifi gear and test with all default settings. If the problem still persist I’ll open a ticket.

I bought it directly from their EU store, one thing that struck me was that the QC test date was July 2022 - seems like pretty old stock for their own store.

17

No misconfiguration then.

I didn’t say that, just that at this point we’ve exhausted my personal ability to help you figure it out.

A factory reset is a good idea to try. When I get new stuff (especially routers) I usually do a firmware upgrade to latest current build and then a factory reset, before I start using it. I do this because I want a completely clean slate, and it’s very common for defaults to change between versions, so a factory reset on the older firmware followed by an upgrade might not result in the same state as an upgrade followed by a factory reset.

QC test date was July 2022 - seems like pretty old stock for their own store.

I’ve got the feeling that the UDM-SE doesn’t sell as well as the UDM. I may be wrong, but it makes sense to me given that for me the internal switch is pretty much useless, so I don’t know why I would want to power any PoE things from it except maybe cameras.

If it had had all 2.5Gbit ports on the switch, and a proper backplane between all switchports, then that would be the perfect product. But when it shipped with the same issue that the UDM-Pro switch has, I just don’t see the point of it over the UDM-Pro, the only added benefit is 2.5Gbit RJ45, but you’ll need an SFP±connected device to actually use it anyway.

18

Thanks, I’ll try a factory reset and take it from there.

Does the UDM Pro have the same flaw with the backplane as SE?

19

Yeah, they are based on the same platform, the SE just has larger eMMC storage, PoE hardware, and a 2.5Gbit RJ45 port. Everything else is the same, including the 1Gbit link between the internal switch and the processor. Think of it as a 9-port switch where the 9th port is hidden and there’s a hidden cable going from it to the motherboard, providing a 1Gbit link, that’s essentially how it works.

On the UDM-Pro it’s just silly, on the UDM-SE which has 2.5Gbit RJ45 WAN it’s even more egregious if you ask me, because the only port that can utilise the 2.5Gbit RJ45 WAN is the SFP+ LAN port, 3x 1Gbit clients on the internal switch can only share a 1Gbit link to the internet, even if you have 2.5Gbit internet.

This illustrates it well: https://community.ui.com/questions/1GB-backplane-clarification-please/fd947197-ad99-404b-9600-fec832ffba9e#answer/c1feea3e-8e02-483f-8ec9-4952b5caf507

That link mentions that HW version 3.1 of the UDM-Pro had a 2.5Gbit link there (HW version 8 was the one that was released on the store in the end), but the UDM-SE never did according to products:unifi:unifi_dream_machine_pro_se [Ubiquiti Community Wiki].

20

Thanks for the details! Really appreciated.

So an update:

I did a factory reset, and tried it with a base config with nothing else connected but my unRAID server, and got the same results - about 15 Mbps. Speedtests results still varied 700-940 Mbps. Connected my ASUS again and immediately ~800 Mbps over WireGuard and 949/949 Mbps on Speedtest.

So for fun with my Asus connected at home, I set up a WireGuard Server on our UDM Pro at work and ran iperf from home over WireGuard:

Connecting to host 192.168.1.151, port 5201
[ 5] local 192.168.3.2 port 61634 connected to 192.168.1.151 port 5201
[ ID] Interval Transfer Bitrate
[ 5] 0.00-1.00 sec 99.8 MBytes 836 Mbits/sec
[ 5] 1.00-2.00 sec 75.0 MBytes 630 Mbits/sec
[ 5] 2.00-3.00 sec 107 MBytes 899 Mbits/sec
[ 5] 3.00-4.00 sec 108 MBytes 903 Mbits/sec
[ 5] 4.00-5.00 sec 108 MBytes 902 Mbits/sec
[ 5] 5.00-6.00 sec 106 MBytes 893 Mbits/sec
[ 5] 6.00-7.00 sec 108 MBytes 902 Mbits/sec
[ 5] 7.00-8.00 sec 105 MBytes 883 Mbits/sec
[ 5] 8.00-9.00 sec 108 MBytes 904 Mbits/sec
[ 5] 9.00-10.00 sec 106 MBytes 893 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate
[ 5] 0.00-10.00 sec 1.01 GBytes 864 Mbits/sec sender
[ 5] 0.00-10.02 sec 1016 MBytes 851 Mbits/sec receiver
iperf Done.

Just rips immediately, as it should. Speedtests at work on the UDM Pro is constantly 949/949 Mbps. So something is definitely wrong with my router.

Now for some offtopic, hope you don’t mind. I have found your answers and help very helpful.

With the details you have given about the internal switch on the UDM Pro and SE it, as you say, renders the SE a bit more useless since I would never utilize the 2.5 Gbps Ethernet WAN with the internal Switch anyway plus limiting the speeds of connected PoE Devices on a future faster internet connection.

With the current sale running on the UDM Pro the difference is around 3000 SEK. Wouldn’t it be more sensible to just return the SE during the return window for a full refund, order the UDM Pro and put that difference towards for example an USW-24-POE and connect that with an DAC to the SFP ports and using that switch as the main switch and completely ignoring the internal switch off the UDM Pro.

As I can use the 10G SFP WAN port for my Fiber connection anyway, I would not need the 2.5GB one and still have proper backplane to a proper Switch (still 1GB for now if not going for the pro). And connecting PoE devices to that instead of the UDM SE. That would make the setup more future-proof for roughly the same money plus a lot of more switch ports.

Thoughts? Thanks and good night. :slight_smile: