GRE Tunnel MTU and MSS Clamping for DDoS Tunnels

A GRE tunnel MTU on a standard 1500-byte underlay should be 1476 bytes, and TCP MSS should be clamped to 1436 for IPv4 (1416 for IPv6). GRE adds 24 bytes of overhead (20-byte outer IPv4 header plus 4-byte GRE header), so anything larger than 1476 inside the tunnel is fragmented or dropped, and if ICMP is filtered the result is a classic PMTUD black hole.

This matters a lot for DDoS tunnels. When a scrubbing provider delivers clean traffic to your router over GRE, a wrong MTU doesn't break the BGP session, because BGP packets are small. It breaks real traffic: TLS handshakes hang, large uploads stall, and some websites load "half way". This guide covers the math, the failure modes, and working configs.

The overhead math

MSS is the tunnel MTU minus 40 bytes (20 IP + 20 TCP) for IPv4, or minus 60 bytes (40 IPv6 + 20 TCP) for IPv6. Optional GRE fields (key, checksum, sequence) each add 4 bytes. Most DDoS tunnels use plain GRE, but check your tunnel parameters before you copy numbers.

If your underlay is not 1500 (PPPoE at 1492, a VLAN or QinQ transport, a cloud network with a smaller MTU), subtract the difference. If both tunnel endpoints and every hop between them support jumbo frames, you can raise the underlay MTU to 1524 or more and carry full 1500-byte packets inside. Only do this if you can prove the whole path carries it.

What goes wrong: fragmentation and PMTUD black holes

There are two failure paths when a 1500-byte packet reaches a tunnel with a 1476 MTU.

1. The packet has DF (Don't Fragment) set. Almost all modern TCP traffic sets DF. The encapsulating router must drop it and send ICMP type 3 code 4 ("fragmentation needed") back to the sender. The sender lowers its segment size and retries. This is Path MTU Discovery (PMTUD). If any firewall or ACL along the way drops that ICMP, the sender never learns, and the large packets vanish silently. That is the black hole. For IPv6 the equivalent is ICMPv6 type 2 "Packet Too Big", and it is not optional, since IPv6 routers never fragment.

2. The packet does not have DF set. The router either fragments the inner packet before encapsulation or encapsulates and then fragments the outer packet. Both work, but they are expensive. Fragments are handled in the slow path on many platforms, reassembly burns CPU, and many ACLs and DDoS filters drop non-initial fragments. During an attack this gets worse, because your router is also handling junk.

Typical symptoms

• The TCP handshake completes, but a TLS connection hangs after the ClientHello or at the server certificate.
• Pings work, ping -s 1400 fails.
• SSH logins work, but ls on a big directory freezes.
• HTTP loads small pages and stalls on large ones.
• Problems appear only for some client networks (those with ICMP filtering or smaller MTUs).

Where to clamp MSS

MSS clamping rewrites the MSS option in TCP SYN and SYN-ACK packets so both ends never send segments larger than the tunnel can carry. It avoids PMTUD entirely for TCP. The direction matters, and it is the part most guides skip.

The MSS a host advertises tells the peer how large a segment it may send. In a DDoS tunnel, internet-to-customer traffic arrives over GRE. The segments from a remote client to your server are limited by the MSS in your server's SYN-ACK. Two cases:

• Symmetric return path (replies also go through the tunnel): clamp SYN and SYN-ACK on the tunnel interface in both directions.
• Asymmetric return path (replies leave via your normal transit): the SYN-ACK never crosses the tunnel interface. Clamp SYN-ACK packets from your prefixes on the egress toward transit, or set the MSS on the servers themselves.

Clamping only lowers the MSS and never raises it, so it is safe to apply broadly. It does nothing for UDP, which we cover below.

Configuration examples

All examples use documentation addresses: your router is 192.0.2.1, the remote tunnel endpoint is 198.51.100.1, and the inner link is 203.0.113.0/30.

Linux

ip link add gre-ddos type gre local 192.0.2.1 remote 198.51.100.1 ttl 255
ip addr add 203.0.113.2/30 dev gre-ddos
ip link set gre-ddos mtu 1476 up

# Clamp to the interface PMTU for traffic crossing the tunnel
iptables -t mangle -A FORWARD -o gre-ddos -p tcp --tcp-flags SYN,RST SYN -j TCPMSS --clamp-mss-to-pmtu
iptables -t mangle -A FORWARD -i gre-ddos -p tcp --tcp-flags SYN,RST SYN -j TCPMSS --clamp-mss-to-pmtu

# Asymmetric return: clamp SYN-ACKs sourced from your prefix leaving toward transit
iptables -t mangle -A FORWARD -s 192.0.2.0/24 -p tcp --tcp-flags SYN,RST SYN -j TCPMSS --set-mss 1436

The nftables equivalent: tcp flags syn tcp option maxseg size set rt mtu.

MikroTik RouterOS v7

/interface gre add name=gre-ddos local-address=192.0.2.1 remote-address=198.51.100.1 mtu=1476 clamp-tcp-mss=yes
/ip address add address=203.0.113.2/30 interface=gre-ddos
/ip firewall mangle add chain=forward protocol=tcp tcp-flags=syn out-interface=gre-ddos \
    action=change-mss new-mss=1436 tcp-mss=1437-65535 passthrough=yes
/ip firewall mangle add chain=forward protocol=tcp tcp-flags=syn in-interface=gre-ddos \
    action=change-mss new-mss=1436 tcp-mss=1437-65535 passthrough=yes

Note that mangle rules apply to forwarded traffic, so on a fast-path or hardware-offloaded setup, verify they actually match.

Cisco IOS / IOS-XE

interface Tunnel0
 ip address 203.0.113.2 255.255.255.252
 ip mtu 1476
 ip tcp adjust-mss 1436
 tunnel source 192.0.2.1
 tunnel destination 198.51.100.1

ip tcp adjust-mss acts on SYNs in both directions through the interface. For asymmetric return, also apply it on the transit-facing interface.

Juniper

set interfaces gr-0/0/0 unit 0 tunnel source 192.0.2.1
set interfaces gr-0/0/0 unit 0 tunnel destination 198.51.100.1
set interfaces gr-0/0/0 unit 0 family inet mtu 1476
set interfaces gr-0/0/0 unit 0 family inet address 203.0.113.2/30

MSS clamping on Junos is platform-dependent. On SRX use set security flow tcp-mss gre-in mss 1436 and gre-out mss 1436. On MX, check whether your line card and services configuration support tcp-mss under family inet, or clamp on the servers or an upstream firewall. Also note that the gr- interface requires a tunnel-capable PIC or inline services.

Verify, don't assume

Test with DF-bit pings of exact sizes. A 1476-byte packet is -s 1448 (1448 + 28 bytes ICMP/IP headers).

# Linux: should pass
ping -M do -s 1448 203.0.113.1
# should fail with "message too long" or a frag-needed error
ping -M do -s 1449 203.0.113.1

On MikroTik: /ping 203.0.113.1 size=1476 do-not-fragment. On Cisco: ping 203.0.113.1 size 1476 df-bit. Windows: ping -f -l 1448 target.

Then confirm the clamp actually works by capturing SYNs:

tcpdump -ni gre-ddos 'tcp[tcpflags] & tcp-syn != 0' -v | grep -o 'mss [0-9]*'

You should see mss 1436 or lower in packets that crossed the tunnel. Finally, test a real TLS connection through the tunnel from an outside network, such as curl -v https://yoursite.example from a host that has no relationship with your infrastructure.

Don't forget UDP and other non-TCP traffic

MSS clamping can't help UDP. DNS, QUIC, VPNs, and game traffic all send datagrams that may exceed the tunnel MTU.

• QUIC clients generally keep datagrams small, but they still depend on a working path.
• For DNS, set the EDNS buffer size to around 1232 bytes to avoid fragmented responses entirely.
• For UDP VPNs (WireGuard, OpenVPN), lower the interface MTU or tun-mtu/mssfix accordingly.
• Allow fragments of legitimate traffic in your ACLs, and filter fragments selectively rather than blanket-dropping them if you carry such services.

Common mistakes

1. Setting the MSS but not the tunnel MTU (or the other way around). Do both. MSS handles TCP, and the tunnel MTU gives PMTUD something correct to report.
2. Filtering ICMP. Permit ICMP type 3 code 4 and ICMPv6 type 2 and 1/3/4 as needed. Rate-limit them if you must, but don't drop them.
3. Using IPv4 MSS values for IPv6. The correct IPv6 value is 20 bytes lower (1416 on a 1476 tunnel).
4. Clamping on one side only, or ignoring the asymmetric return path.
5. Forgetting underlay overhead. PPPoE, extra VLAN tags, or an IX/colo link with a smaller MTU will shift every number.
6. Hard-coding 1476 on both ends without agreeing on GRE options. A GRE key or checksum on one side changes the overhead.
7. Not retesting after changes such as a new transit, a new tunnel endpoint, or router software upgrades that reset interface MTU.

DDoS-specific considerations

During an attack, sloppy MTU handling hurts more. Fragmented traffic bypasses 5-tuple-based filters, fragment floods are a common attack vector in their own right, and CPU spent reassembling fragments isn't available for forwarding. Keeping the inner packets small enough to avoid fragmentation completely removes a whole class of problems.

If you use a cloud scrubbing service with a GRE tunnel, ask for the exact tunnel parameters (endpoint addresses, GRE options, expected MTU) and test with DF pings before moving traffic. With NixGuard Edge, clean traffic returns over a GRE tunnel with a BGP session inside, so the same checks apply: confirm the MTU and MSS values for your endpoints during onboarding, and test before you announce prefixes in production. If you also want flow-based attack detection and automatic triggering of mitigation, you can start a 15-day free trial with no credit card.