DDoS Protection Architecture for Small ISPs: A Layered Design

A solid DDoS protection architecture for a small ISP has four layers: flow-based detection, RTBH as a last-resort safety valve, FlowSpec for surgical filtering, and cloud scrubbing over a BGP tunnel for attacks bigger than your uplinks. Most small networks should start with detection plus RTBH, add FlowSpec where their routers and upstreams support it, and use on-demand scrubbing for the attacks they cannot absorb.

This article walks through each layer with example configs, explains when to escalate from one to the next, and compares on-demand and always-on scrubbing.

The layered model

Think of it as escalation. Each layer handles what the previous one cannot, and all of them depend on fast, accurate detection.

Layer 1: Flow-based detection

You cannot mitigate what you cannot see. Export NetFlow v9, IPFIX, or sFlow from your edge and border routers to a collector that evaluates per-IP and per-prefix bps and pps against thresholds. Export from the ingress direction on transit and peering interfaces.

Juniper MX (inline IPFIX, sampling 1:1000):

set chassis fpc 0 sampling-instance SAMPLE-1
set services flow-monitoring version-ipfix template T4 ipv4-template
set services flow-monitoring version-ipfix template T4 flow-active-timeout 10
set services flow-monitoring version-ipfix template T4 flow-inactive-timeout 10
set forwarding-options sampling instance SAMPLE-1 input rate 1000
set forwarding-options sampling instance SAMPLE-1 family inet output flow-server 192.0.2.10 port 2055
set forwarding-options sampling instance SAMPLE-1 family inet output flow-server 192.0.2.10 version-ipfix template T4
set forwarding-options sampling instance SAMPLE-1 family inet output inline-jflow source-address 192.0.2.1
set interfaces xe-0/0/0 unit 0 family inet sampling input

MikroTik RouterOS:

/ip traffic-flow set enabled=yes interfaces=ether1 active-flow-timeout=30s inactive-flow-timeout=15s
/ip traffic-flow target add dst-address=192.0.2.10 port=2055 version=9

Cisco IOS (traditional NetFlow; newer IOS XE uses Flexible NetFlow with the same idea):

ip flow-export version 9
ip flow-export destination 192.0.2.10 2055
interface GigabitEthernet0/0/0
 ip flow ingress

Linux router or mirror port with softflowd:

softflowd -i eth0 -v 9 -n 192.0.2.10:2055

Tips that save you pain later:

• Use short active timeouts (10–30 seconds). Long timeouts delay detection.
• Set thresholds from your own baselines, not from copied numbers. A single customer /32 and a whole /24 need different limits.
• Alert on pps as well as bps. Small-packet floods can crush a router CPU or line card while bandwidth looks modest.
• Also monitor per-prefix totals, otherwise carpet-bombing attacks spread across a /24 stay under per-IP limits.

Layer 2: RTBH as the safety valve

Remotely triggered blackhole routing drops all traffic to a victim /32. It is blunt, but it protects your transit links and every other customer. The detector announces the /32 over iBGP with a blackhole community (RFC 7999 defines 65535:666), and your edge routers translate that into a discard next-hop.

Cisco IOS example on the edge router:

ip route 192.0.2.254 255.255.255.255 Null0
ip community-list standard BLACKHOLE permit 65535:666
route-map RTBH-IN permit 10
 match community BLACKHOLE
 set ip next-hop 192.0.2.254
route-map RTBH-IN permit 20

Juniper equivalent:

set policy-options community BLACKHOLE members 65535:666
set policy-options policy-statement RTBH-IN term bh from community BLACKHOLE
set policy-options policy-statement RTBH-IN term bh from route-filter 0.0.0.0/0 prefix-length-range /32-/32
set policy-options policy-statement RTBH-IN term bh then next-hop discard
set policy-options policy-statement RTBH-IN term bh then accept

On a MikroTik router without automation, a manual fallback is /ip route add dst-address=198.51.100.10/32 blackhole.

To stop the attack before it fills your uplink, also send the /32 to your upstream with their blackhole community. Check each provider's documentation, because communities differ and some limit it to /32 only. For more detail, see our guide on automating RTBH with blackhole communities.

Guardrails: accept /32s only from your detector session, only inside your own prefixes, set a max-prefix limit, and auto-withdraw after a timer.

Layer 3: FlowSpec for surgical filtering

RTBH takes the victim offline. FlowSpec (RFC 8955) lets you drop or rate-limit only the attack traffic, such as UDP source port 123 to one destination, while legitimate traffic keeps flowing.

Example rule in ExaBGP-style syntax:

flow route {
  match {
    destination 198.51.100.10/32;
    protocol udp;
    source-port =123;
  }
  then { discard; }
}

Juniper import side, restricted to your own space:

set protocols bgp group FS neighbor 192.0.2.10 family inet flow no-validate ACCEPT-FS
set policy-options policy-statement ACCEPT-FS term mine from route-filter 198.51.100.0/24 orlonger
set policy-options policy-statement ACCEPT-FS term mine then accept
set policy-options policy-statement ACCEPT-FS then reject

Things to check before relying on FlowSpec:

• Does your router model support it, and how many rules fit in hardware?
• Does your upstream accept FlowSpec from you? Many do not, and local filtering will not help if the uplink is already saturated.
• Does the detector only generate rules for destinations you own, and expire them automatically?

Layer 4: Cloud scrubbing over a BGP tunnel

When an attack exceeds your uplink capacity, local filtering is too late because the pipe is already full. The answer is to pull traffic to a scrubbing network. You announce your prefixes from the scrubbing provider, they filter the traffic, and clean traffic returns to you over a GRE tunnel with a BGP session inside (a "DDoS tunnel" or "BGP tunnel").

Requirements are usually strict. You need your own ASN and prefixes the provider can announce (NixGuard Edge requires an ASN and at least one /24). If you have neither, you can't use this model, and RTBH plus upstream filtering is what remains.

Operational points:

• MTU: GRE adds 24 bytes with the key-less IPv4 encapsulation. Set the tunnel MTU to 1476 and clamp TCP MSS, or you will see mysterious stalls on large transfers.
• Prefix filtering: the provider should filter your announcements using your AS-SET or IRR objects. Keep those objects current.
• Return path: outbound traffic usually leaves through your normal transit, so the return path is asymmetric. Stateful services such as SYN-protected ones need a design that accounts for that.
• Routing policy: keep the tunnel session's local-preference and communities documented so that a withdrawal returns traffic to normal paths cleanly.

We cover the tunnel mechanics in more depth in our earlier posts on GRE clean-traffic delivery and the GRE + BGP config guide.

On-demand vs always-on

A practical rule: if you see large attacks rarely and your detection is good, start on-demand. If specific prefixes are attacked regularly, or even short outages cost you customers, put those prefixes in always-on. Many networks mix both by prefix.

A practical escalation policy

1. Detector sees a threshold breach on 198.51.100.10/32.
2. If the attack matches a known signature (UDP reflection, for example) and your uplink is fine, send a FlowSpec rule.
3. If it is not filterable and your links are at risk, send RTBH, locally and to the upstream if needed.
4. If volume exceeds your uplink capacity or recurs, announce the prefix to the scrubbing edge and receive clean traffic over the tunnel.
5. Withdraw everything after a hold-down timer, and review the incident.

Common mistakes

• Blackholing a whole customer /24 instead of a /32 because the attack looked like carpet bombing.
• No auto-withdrawal, leaving blackholes in place for days.
• Exporting flows on egress interfaces or from only one of several borders, which hides attacks.
• Using sampling rates so high that small floods are invisible, or so low that the collector drowns.
• Leaving the GRE MTU at the default.
• Never testing. Run a drill with a documentation prefix or a lab customer before you need it.
• Out-of-date AS-SET or IRR objects, which cause scrubbing announcements to be filtered.

Where NixMon fits

NixMon detects attacks from NetFlow v5/v9, IPFIX, and sFlow with sub-second analytics, per-IP and per-prefix bps/pps thresholds, and can trigger BGP blackhole, BGP FlowSpec, the on-prem NixGuard firewall, or NixGuard Edge. NixGuard Edge announces your prefixes from its scrubbing edge and delivers clean traffic over a GRE tunnel with BGP inside, either always-on or on-demand. You can test the detection and mitigation workflow with the 15-day free trial, no credit card required.

Whatever tooling you choose, evaluate it by running a real drill: export flows, trigger a test threshold, and watch the full path from alert to announcement to withdrawal.