Keep a VPS online under DDoS

“DDoS” describes an intent, not a mechanism, and the mechanisms have almost nothing in common. Sorting them properly is not pedantry: it is the whole of the work, because each one is stoppable at exactly one layer and invisible at the others.

Volumetric floods aim at your bandwidth. UDP amplification — DNS, NTP, memcached, and lately anything that answers a small question with a large answer — lets an attacker turn 1 Gbps of their capacity into 50 Gbps aimed at you. The target is the link, not the server. Your CPU will be bored throughout.

Protocol and state attacks aim at a finite table in your kernel. A SYN flood tries to exhaust the accept queue; a generic small-packet flood tries to exhaust connection tracking. Both are measured in packets per second, not bits per second, and both can kill a machine on a link that is 97% idle. This is the class that takes down small servers, and the class most guides skip.

Application-layer floods aim at your CPU or your database, using requests that are indistinguishable from real ones because they are real ones. A hundred requests a second to a search endpoint is nothing to a network and fatal to a PHP application. No upstream scrubber can filter this for you: from the outside it looks exactly like success.

There is a fourth thing that arrives dressed as all three and is not an attack at all: a link that did well somewhere, a misbehaving client of your own, or a crawler with no manners. Ruling that out first is free, and it is embarrassing how often it is the answer.

If 60 Gbps is aimed at your address and your port is a 1 Gbps port, the decision about those packets is made in a router upstream of you, several hops before anything you administer. Your firewall never sees them. It cannot: they were discarded to protect a link you do not own. This is the single most important structural fact about volumetric attacks, and the reason “harden your firewall against DDoS” is mostly nonsense.

So the only meaningful questions are what your provider does automatically and where their thresholds sit. Ours are published rather than promised: under 10 Gbps is absorbed with no visible effect, 10–100 Gbps is scrubbed at the transit provider and you may notice a latency bump, and above 100 Gbps the prefix is announced onto a dedicated scrubbing path — latency rises more noticeably, and the service stays reachable. Null-routing an address is on the table only for sustained attacks that threaten the wider PoP, and we tell you within minutes if it happens. There is nothing to buy and nothing to enable.

Two things are worth knowing about scrubbing that vendors rarely volunteer. The first is that you are protected by the neighbourhood, not just by your own defences: an attack on a customer sharing your upstream /20 is absorbed before it reaches anyone’s prefix — a ~340 Gbps flood aimed at a neighbouring range in Paris passed with no measurable impact on ours, which is exactly the outcome nobody notices. The second is that scrubbers are heuristics and heuristics are wrong sometimes: on the same network, a scrubber once spent nine minutes issuing TCP resets against legitimate connections while filtering an attack on a neighbour. Both incidents are in the public incident log. If your sessions die in a way that looks like an active reset rather than a timeout during someone else’s attack, that is a real failure mode and worth reporting rather than debugging locally for an hour.

Every wrong DDoS response starts with a change made before anyone knew what was happening. Get four numbers first. They fit on one screen and they name the layer for you.

# 1. bits vs packets  — which axis is saturated?
sar -n DEV 1 5            # or: ifstat -i eth0 1

# 2. socket states     — SYN-RECV piling up means a SYN flood
ss -s
ss -tan state syn-recv | wc -l

# 3. kernel drop counters (these are the ones that matter)
nstat -az | grep -E 'ListenDrops|ListenOverflows|SyncookiesSent|TCPReqQFullDrop'
dmesg -T | tail -20       # look for: nf_conntrack: table full, dropping packet

# 4. connection tracking headroom
cat /proc/sys/net/netfilter/nf_conntrack_count
cat /proc/sys/net/netfilter/nf_conntrack_max

Read them together. High bits per second, low packets per second, low CPU is a volumetric attack, and your job is to confirm it and stop typing. High packets per second, low bits per second — lots of tiny packets — is a state attack; look immediately at SyncookiesSent and the conntrack count. Modest traffic on both axes with the CPU pinned is layer 7, and the answer is in your web server and your database, not your firewall.

One counter-intuitive reading is worth internalising: during a genuine volumetric flood, your interface may look almost calm. You are seeing the survivors — what fits through a link that is already full, or what the upstream scrubber has passed. An interface that shows 900 Mbps on a 1 Gbps port while users report total unreachability is not evidence against an attack. It is the shape of one.

Then check the traffic is not simply real. tail -f your access log for ten seconds: one path repeated by thousands of distinct addresses with no referrer is an attack; a spread of normal paths from normal browsers is an audience, and rate-limiting it will do the attacker’s job for them.

Every open port is a queue an attacker can fill, and every packet your guest has to look at costs you CPU and a conntrack entry even when you drop it. The cheapest packet is the one your machine never receives, which is what makes filtering above the guest worth more than filtering inside it.

There are two layers here and they are not the same thing. The edge filter is optional, configured per server, stateful and drop-by-default, and it runs on the hypervisor — traffic you reject there never reaches your virtual NIC at all, so it costs you no CPU, no memory and no state-table entry. The guest firewall — your nftables ruleset — is entirely yours and we never touch it. The pattern that survives contact with an attack is to express the coarse, stable layer-4 truth at the edge (“this machine serves 80, 443 and SSH from these addresses, and nothing else exists”) and keep the guest ruleset for the fine-grained work that changes with your application. Both are described in the firewall documentation.

Then narrow what remains. SSH restricted to the addresses you actually administer from is not a hardening nicety here — it removes an entire class of connection-exhaustion attack from your machine. A database bound to 127.0.0.1 or a WireGuard address cannot be flooded from the internet at all. And if you serve a purely internal control plane, put it behind a WireGuard interface rather than a public port with a password on it.

A SYN flood exploits the handshake: the attacker sends a stream of connection requests and never completes them, and each one occupies a slot in the kernel’s SYN queue until it times out. Fill the queue and legitimate handshakes are dropped — your service is up, listening, and unreachable.

Linux has had a clean answer since the nineties, and it is on by default. SYN cookies let the kernel stop allocating memory for half-open connections entirely: it encodes the connection state into the sequence number it sends back, and reconstructs it if the client completes the handshake. Confirm it rather than assume it, and give the queues enough room to ride out a burst:

# /etc/sysctl.d/99-flood.conf
net.ipv4.tcp_syncookies = 1
net.ipv4.tcp_max_syn_backlog = 8192
net.core.somaxconn = 8192
net.ipv4.tcp_synack_retries = 2
net.ipv4.tcp_abort_on_overflow = 0

sysctl --system

tcp_synack_retries = 2 matters more than it looks: the default of five means a forged handshake occupies the kernel for roughly three minutes, and two brings that down to about seven seconds. Raising somaxconn is only half the job — the queue depth is the minimum of that value and what the application asked for, so nginx needs listen 443 ssl backlog=8192; and a reload before the kernel setting means anything. This is the classic case of a sysctl that appears to have been applied and does nothing.

If you want to stop the flood before it reaches the application at all, nftables can complete handshakes on the kernel’s behalf and only hand over connections that turn out to be real:

table inet filter {
  chain input {
    type filter hook input priority filter; policy drop;
    ct state established,related accept
    iif lo accept
    tcp dport { 80, 443 } ct state new synproxy mss 1460 wscale 7 timestamp sack-perm
    tcp dport { 80, 443 } accept
    ip protocol icmp icmp type echo-request limit rate 5/second accept
  }
}

Test synproxy on a machine you can still reach by console before you rely on it; misconfigured, it is an excellent way to firewall yourself out of your own server. For most people the sysctl block above is enough, and the honest ranking is: syncookies first, backlog second, synproxy only if you have measured that the first two were not sufficient.

This is the one that catches experienced people. A stateful firewall has to remember every flow it has seen, and that memory is nf_conntrack, a hash table with a fixed size chosen at boot. Once it is full, the kernel drops new connections — all of them, attacker and customer alike — and logs a single line to dmesg that nobody is watching:

nf_conntrack: table full, dropping packet

The reason it is so effective is arithmetic. A default table on a small VPS holds somewhere in the low tens of thousands of entries, and every packet from a new source address creates one — including UDP packets, including packets you drop, including the flood. Twenty thousand packets per second from spoofed sources fills it in under two seconds, on a link carrying a few megabits. Your bandwidth graph will show nothing at all.

# /etc/sysctl.d/99-conntrack.conf
net.netfilter.nf_conntrack_max = 262144
net.netfilter.nf_conntrack_buckets = 65536
net.netfilter.nf_conntrack_tcp_timeout_established = 3600
net.netfilter.nf_conntrack_tcp_timeout_syn_recv = 20
net.netfilter.nf_conntrack_udp_timeout = 20

The timeouts do more good than the size. The default lifetime for an established TCP flow is five days, which means a server that has been up a week is holding state for connections that ended on Tuesday; an hour is plenty for anything that is not an idle SSH session, and net.ipv4.tcp_keepalive_time = 600 keeps those alive properly rather than by accident. Budget roughly 300 bytes of kernel memory per entry when sizing: 262,144 entries is about 80 MB, which is fine on a 4 GB machine and not fine if you set it to ten million because a forum said so.

If you run something genuinely stateless and high-volume — an authoritative DNS server, a public game server, a Tor relay — the better answer is to stop tracking it at all. Connection tracking for a service that has no meaningful connections is pure cost:

table ip raw {
  chain prerouting {
    type filter hook prerouting priority raw; policy accept;
    udp dport 51820 notrack
    tcp dport 9001 notrack
  }
}

Remember that such traffic then bypasses your ct state established rules, so it needs explicit accept rules in the filter chain. That is the trade: you give up statefulness for that port and you get an unfillable table.

The most efficient attack on a small server is not a flood at all. It is a few hundred well-chosen HTTP requests per second, each one perfectly valid, each one hitting the single endpoint that runs an unindexed query or renders an uncached page. The economics are brutal: the request costs the attacker a few hundred bytes and costs you 200 milliseconds of CPU and a database connection. You lose that race at any volume you can afford to serve.

The instinct is to block the attacker. Against a botnet spread across thousands of residential addresses, each sending two requests a second, blocking by IP is theatre — no per-address rate limit that permits your real users will ever trigger, and you will spend the outage adding rules while the site stays down. The winning move is to change the cost, not the count.

Cache first, and cache the miss. A full-page cache turns an expensive request into a memory read, and the directive that matters most in an attack is the one that stops a thousand simultaneous misses from becoming a thousand simultaneous database queries:

proxy_cache_path /var/cache/nginx levels=1:2 keys_zone=app:64m max_size=2g inactive=60m;

server {
  location / {
    proxy_cache app;
    proxy_cache_valid 200 301 302 10m;
    proxy_cache_lock on;              # one origin request per key, not a thousand
    proxy_cache_lock_timeout 5s;
    proxy_cache_use_stale error timeout updating http_500 http_502 http_503 http_504;
    proxy_cache_background_update on; # serve stale, refresh behind it
    add_header X-Cache $upstream_cache_status always;
    proxy_pass http://127.0.0.1:8080;
  }
}

proxy_cache_lock and use_stale are the two lines that decide whether a traffic spike is survivable. Without them, the moment a hot key expires every in-flight request becomes an origin request — the stampede that turns a manageable attack into an outage. With them, your backend serves one request per key per interval and everyone else gets a slightly stale page, which is the correct trade in every situation where the alternative is no page at all.

Then rate-limit the expensive paths specifically, generously, and in a way that fails visibly:

limit_req_zone $binary_remote_addr zone=general:16m rate=20r/s;
limit_req_zone $binary_remote_addr zone=costly:16m  rate=2r/s;
limit_conn_zone $binary_remote_addr zone=conn:16m;
limit_req_status 429;
limit_conn_status 429;

server {
  limit_req  zone=general burst=40 nodelay;
  limit_conn conn 24;

  location /search { limit_req zone=costly burst=5 nodelay; proxy_pass http://127.0.0.1:8080; }
  location /login  { limit_req zone=costly burst=3 nodelay; proxy_pass http://127.0.0.1:8080; }
}

burst without nodelay queues excess requests instead of rejecting them, which under attack means nginx politely holding thousands of connections open on your behalf — you have converted a request flood into a connection flood. Use nodelay, return 429 immediately, and let the client deal with it. And behind any proxy, remember that $binary_remote_addr is the proxy unless you have configured set_real_ip_from and real_ip_header correctly; a rate limit keyed on your own front-end address will lock out every user at once the first time it fires.

Slowloris-style attacks — many connections, each dribbling a request one byte at a time — barely trouble nginx’s event model, but the timeouts are worth tightening anyway: client_header_timeout 8s; client_body_timeout 8s; send_timeout 10s; reset_timedout_connection on;. Combined with limit_conn, that is the whole defence.

fail2ban is genuinely useful and it is aimed at a different problem. It reads log files, decides after N failures in M minutes, and inserts a firewall rule. Every part of that is the wrong shape for a flood.

It is too slow: a burst that lasts forty seconds is over before the ban window closes. It reads logs, so the attack has already cost you the full price of every request — the parsing happens after the damage. It bans individual addresses, so against ten thousand sources it either does nothing or inserts ten thousand linear rules, at which point the firewall itself becomes the bottleneck and you have completed the attack on your attacker’s behalf. And it is driven by your logs, so a flood large enough to fill a disk with log lines can take the server down through a path you never considered.

If you want dynamic blocking, do it in the data plane where the lookup is a hash and the expiry is automatic. An nftables dynamic set is O(1) regardless of size, and forgets on its own:

table inet filter {
  set flooders {
    type ipv4_addr
    flags dynamic, timeout
    timeout 10m
    size 65535
  }

  chain input {
    type filter hook input priority filter; policy drop;
    ct state established,related accept
    iif lo accept
    ip saddr @flooders drop
    tcp dport { 80, 443 } ct state new add @flooders { ip saddr limit rate over 50/second burst 100 packets } drop
    tcp dport { 80, 443 } accept
  }
}

Read the rule carefully, because the negation trips people: the address is added to the set and dropped only when the rate is over the limit. Keep fail2ban for what it is good at — slow credential guessing against SSH and application logins, where a human-timescale ban on a specific address is exactly right.

Every large volumetric attack is powered by servers whose operators did not know they were participating. The mechanism is a UDP service that answers a small forged request with a large reply: an open DNS resolver returns 50x what it was asked, misconfigured NTP and memcached are worse, and any protocol that responds before it can verify who is asking is a candidate.

The consequences land on you before they land on the victim. Your uplink fills with your own outbound replies, your provider sees sustained abusive traffic leaving your port, and the transit suppliers who receive it start null-routing the ranges it comes from — which is why the acceptable-use policy draws a hard line at attack origination and amplification while permitting essentially everything else. Being an unwitting reflector is the fastest way to lose an address that has nothing to do with anything you meant to run.

# bind a recursive resolver to localhost, never 0.0.0.0
# unbound: interface: 127.0.0.1  +  access-control: 0.0.0.0/0 refuse

# what is actually listening on a public address?
ss -ulpn
ss -tlpn

# from another machine, confirm you do not answer strangers
dig @your.server.ip example.com +short          # must time out
ntpq -c rv your.server.ip                        # must fail

If you run a public UDP service on purpose — a game server, an authoritative nameserver, a WireGuard endpoint — the rule is the same as everywhere else: rate-limit the response, never the request. bind and knot both implement response rate limiting; use it, because it is the difference between serving your users and shipping someone else’s attack.

Everything above is damage control for an attack that has already found your address. It is worth doing, and it is second best. An origin nobody can name is not attackable at layer 3 or 4 at all, and separating the address that serves traffic from the address the world knows is the highest-leverage thing on this page.

Origin addresses almost never leak through clever attacks. They leak through history and through carelessness, in a short and well-known list. Old DNS records are the usual culprit: passive-DNS archives remember the A record you had before you put a front-end in place, forever. Certificate transparency logs are public and permanent, so a certificate issued for a hostname that pointed straight at the origin is a signed, timestamped record of where you used to live. Outbound mail stamps the sending server’s address into the headers of every message. And the quiet one: your web server answering on its bare IP, which lets anyone who scans the address space for your site’s HTML find you by content in an afternoon.

That last one is a two-line fix and almost nobody does it. Make the default server refuse everything that did not arrive with a hostname you serve:

server {
  listen 80 default_server;
  listen 443 ssl default_server;
  ssl_reject_handshake on;      # nginx 1.19.4+: no certificate, no fingerprint
  return 444;                   # close without a response
}

Then choose a front. A commercial CDN is the obvious answer and comes with a real cost that matters here: you are adding a company that terminates your TLS, sees your plaintext, knows your origin, and can be served with legal process in a jurisdiction you did not choose — which undoes a good part of why the server was paid for in crypto without a name attached. If you do it anyway, understand that the origin must never have been public, and that the whole arrangement fails the moment someone finds an old record.

The alternative that keeps the property you paid for is to be your own front. A $5 instance in a second region running nginx as a reverse proxy, with the origin’s edge filter accepting traffic from that one address and nothing else, gives you a sacrificial address you can renumber in five minutes and a control plane that answers only to you. It is the same architecture, minus the third party. And where the audience can use it, a Tor onion service removes the IP from the equation entirely — there is no address to flood, though onion services have their own attack surface at the introduction points, which is why modern Tor ships a proof-of-work defence for exactly this.

Sooner or later someone suggests blocking a country. It is blunt, occasionally correct, and mostly a way to feel busy.

It is defensible when your service has a genuinely bounded audience — a regional game server, an internal tool, a control panel — and you are prepared to own the false positives, which include your own users travelling and anyone routed oddly. It is close to useless against a modern botnet, which is distributed across residential connections in every country including yours, and it is actively harmful for anything public: you will silently lose real people and never see the ones who left.

Blocking by ASN is sharper. Attack traffic that originates from a handful of hosting providers — cheap VPS ranges rented by the hour — can be dropped by prefix with far fewer casualties than a country block, because ordinary users do not browse from a datacenter. Datacenter ranges are also, conveniently, where scraping and credential stuffing come from.

But the durable version of this idea is not geography at all: it is filtering on the shape of the traffic rather than its origin. An attack usually shares something structural — the same user agent, the same missing header, the same URL with the same query parameter, the same TLS fingerprint, a complete absence of the second request a real browser always makes. Find the shared property and you write one rule that costs nothing and survives the attacker changing addresses, which they will do faster than you can list them.

# in an attack: what do these requests have in common?
awk '{print $1}'  /var/log/nginx/access.log | sort | uniq -c | sort -rn | head -20   # addresses
awk -F'"' '{print $6}' /var/log/nginx/access.log | sort | uniq -c | sort -rn | head -20   # user agents
awk '{print $7}'  /var/log/nginx/access.log | sort | uniq -c | sort -rn | head -20   # paths

If one user agent accounts for 90% of requests, you are ninety seconds from a fix. If the top twenty addresses account for 2% of traffic each, stop looking for addresses — that is a distributed attack and the answer is caching and rate limits.

Sometimes the attack is targeted, persistent and aimed at you personally rather than at a random address, and the correct response is to stop defending the address and abandon it. This is not defeat; for a small service it is often the cheapest possible outcome, and it is only painful if you have not rehearsed it.

What makes it fast is decided in advance. Keep DNS TTLs at 300 seconds as a standing posture — the cost is negligible and the benefit is that you can move in five minutes rather than a day. Keep your deployment reproducible, because a server you can rebuild is a server you can move; if rebuilding means remembering what you did in March, you do not have a plan, you have a hostage. Keep a restorable backup in a different region, and know from a real drill how long a restore takes.

Then the actual move is short: deploy in another region, restore, verify on the new address by hostname override before you touch DNS, flip the record, and keep the old server running long enough for the last resolvers to catch up. The migration guide covers the sequencing properly, and it is the same sequence whether you are moving for performance or moving because someone is angry.

The billing model makes the rehearsal essentially free: charges are prorated daily against your balance, so a second machine you deploy, test and destroy inside an afternoon costs cents, and there is no contract, no card and no renewal to cancel. There is no reason for your first renumbering to happen under attack.

Two honest closing notes, because the rest of this page is optimistic by construction.

The first is that no host absorbs everything, and any provider claiming otherwise is selling. Capacity is finite, scrubbing capacity is more finite, and at a large enough scale the economically rational act for a network protecting thousands of customers is to stop announcing one address. We treat that as a last resort against sustained attacks that threaten the wider PoP, and we tell you rather than let you debug it — but a promise that it can never happen would be a lie, and you should discount any host that makes one.

The second is that retaliation is not on the menu. Booter and stresser services are attacks-for-hire dressed as testing tools, they are illegal in most of the jurisdictions that matter, they are extensively monitored, and they will get your server terminated here under a policy that permits nearly everything else. The asymmetry that makes DDoS attractive to an attacker makes it worthless to a defender: you cannot out-flood someone who has nothing to lose. Absorb, cache, move, and let it get boring — which, for anyone paying for the traffic they send, it eventually does.