Proxies, Load Balancers and Deployment

This content was generated with the assistance of AI and should be verified against the official documentation before being relied on in production.

See the section bibliography for the reference material consulted while preparing these pages.

A WebSocket server that works on a laptop usually breaks the first time it is put behind a proxy: the handshake is answered with a plain HTTP page, the socket dies after exactly 60 seconds, every client appears to come from the same address, or a deploy cuts every call. None of this is a bug in the application. A WebSocket is an HTTP request that turns into a long-lived tunnel, and every hop in front of the server (reverse proxy, cloud load balancer, CDN, ingress controller) has its own opinion about upgrades, idle time, buffering and shutdown. This page lists those opinions for the infrastructure most often used, shows configurations that were run, and ends with the part that is different for media: TURN servers and SFUs speak UDP, and their addresses and ports end up inside ICE candidates, so they cannot sit behind an ordinary HTTP load balancer.

The three applications of this section are used as the running examples: Chatterbox (a chat on port 8300, see Building a Chat App), Huddle (a videoconference with a signalling server, an SFU and coturn, see Building a Videoconference App) and Pixel Arena (an authoritative game server, see Building a Multiplayer Game). Horizontal scaling and backplanes are on Scaling WebSockets, and media-server capacity on Scaling WebRTC and TURN.

Versions were checked on 2026-10-24: nginx 1.30 (stable branch, 1.30.6; the mainline is 1.31), HAProxy 3.4.7, Caddy 2.11.7, ws 8.22.0 on Node.js 24, coturn 4.18.0, Kubernetes 1.37 with Gateway API 1.6 (see the site’s Gateway API page) and the AWS Elastic Load Balancing documentation of that date. The nginx, HAProxy and Caddy configurations, the PROXY protocol chain, the TLS block, the sticky-routing variants and the coturn configuration were run in Docker with the images above in front of a small WebSocket server; the Kubernetes manifests were validated against the Kubernetes 1.34 and Gateway API JSON schemas with kubeconform (no cluster was involved) and the Compose files with docker compose config. The AWS and Cloudflare settings come from their documentation and were not run.

What a proxy has to do for a WebSocket

A WebSocket starts as an ordinary HTTP/1.1 request with Connection: Upgrade and Upgrade: websocket; the server answers 101 Switching Protocols and from then on the connection carries frames in both directions (see The WebSocket Protocol). Connection and Upgrade are hop-by-hop headers, so a proxy that does not know about WebSockets drops them, forwards a plain request and the application answers it as normal HTTP. The browser sees a 200 instead of a 101 and reports a failed connection.

Six things matter at every hop:

  1. Forward the upgrade. The proxy has to recognise the request, speak HTTP/1.1 to the upstream and pass the two headers. nginx needs explicit directives for that, HAProxy and Caddy do it by themselves, and cloud load balancers such as ALB and Cloudflare support it natively.

  2. Idle timeouts. Every hop closes a connection that carries no data for some time. The defaults range from 60 seconds (nginx, ALB) to 350 seconds (NLB). The Chatterbox heartbeat (20 seconds, see Connection Lifecycle, Heartbeats and Reconnection) must be at most a third of the shortest timeout on the path.

  3. Buffering. A reverse proxy must not hold frames back. proxy_buffering off in nginx states that intent for the WebSocket location and also protects the plain HTTP long-polling and Server-Sent Events fallbacks of Socket.IO and SignalR, whose responses nginx would otherwise buffer.

  4. Stickiness. Only needed for transports built from several HTTP requests (long polling), not for a WebSocket that is already established. See the section on sticky routing below.

  5. Client address and Origin. The server sees the proxy’s address, so the client address has to be passed in X-Forwarded-For or with the PROXY protocol, and the Origin header has to reach the server (or be checked at the proxy) for the cross-site WebSocket hijacking defence of Security.

  6. Deploys. Reloading a proxy or replacing an upstream must not drop the sockets, or if it has to, it must do so in a way that the client treats as a reconnect, not a failure.

The timeout defaults of the infrastructure covered below:

Component Default behaviour The setting to change

nginx

Closes the proxied connection after 60 seconds without data from the upstream

proxy_read_timeout (and proxy_send_timeout)

HAProxy

No default for timeout client, timeout server and timeout tunnel in a configuration that does not set them; after the upgrade timeout tunnel replaces the other two

timeout tunnel

Caddy

No idle timeout for a proxied stream; optional hard limit

stream_timeout, stream_close_delay

AWS Application Load Balancer

60 seconds idle, 1 to 4000 allowed; HTTP/2 PING frames do not reset it

idle_timeout.timeout_seconds

AWS Network Load Balancer

350 seconds for TCP (60 to 6000 allowed), fixed 350 seconds for a TLS listener, 120 seconds for UDP flows

TCP idle timeout attribute

Cloudflare

Closes a WebSocket with no data in either direction for a period it does not publish; custom value for Enterprise

An application heartbeat

ingress-nginx

60 seconds (proxy-read-timeout)

nginx.ingress.kubernetes.io/proxy-read-timeout annotation

Sources: nginx: ngx_http_proxy_module, HAProxy 3.4 configuration manual, Caddy: reverse_proxy, AWS: ALB attributes, AWS: Network Load Balancers, Cloudflare: WebSockets and ingress-nginx: ConfigMap.

A small lab: one server, three proxies

All the proxy configurations below were run against the same stand-in for the Chatterbox server. It speaks no chat protocol: it accepts the chatterbox.v1 subprotocol, sends one JSON frame with its replica name and the headers it saw, echoes every message and answers /healthz. Setting PING_MS makes it send WebSocket ping frames, as the real server does every heartbeatMs. Replace it with the real image to use the configurations in a project.

// server.mjs -- Node.js 24, "ws": "8.22.0"
import { createServer } from 'node:http';
import { hostname } from 'node:os';
import { WebSocketServer } from 'ws';

const port = Number(process.env.PORT ?? 8300);
const replica = process.env.REPLICA ?? hostname();
const pingMs = Number(process.env.PING_MS ?? 0);          // 0 = send no protocol pings

const server = createServer((req, res) => {
  if (req.url === '/healthz') {
    res.writeHead(200, { 'content-type': 'text/plain' }).end('ok\n');
    return;
  }
  res.writeHead(200, { 'content-type': 'application/json' });
  res.end(JSON.stringify({ replica, xff: req.headers['x-forwarded-for'] ?? null }) + '\n');
});

const wss = new WebSocketServer({
  server,
  path: '/ws',
  handleProtocols: (offered) => (offered.has('chatterbox.v1') ? 'chatterbox.v1' : false),
});
wss.on('connection', (ws, req) => {
  ws.send(JSON.stringify({
    type: 'hello', replica, xff: req.headers['x-forwarded-for'] ?? null,
    origin: req.headers.origin ?? null, subprotocol: ws.protocol,
  }));
  ws.on('message', (data) => ws.send(data.toString()));
});
if (pingMs > 0) setInterval(() => wss.clients.forEach((c) => c.ping()), pingMs).unref();

process.on('SIGTERM', () => {                              // graceful shutdown: tell clients, then exit
  wss.clients.forEach((c) => c.close(1001, 'going away'));
  server.close(() => process.exit(0));
});
server.listen(port, () => console.log(`${replica} listening on ${port}`));

Source: ws: API documentation

The test client uses the WebSocket class that Node.js 24 ships, which is the same API as the browser’s. It connects, echoes once, optionally stays open for a number of seconds and prints how it ended.

// client.mjs -- usage: node client.mjs ws://127.0.0.1:18401/ws [idleSeconds]
const [url, idle = '0'] = process.argv.slice(2);
const ws = new WebSocket(url, 'chatterbox.v1');
const t0 = Date.now();
let hello;
ws.onmessage = (e) => {
  if (e.data.startsWith('{')) { hello = JSON.parse(e.data); ws.send('ping-from-client'); }
  else console.log('echo:', e.data);
};
ws.onclose = (e) => {
  console.log(JSON.stringify({ sub: ws.protocol, replica: hello?.replica, xff: hello?.xff,
                               origin: hello?.origin, code: e.code, afterMs: Date.now() - t0 }));
  process.exit(0);
};
ws.onerror = () => {};
setTimeout(() => ws.close(1000), Number(idle) > 0 ? Number(idle) * 1000 : 300);

Source: MDN: WebSocket

The Compose file starts two replicas (app-a, app-b) and puts the three proxies in front of them on loopback ports 18401 (nginx), 18402 (HAProxy) and 18403 (Caddy). The proxy configurations follow in the next sections.

name: g7-proxy
x-app: &app
  image: node:24-alpine
  working_dir: /srv
  command: ["node", "server.mjs"]
  volumes:
    - ./app:/srv:ro            # server.mjs, package.json and node_modules
services:
  app-a:
    <<: *app
    environment: { PORT: "8300", PING_MS: "20000", REPLICA: app-a }
  app-b:
    <<: *app
    environment: { PORT: "8300", PING_MS: "20000", REPLICA: app-b }
  nginx:
    image: nginx:1.30-alpine
    volumes: ["./conf/nginx.conf:/etc/nginx/conf.d/default.conf:ro"]
    ports: ["127.0.0.1:18401:80"]
    depends_on: [app-a, app-b]
  haproxy:
    image: haproxy:3.4-alpine
    volumes: ["./conf/haproxy.cfg:/usr/local/etc/haproxy/haproxy.cfg:ro"]
    ports: ["127.0.0.1:18402:80"]
    depends_on: [app-a, app-b]
  caddy:
    image: caddy:2.11-alpine
    volumes: ["./conf/Caddyfile:/etc/caddy/Caddyfile:ro"]
    ports: ["127.0.0.1:18403:80"]
    depends_on: [app-a, app-b]

Source: Docker docs: Compose file reference

The site’s Docker networking and Compose Fundamentals pages explain the user-defined network that lets the proxies resolve app-a and app-b by name. Connecting four times through each proxy with the client above printed the subprotocol chatterbox.v1, the echo, close code 1000 and the client address (172.19.0.1, the Docker network gateway) in X-Forwarded-For, and /healthz answered 200 through all three. depends_on only orders the start: when nginx started before its upstream hostnames resolved, it exited with host not found in upstream, so start the upstreams first or give nginx a resolver and a variable in proxy_pass.

nginx

map $http_upgrade $connection_upgrade {
  default upgrade;
  ''      close;
}

upstream chatterbox {
  least_conn;
  server app-a:8300 max_fails=2 fail_timeout=10s;
  server app-b:8300 max_fails=2 fail_timeout=10s;
}

server {
  listen 80;
  server_name chat.example.test;

  location /ws {
    proxy_pass http://chatterbox;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection $connection_upgrade;
    proxy_set_header Host $host;
    proxy_set_header Origin $http_origin;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_read_timeout 120s;   # silence tolerated between two frames from the app
    proxy_send_timeout 120s;
    proxy_buffering off;
  }

  location / {
    proxy_pass http://chatterbox;
    proxy_set_header Host $host;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
  }
}

Source: nginx: WebSocket proxying

What each part does:

  • proxy_http_version 1.1 is required because nginx talks HTTP/1.0 to upstreams by default, and HTTP/1.0 has no Upgrade.

  • The map sends Connection: upgrade when the client asked for an upgrade and close for ordinary requests, which keeps plain HTTP requests from holding upstream connections open.

  • proxy_read_timeout is the time nginx waits for the upstream to send something. With the 20 second server ping the default of 60 seconds is already enough, 120 seconds leaves room for lost pings. It is the setting behind the "dies after exactly 60 seconds" symptom. proxy_send_timeout is its counterpart for writing to a slow upstream.

  • least_conn balances by open connections, which suits long-lived sockets better than round robin: after a deploy the new replicas start empty and round robin would not fill them quickly.

  • max_fails and fail_timeout are nginx’s passive health check. The open source edition has no active checks, so a replica that answers connections but fails /healthz stays in the pool until it refuses connections.

Two tests were run against this configuration. A client connected through a variant with proxy_read_timeout 3s to a server that sends no pings was closed after 3004 ms with code 1006; the same variant in front of a server that pings every second stayed open for the 8 seconds of the test. And a configuration that omits proxy_http_version, Upgrade and Connection answered the upgrade request with 200 and the client failed with code 1006 after 6 ms. Both are the two most common reports, and they have different fixes.

Checking the Origin at the proxy

The application must validate Origin itself (see Security and CORS, which does not apply to WebSockets), but a proxy can reject foreign origins before they cost a connection upstream:

map $http_origin $origin_allowed {
  default                      0;
  ""                           1;   # non-browser clients: authenticated by the ticket instead
  "https://chat.example.test"  1;
}

server {
  listen 80;
  location /ws {
    if ($origin_allowed = 0) { return 403; }
    proxy_pass http://app-a:8300;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection $connection_upgrade;
  }
}

Source: nginx: ngx_http_map_module

The upgrade request with Origin: https://chat.example.test got 101, the one with Origin: https://evil.example.test got 403, and a request without an Origin header (a native client) got 101. This is the same rule as the Chatterbox handshake: a request whose Origin is present and not on the allow-list is refused.

TLS termination

Browsers use wss:// from pages served over HTTPS, so the proxy normally terminates TLS and talks plain ws:// to the upstream over a private network:

server {
  listen 443 ssl;
  http2 on;
  server_name chat.example.test;
  ssl_certificate     /etc/nginx/tls/fullchain.pem;
  ssl_certificate_key /etc/nginx/tls/privkey.pem;

  location /ws {
    proxy_pass http://app-a:8300;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection $connection_upgrade;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
  }
}

Source: nginx: Configuring HTTPS servers

With a self-signed certificate for the test name, a wss://127.0.0.1:18432/ws client connected, negotiated chatterbox.v1 and echoed. The upstream sees X-Forwarded-Proto: https, which the application can use to mark cookies Secure and to refuse ws:// on a public hostname.

Reloads and long-lived connections

nginx -s reload starts new workers and lets the old ones finish: an established WebSocket is served by an old worker until it closes, so a reload does not drop sockets. The cost is that old workers can stay for hours. worker_shutdown_timeout in the main context bounds that: when it expires the remaining connections are closed. A client connected before nginx -s reload stayed connected for the 8 seconds of the test.

Source: nginx: worker_shutdown_timeout

HAProxy

global
  log stdout format raw local0
  hard-stop-after 30s         # old processes are killed this long after a reload

defaults
  mode http
  log global
  option httplog
  timeout connect 5s
  timeout client  30s         # applies before the upgrade
  timeout server  30s
  timeout tunnel  1h          # replaces both once the connection is upgraded
  timeout client-fin 5s

frontend chat_in
  bind :80
  option forwardfor
  http-request set-header X-Forwarded-Proto http
  acl is_ws hdr(Upgrade) -i websocket
  use_backend chatterbox_ws if is_ws
  default_backend chatterbox_http

backend chatterbox_ws
  balance leastconn
  option httpchk
  http-check send meth GET uri /healthz
  http-check expect status 200
  default-server check inter 2s fall 2 rise 2
  server a app-a:8300
  server b app-b:8300

backend chatterbox_http
  balance roundrobin
  option httpchk
  http-check send meth GET uri /healthz
  default-server check inter 2s
  server a app-a:8300
  server b app-b:8300

Source: HAProxy 3.4 configuration manual

  • No upgrade directives are needed. HAProxy detects the upgrade, completes the handshake and switches the connection to tunnel mode.

  • timeout tunnel is the WebSocket idle timeout. Before the upgrade, timeout client and timeout server apply; after it, only timeout tunnel. Without a timeout tunnel, the connection keeps the timeout client and timeout server values (30 seconds here), which is the HAProxy version of the "closes after a minute" bug. The test with timeout tunnel 3s closed the silent connection after 3.0 seconds with 1006, and kept the pinged one open.

  • Splitting the backend by Upgrade lets the WebSocket backend use leastconn and a long health-check history while ordinary requests use round robin. One backend works too.

  • option forwardfor adds X-Forwarded-For (verified: the server saw 172.19.0.1).

  • Active health checks are built in (check). With http-check expect status 200 a replica that returns anything else is removed after fall failures. This is the main practical advantage over the open source nginx.

  • hard-stop-after bounds how long the old process lives after a reload (SIGUSR2 in master-worker mode). Without it, old processes keep running while connections remain, as with nginx. A socket opened before the reload stayed open for the whole test.

Caddy

{
  auto_https off              # TLS is terminated elsewhere in this lab
}

:80 {
  reverse_proxy app-a:8300 app-b:8300 {
    lb_policy least_conn
    health_uri /healthz
    health_interval 5s
    stream_timeout 24h        # force-close sockets older than a day
    stream_close_delay 5m     # keep sockets open this long after a config reload
  }
}

Source: Caddy: reverse_proxy

Caddy upgrades and tunnels WebSockets without configuration and sets X-Forwarded-For, X-Forwarded-Proto and X-Forwarded-Host itself. For a real hostname (chat.example.test { ... } in place of :80) it also obtains and renews certificates automatically, which makes it the shortest path to a working wss:// endpoint. The two stream options are the part to know:

  • stream_timeout closes a proxied stream after a fixed total duration. It is a maximum lifetime, not an idle timeout: with no value a stream lives as long as both sides keep it. A variant with stream_timeout 3s closed a connection after 3.0 seconds with 1006 even though the server was pinging it.

  • stream_close_delay postpones closing streams when the configuration is reloaded; by default a reload (the caddy reload command or an API change) closes them. Combine it with clients that reconnect, because the delay only postpones the close.

Caddy has no idle timer for a proxied stream unless stream_timeout is set, so an idle-timeout problem on a Caddy deployment is usually in another hop. The Chatterbox traffic through it behaved like the other two proxies (alternating replicas, subprotocol, address header).

Sticky routing

A WebSocket that is established stays on its replica, so a proxy needs affinity only for two cases: transports that issue several requests per session (Socket.IO and SignalR long polling, Atmosphere fallbacks) and designs where a client must reach the replica that holds its state (a Pixel Arena match). Chatterbox with transports: ['websocket'] needs neither (see Socket.IO for the transport options and Scaling WebSockets for the backplane that makes affinity unnecessary).

Proxy Affinity configuration

nginx

hash $cookie_name consistent; or hash $arg_room consistent; in the upstream block; ip_hash; for the client address

HAProxy

cookie SRV insert indirect nocache httponly in the backend and cookie a or cookie b on each server line; balance source for the address

Caddy

lb_policy cookie chatterbox_node (cookie assigned on the first request) or lb_policy ip_hash

ALB

Target group stickiness: stickiness.enabled, stickiness.type, stickiness.lb_cookie.duration_seconds; not used after a WebSocket upgrade

ingress-nginx

nginx.ingress.kubernetes.io/affinity: cookie or nginx.ingress.kubernetes.io/upstream-hash-by

Sources: nginx: hash, HAProxy: cookie, Caddy: lb_policy, AWS: target group attributes

Each variant was run. HAProxy answered the first request with Set-Cookie: SRV=a and sent five requests with the cookie jar to app-a; Caddy set chatterbox_node=<hash> and kept five requests on app-b; nginx with hash $cookie_chatterbox_node consistent mapped the cookie values u1, u2, u3 to app-a, app-b, app-a repeatably. Hashing by a request attribute rather than a cookie is how Pixel Arena keeps one room on one server:

upstream arena {
  hash $arg_room consistent;     # /play?room=r1 always lands on the same server
  server app-a:8300;
  server app-b:8300;
}
server { listen 80; location / { proxy_pass http://arena; } }

Source: nginx: ngx_http_upstream_module hash

Requests for the rooms lobby, r1, r2, r3 repeated twice went to the same replica each time. consistent selects ketama consistent hashing, so adding a server moves only a fraction of the rooms. If the replica set changes, rooms hash to a different server: a match server needs a directory (a matchmaker that returns the server address) rather than hashing for anything that must survive scaling, see Game Servers and Colyseus.

Client addresses and the PROXY protocol

Behind any proxy the server’s remoteAddress is the proxy. X-Forwarded-For is an ordinary header that a client can send itself, so trust it only from known proxy addresses (the address list is what nginx’s set_real_ip_from and Express’s trust proxy configure) and use the right-most entries that your proxies appended. For a TCP-level hop (an NLB, or HAProxy in mode tcp) there is no HTTP header to add, and the PROXY protocol carries the client address in a prefix of the TCP stream instead:

defaults
  mode tcp
  timeout connect 5s
  timeout client 1h
  timeout server 1h
frontend tcp_in
  bind :80
  default_backend nginx_tier
backend nginx_tier
  server n nginx-tier:8081 send-proxy-v2
server {
  listen 8081 proxy_protocol;
  set_real_ip_from 172.16.0.0/12;           # the address of the proxy in front
  real_ip_header proxy_protocol;
  location /ws {
    proxy_pass http://app-a:8300;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection $connection_upgrade;
    proxy_set_header X-Forwarded-For $proxy_protocol_addr;
  }
}

Sources: HAProxy: PROXY protocol specification, nginx: ngx_http_realip_module

A client through that chain reached app-a and the upstream saw X-Forwarded-For: 172.19.0.1, the client address, not the address of either proxy. Both ends have to agree: a listener that expects the PROXY header and receives plain HTTP (or the reverse) fails the connection immediately, which looks like a bad gateway error.

AWS Application and Network Load Balancers

ALB supports WebSockets natively on the HTTP and HTTPS listeners: there is nothing to enable, and the target that answers 101 keeps the connection. The settings that matter:

# Raise the idle timeout (default 60 s, range 1 to 4000 s) -- must stay above the heartbeat interval
aws elbv2 modify-load-balancer-attributes \
    --load-balancer-arn "$ALB_ARN" \
    --attributes "Key=idle_timeout.timeout_seconds,Value=120"

# Shorten deregistration so a draining replica does not hold sockets for the default 300 s
aws elbv2 modify-target-group-attributes \
    --target-group-arn "$TG_ARN" \
    --attributes "Key=deregistration_delay.timeout_seconds,Value=60" \
                 "Key=load_balancing.algorithm.type,Value=least_outstanding_requests"

Source: AWS: Edit attributes for your Application Load Balancer

  • Idle timeout is 60 seconds by default; data in either direction resets it, but an HTTP/2 PING frame does not. AWS also recommends that the application’s own idle timeout be larger than the load balancer’s, otherwise the balancer may send data into a connection the application has closed and return 502.

  • Stickiness is not needed after the upgrade. The documentation states that WebSocket connections are inherently sticky and that cookie stickiness is not used once the upgrade completes. For long-polling fallbacks, enable load-balancer-cookie stickiness (AWSALB) on the target group.

  • Least outstanding requests selects the target at upgrade time; the connection then stays on it. It spreads new sockets better than round robin after a deploy.

  • Deregistration delay defaults to 300 seconds. A draining target keeps its established connections until the delay elapses, and a target that closes them earlier makes the client see a connection error. Drain in the application (send close 1001 or 4503 with a jittered reconnect, see the lifecycle page) and set the delay to match your drain time.

  • The HTTP client keepalive duration (default 3600 seconds, up to 7 days) limits how long an HTTP connection to a client is kept, but it concerns HTTP request reuse, not an upgraded WebSocket, so do not tune it for sockets.

The Network Load Balancer works at TCP: it does not understand the upgrade, which also means it cannot route by path or host. It is the right front for TCP and UDP services (TURN, an SFU) and for a TLS passthrough to your own proxies. The TCP idle timeout is 350 seconds by default and configurable from 60 to 6000 seconds; a TLS listener has a fixed 350 seconds and generates keepalives itself every 20 seconds; UDP flows are tracked for a fixed 120 seconds. When data arrives on a connection that has expired the client receives a TCP RST. If the timeout is raised above 350 seconds, the instance’s connection-tracking timeout must be raised with it (the NLB documentation describes TcpEstablishedTimeout). To preserve the client address enable proxy protocol v2 on the target group and configure the target as in the previous section.

Sources: AWS: Network Load Balancers, AWS: Update the idle timeout of your Network Load Balancer, AWS: Target groups for NLB

Cloudflare

Cloudflare proxies WebSockets on every plan; the WebSockets setting in the dashboard (Network) has to be on, and the same setting is available as the websockets zone setting in the API. Two limits matter. A connection with no data in either direction for a period is closed (the period is not published, and Enterprise customers can ask for a custom one), so the application heartbeat is the supported fix. And when Cloudflare deploys new software it may restart servers, which terminates WebSockets, so a client that does not reconnect will eventually be disconnected for no reason it can see. A reconnecting client with the jitter of the lifecycle page covers both. Its HTTP proxy does not carry arbitrary UDP: TURN and SFU media bypass it, and the media hostnames must be DNS-only records that point at the servers.

Source: Cloudflare: WebSockets

Kubernetes

The manifests for Chatterbox: a Deployment with a readiness probe, a graceful shutdown, a Service, a PodDisruptionBudget and an Ingress. The image name is a placeholder. The probes and rollout settings are explained on the site’s health probes and lifecycle, Deployments and Ingress pages.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: chatterbox
  namespace: chatterbox
spec:
  replicas: 3
  selector:
    matchLabels: { app: chatterbox }
  strategy:
    type: RollingUpdate
    rollingUpdate: { maxSurge: 1, maxUnavailable: 0 }
  template:
    metadata:
      labels: { app: chatterbox }
    spec:
      terminationGracePeriodSeconds: 60      # longer than preStop plus the close/drain time
      containers:
        - name: server
          image: registry.example.test/chatterbox/server:1.0.0
          ports:
            - { name: http, containerPort: 8300 }
          env:
            - { name: PORT, value: "8300" }
            - { name: ALLOWED_ORIGINS, value: "https://chat.example.test" }
            - name: REDIS_URL
              valueFrom: { secretKeyRef: { name: chatterbox-secrets, key: redis-url } }
          readinessProbe:
            httpGet: { path: /healthz, port: http }
            periodSeconds: 5
            failureThreshold: 2
          livenessProbe:
            httpGet: { path: /healthz, port: http }
            periodSeconds: 10
            failureThreshold: 3
          lifecycle:
            preStop:
              exec: { command: ["sleep", "10"] }   # keep serving while endpoints and the proxy catch up
          resources:
            requests: { cpu: 250m, memory: 256Mi }
            limits: { memory: 512Mi }
---
apiVersion: v1
kind: Service
metadata:
  name: chatterbox
  namespace: chatterbox
spec:
  selector: { app: chatterbox }
  ports:
    - { name: http, port: 80, targetPort: http }
---
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: chatterbox
  namespace: chatterbox
spec:
  maxUnavailable: 1
  selector:
    matchLabels: { app: chatterbox }

Source: Kubernetes: Pod termination

How a rollout treats sockets: when a pod is deleted, the control plane removes it from the Service endpoints and kubelet runs the preStop hook at the same time, then sends SIGTERM, and SIGKILL after terminationGracePeriodSeconds. The ingress controller learns about the endpoint change asynchronously, so for a few seconds it can still send new connections to a terminating pod: the sleep 10 keeps the server accepting during that window. The application then handles SIGTERM as the lab server does: close every socket with 1001 (or the Chatterbox 4503 code with a retry hint) and stop listening. With three replicas, maxUnavailable: 0 and a budget of one, a rollout moves one third of the sockets at a time, and the clients reconnect with jitter (see Scaling WebSockets for reconnection storms). Size the grace period for preStop plus the time the server needs to flush and close.

Ingress with ingress-nginx

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: chatterbox
  namespace: chatterbox
  annotations:
    nginx.ingress.kubernetes.io/proxy-read-timeout: "3600"
    nginx.ingress.kubernetes.io/proxy-send-timeout: "3600"
    nginx.ingress.kubernetes.io/limit-connections: "50"
spec:
  ingressClassName: nginx
  tls:
    - hosts: [chat.example.test]
      secretName: chat-example-test-tls
  rules:
    - host: chat.example.test
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: chatterbox
                port: { name: http }

Source: ingress-nginx: Annotations

ingress-nginx supports WebSockets out of the box; the documentation’s only requirement is to raise proxy-read-timeout and proxy-send-timeout (default 60 seconds) when connections are expected to stay idle for longer than that. The annotations used by real-time services:

Annotation (nginx.ingress.kubernetes.io/) Use

proxy-read-timeout, proxy-send-timeout

Idle limits in seconds for the proxied connection; the first one is the WebSocket idle timeout

affinity: cookie, session-cookie-name, session-cookie-samesite

Cookie affinity for long-polling transports (the default cookie is INGRESSCOOKIE)

upstream-hash-by

Consistent hashing by an nginx variable, such as $arg_room for game rooms or $http_x_tenant

limit-connections, limit-rps

Per client IP and per controller replica; exceeding the limit returns 503

backend-protocol

HTTP (default) or HTTPS to the pod

proxy-body-size

Request body limit, relevant for uploads next to the socket, not for frames

The controller’s own ConfigMap has the global defaults: worker-shutdown-timeout is 240 seconds, so after a configuration reload the old nginx workers (and their sockets) are closed after four minutes, and allow-snippet-annotations is false. Prefer the typed annotations above over a configuration-snippet.

Ingress-NGINX is retired. The Kubernetes project announced on 2026-01-29 that Ingress NGINX maintenance ended in March 2026, with no further releases, bug fixes or security patches. Existing installations keep running, but new work should use another controller or Gateway API (see the site’s Ingress page for the migration). The annotations above stay useful as a map of the concepts, because every replacement has the same knobs under other names.

Source: Kubernetes: Ingress NGINX statement

Gateway API

apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: chat
  namespace: chatterbox
spec:
  gatewayClassName: example-class
  listeners:
    - name: https
      protocol: HTTPS
      port: 443
      hostname: chat.example.test
      tls:
        certificateRefs:
          - { name: chat-example-test-tls }
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: chatterbox
  namespace: chatterbox
spec:
  parentRefs:
    - { name: chat }
  hostnames: [chat.example.test]
  rules:
    - matches:
        - path: { type: PathPrefix, value: /ws }
      timeouts:
        request: "0s"        # no request timeout: the upgraded connection is long-lived
      backendRefs:
        - { name: chatterbox, port: 80 }
    - backendRefs:
        - { name: chatterbox, port: 80 }

Source: Gateway API: HTTP timeouts

timeouts.request: 0s disables the request timeout for the rule. That field is an extended feature, so check that your controller declares support for it, and look up in its documentation how it sets the idle timeout of upgraded connections, which Gateway API does not standardise: Envoy-based controllers use their own policy resources, and nginx-based ones their own annotations or CRDs. gatewayClassName is a placeholder for the class of the controller you installed. The Gateway API page of this site covers the resources.

Pixel Arena: StatefulSet and room hashing

A match lives in the memory of one server, so players of one room must reach the same pod, and a pod must not be killed while a match runs. A StatefulSet gives each pod a stable name (pixel-arena-0), and the Ingress hashes by the room query parameter:

apiVersion: v1
kind: Service
metadata:
  name: pixel-arena
  namespace: arena
spec:
  clusterIP: None                       # headless: every pod gets a stable DNS name
  selector: { app: pixel-arena }
  ports:
    - { name: ws, port: 9001 }
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: pixel-arena
  namespace: arena
spec:
  serviceName: pixel-arena
  replicas: 4
  podManagementPolicy: Parallel
  selector:
    matchLabels: { app: pixel-arena }
  template:
    metadata:
      labels: { app: pixel-arena }
    spec:
      terminationGracePeriodSeconds: 120   # let running matches finish
      containers:
        - name: game
          image: registry.example.test/pixel-arena/server:1.0.0
          ports:
            - { name: ws, containerPort: 9001 }
          readinessProbe:
            httpGet: { path: /healthz, port: ws }
          resources:
            requests: { cpu: "1", memory: 256Mi }
---
apiVersion: v1
kind: Service
metadata:
  name: pixel-arena-lb
  namespace: arena
spec:
  selector: { app: pixel-arena }
  ports:
    - { name: ws, port: 80, targetPort: ws }
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: pixel-arena
  namespace: arena
  annotations:
    nginx.ingress.kubernetes.io/upstream-hash-by: "$arg_room"      # same room, same pod
    nginx.ingress.kubernetes.io/proxy-read-timeout: "3600"
    nginx.ingress.kubernetes.io/proxy-send-timeout: "3600"
spec:
  ingressClassName: nginx
  rules:
    - host: arena.example.test
      http:
        paths:
          - path: /play
            pathType: Prefix
            backend:
              service:
                name: pixel-arena-lb
                port: { name: ws }

Source: Kubernetes: StatefulSets

Hashing is a convenience for a fixed number of replicas; when the count changes, rooms move. For a game the more robust design is a matchmaker that allocates a room to a pod and hands the client the pod’s address (through the headless Service’s DNS name or a per-pod Service). The site’s StatefulSets page covers identity and ordering. The 120 second grace period lets a shutting-down pod finish or hand over its matches; a pod that is killed mid-match loses it.

TURN servers and media servers

Everything above is TCP and HTTP. The media path is not. WebRTC media travels over UDP between the browser and the SFU or the TURN relay, and the address and port are written into the ICE candidates that the signalling carries (see ICE, STUN and TURN). Consequences for deployment:

  • The advertised address must be the one clients can reach. A server in a private network behind NAT advertises its private address unless told otherwise: coturn’s external-ip=<public>/<private>, mediasoup’s announcedAddress, LiveKit’s use_external_ip or node_ip.

  • The port number is also advertised, so a mapping such as host 40000 to container 4000 puts the wrong number in the candidates. Publish the same port number on both sides, or use host networking.

  • UDP port ranges must be open end to end: the cloud security group, the firewall and the container runtime. One coturn allocation uses one relay port, one mediasoup WebRtcTransport uses one port unless a WebRtcServer multiplexes them on a single port (see mediasoup).

  • HTTP load balancers cannot front this traffic. Signalling (WebSocket) goes through the ordinary proxy, media goes directly to the node that holds the session. An NLB can front TURN on 3478 and 443, but the relay ports are still on the node.

  • Cloudflare and other HTTP CDNs do not carry it and the TURN and SFU hostnames must not be proxied.

Ports of the Huddle stack (from the media pages of this section):

Service Ports Notes

Signalling (WebSocket)

3000/tcp behind the proxy

Normal reverse proxy, TLS terminated at the proxy

mediasoup

One UDP and TCP port (RTC_PORT, 40000 in the examples)

With a WebRtcServer; one port per worker otherwise

LiveKit

7880/tcp (HTTP and WebSocket), 7881/tcp, UDP range 50000-60000 by default

The range is configurable (rtc.port_range_start, rtc.port_range_end)

coturn

3478 UDP and TCP, 5349 TLS (443 as a firewall fallback), relay range such as 49160-49200/udp

Needs external-ip behind NAT

Docker Compose

Compose for the three applications: Chatterbox is the lab file above (two replicas behind nginx) and, with PostgreSQL and Redis, the file of Building a Chat App; Pixel Arena is one container that publishes its WebSocket port (9001) behind the same proxy, plus a UDP range if the unreliable data-channel variant of Building a Multiplayer Game is used; Huddle’s signalling goes behind the proxy and its media services need the treatment below.

On Linux, host networking is the simplest way to satisfy the "same port inside and out" rule for a range of UDP ports, because there is no NAT and no port-by-port publishing:

name: huddle-media
services:
  coturn:
    image: coturn/coturn:4.18.0
    network_mode: host                # no NAT, no port publishing: 3478 and the whole relay range are bound on the host
    command: ["-c", "/etc/coturn/turnserver.conf", "--static-auth-secret=${TURN_SHARED_SECRET:?set TURN_SHARED_SECRET}"]
    volumes:
      - ./turnserver.conf:/etc/coturn/turnserver.conf:ro
    restart: unless-stopped
  mediasoup:
    image: registry.example.test/huddle/server:1.0.0
    network_mode: host
    environment:
      HTTP_PORT: "3000"
      RTC_PORT: "40000"
      ANNOUNCED_ADDRESS: ${ANNOUNCED_ADDRESS:?set ANNOUNCED_ADDRESS}
    restart: unless-stopped

Source: Docker docs: Host network driver

docker compose config accepted the file and reported the two missing variables when they were unset, which is what the :? syntax is for. Host networking does not behave like the Linux feature on Docker Desktop for macOS or Windows, where the lab setup published the ports instead (see the videoconference page). The TURN secret is passed as a command line flag only because this is an example; use a Docker secret or a mounted file in production, since command lines are visible in docker inspect and ps. On a bridge network the alternative is to publish every port with the mapping 49160-49200:49160-49200/udp plus the signalling and TURN ports, each with the same number on both sides. Publishing a range of thousands of ports can be slow and uses resources per port, which is a reason to keep relay and RTC ranges small (see Scaling WebRTC and TURN).

Kubernetes

For the same reason, media pods run on hostNetwork on dedicated nodes that have a public address, and a nodeSelector pins the pod to one of them:

apiVersion: v1
kind: ConfigMap
metadata:
  name: coturn
  namespace: huddle
data:
  turnserver.conf: |
    listening-port=3478
    min-port=49160
    max-port=49200
    use-auth-secret
    realm=huddle.example.test
    # replace with the node's public address / private address
    external-ip=203.0.113.10/10.0.1.10
    no-multicast-peers
    denied-peer-ip=10.0.0.0-10.255.255.255
    denied-peer-ip=172.16.0.0-172.31.255.255
    denied-peer-ip=192.168.0.0-192.168.255.255
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: coturn
  namespace: huddle
spec:
  replicas: 1
  strategy: { type: Recreate }           # the host ports cannot be bound twice on one node
  selector:
    matchLabels: { app: coturn }
  template:
    metadata:
      labels: { app: coturn }
    spec:
      hostNetwork: true
      dnsPolicy: ClusterFirstWithHostNet
      nodeSelector:
        huddle.example.test/role: turn   # a node with a public address and open firewall
      containers:
        - name: coturn
          image: coturn/coturn:4.18.0
          args:
            - "-c"
            - "/etc/coturn/turnserver.conf"
            - "--static-auth-secret=$(TURN_SHARED_SECRET)"
          env:
            - name: TURN_SHARED_SECRET
              valueFrom: { secretKeyRef: { name: huddle-secrets, key: turn-shared-secret } }
          volumeMounts:
            - { name: conf, mountPath: /etc/coturn, readOnly: true }
      volumes:
        - name: conf
          configMap: { name: coturn }
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: mediasoup
  namespace: huddle
spec:
  replicas: 1
  strategy: { type: Recreate }
  selector:
    matchLabels: { app: mediasoup }
  template:
    metadata:
      labels: { app: mediasoup }
    spec:
      hostNetwork: true                  # RTP ports are advertised as-is in ICE candidates
      dnsPolicy: ClusterFirstWithHostNet
      nodeSelector:
        huddle.example.test/role: media
      containers:
        - name: server
          image: registry.example.test/huddle/server:1.0.0
          env:
            - { name: HTTP_PORT, value: "3000" }
            - { name: RTC_PORT, value: "40000" }
            - name: ANNOUNCED_ADDRESS        # the node's address as clients see it
              value: "203.0.113.20"
---
apiVersion: v1
kind: Service
metadata:
  name: mediasoup-signaling
  namespace: huddle
spec:
  selector: { app: mediasoup }
  ports:
    - { name: http, port: 80, targetPort: 3000 }

Source: Kubernetes: Pod DNS policy

The ConfigMap content was extracted and run with the coturn 4.18.0 image: the server started, accepted the file (external-ip was parsed and its private part whitelisted) and answered a STUN binding request with the reflexive address. The denied-peer-ip lines are the protection against relaying into the cluster (see Security); the lab on the ICE page leaves 172.16.0.0/12 open only because its peers are in that range. Points about this layout:

  • hostNetwork: true needs dnsPolicy: ClusterFirstWithHostNet to keep resolving cluster Service names.

  • The Recreate strategy matters: a rolling update would start the new pod on the same node while the old one still holds the ports.

  • The signalling Service is an ordinary ClusterIP behind the Ingress; only media bypasses it. Signalling must tell the clients the address of the specific media node a room lives on when there are several (see Scaling WebRTC and TURN).

  • Alternatives to hostNetwork: hostPort entries (no range support, so it suits single ports such as a multiplexed mediasoup or LiveKit UDP port), a NodePort Service (the node port range is 30000-32767 and the number is advertised, so the application must announce that port), or a LoadBalancer Service for UDP (it adds a hop, and the load balancer’s UDP flow timeout, 120 seconds on NLB, applies). With externalTrafficPolicy: Local the client address is preserved and traffic goes only to nodes that run a pod. See the site’s Services page.

  • Restrict who can reach these nodes with security groups, not with NetworkPolicy, because host-networked pods are often not subject to pod network policy (see Network Policies).

The manifests of this page (Chatterbox, Pixel Arena, Huddle media and the Gateway) parsed as valid resources against the Kubernetes 1.34 schema in strict mode, and the Gateway and HTTPRoute against the Gateway API CRD schemas.

Common errors and pitfalls

Symptom Cause Fix

Browser reports a failed handshake; the response is a 200 HTML or JSON page

The proxy dropped Upgrade and Connection or speaks HTTP/1.0 to the upstream

nginx: proxy_http_version 1.1 and the two proxy_set_header lines

Connection closes after exactly 60 seconds

Idle timeout: nginx proxy_read_timeout, ALB 60 seconds, ingress-nginx 60 seconds

Heartbeat shorter than a third of the timeout, raise the timeout where it is yours

Connection closes after a fixed time even with traffic

A maximum lifetime: Caddy stream_timeout, a load balancer session limit, ingress worker shutdown

Remove or raise the limit and let the client reconnect with jitter

Closes right after a deploy or reload

Connection: close from a reload without drain, worker_shutdown_timeout, hard-stop-after, ALB deregistration

Drain in the application with close 1001 or 4503, size the grace periods

nginx exits with host not found in upstream

The upstream hostname does not resolve at start

Start order, resolver with a variable in proxy_pass, or a Kubernetes Service

All clients show the proxy’s address in logs and rate limits

X-Forwarded-For or PROXY protocol not used or trusted

set_real_ip_from, trust proxy, PROXY protocol v2 on an NLB

Anyone can spoof their address

X-Forwarded-For trusted from the internet

Take the right-most address added by your own proxies

403 on the handshake only from the browser

The Origin check rejects the page’s origin, or the proxy rewrote Host

Check the allow-list and pass Host/Origin through

502 from an ALB occasionally

The application closed the connection before the ALB’s idle timeout

Keep the application’s idle timeout longer than the balancer’s

Socket.IO or SignalR connects, then flaps between transports

Long polling requests are spread over replicas without affinity

Cookie stickiness, or force the WebSocket transport

Calls work on the LAN but not from outside

Media address is private or the UDP range is closed

external-ip, announcedAddress, open the UDP range, same port inside and out

A TURN relay candidate appears but media never flows

Relay ports not published or not allowed in the cloud firewall

Publish the full min-port to max-port range

Reverse proxy buffers frames in a plain HTTP fallback

proxy_buffering on for long polling or SSE

proxy_buffering off for those locations

Checklist

  • The proxy forwards the upgrade, speaks HTTP/1.1 upstream and does not buffer.

  • Every timeout on the path is at least three times the heartbeat interval; the application’s idle timeout is longer than the balancer’s.

  • Health checks use a real endpoint and remove a bad replica; draining is done in the application and the grace periods match.

  • Origin is validated (at the proxy and in the application), wss:// terminates at a hop you control, and client addresses come only from trusted hops.

  • Affinity only where it is needed, with a documented reason; hashing by room only with a fixed replica count or a directory.

  • TURN and media servers have a public address that appears in the candidates, the same port inside and out, an open UDP range and denied-peer-ip.

  • Reconnection with jitter is enabled in the clients, so a reload, a deploy or a CDN restart is a short blip.

References