Message Design and Serialization
|
This content was generated with the assistance of AI and should be verified against the official documentation before being relied on in production. See the section bibliography for the reference material consulted while preparing these pages. |
A WebSocket gives you an ordered stream of frames and nothing else: no message names, no request/response pairing, no retry safety, no versioning and no schema. Everything that makes the stream a protocol is yours to design, and a mistake here is paid for in every client you ever ship, because old mobile builds and cached browser tabs keep talking your first design for years. This page designs one complete protocol, Chatterbox v1, and then uses it as the contract that every WebSocket server page of this section implements, so that one browser client can talk to every stack.
It was written against Node.js 24.21 (type stripping on by default, the global WebSocket client stable since
22.4.0), ws 8.22.0, Zod 4.6.7, Ajv 8.20.0, @msgpack/msgpack 3.1.3, protobufjs 8.8.1 and TypeScript 7.0.2. The
standards behind it are JSON in RFC 8259 (STD 90) with the I-JSON profile of RFC 7493, the WebSocket protocol in RFC 6455
and UUIDs (including UUIDv7) in RFC 9562. Everything was checked on 2026-10-24, and every project listing on this page (the schema, server, client and scripts) was run and type-checked.
Who uses what. The chat scenario (Chatterbox) uses the whole protocol: acknowledged messages, idempotent retries, per-room ordering, history, presence, typing and receipts. The videoconference scenario (Huddle) reuses the envelope, acknowledgements and error model for its signalling messages (offers, answers and candidates, see Signaling and Perfect Negotiation), because signalling is also request/response over one socket. The game scenario (Pixel Arena) uses the same ideas but a binary encoding for its high-rate state, as measured in Choosing an encoding and used in Multiplayer Game Architecture.
Envelopes and versioning
Every frame carries a small, fixed envelope with routing and bookkeeping fields, and a type-specific payload. The envelope is the part you can never change, so keep it small and boring. Chatterbox uses six fields, which have been enough for chat, signalling and presence:
| Field | Type | Meaning |
|---|---|---|
|
integer |
Protocol major version, always |
|
string |
One of the twelve types: |
|
string |
Generated by the client for every client-to-server envelope (a UUIDv7 or a ULID, 26 to 36 characters). Server-originated
envelopes carry a server-generated id made of the prefix |
|
string or null |
A room name matching |
|
integer |
Server time in Unix epoch milliseconds, set by the server on every outbound envelope. A client-supplied |
|
object |
The type-specific body. Use |
A second field lives inside the payload of message, edit and react broadcasts: seq, a per-room, strictly
increasing integer assigned by the server. It is the ordering key (see Ordering).
Four design decisions are worth copying even if you never use Chatterbox:
-
A
typediscriminator and apayloadobject, rather than flat fields. A receiver can route ontypewithout understanding the payload, log the envelope without logging user text, and apply one validation schema per type. Flat messages with a mix of optional fields cannot be validated this strictly. -
Names that never change meaning. A field can be added (receivers ignore unknown fields, so adding an optional field is compatible within v1) but never renamed, retyped or repurposed. Removing one needs a new major version.
-
Integers for time, strings for ids. Epoch milliseconds are smaller than ISO 8601 strings and cannot be mis-parsed; ids and any 64-bit quantity are strings, because JSON numbers are IEEE 754 doubles in JavaScript and silently lose integer precision above 2^53 (RFC 7493 says to send such values as strings).
-
nullfor "explicitly nothing", absent for "unknown or unchanged".replyTo: nullmeans "not a reply"; a missingreplyTois invalid for a new message. Never use the text "null" or an empty string as a sentinel.
Version negotiation
Chatterbox negotiates the version twice, on purpose. The WebSocket handshake carries the
subprotocol token chatterbox.v1 in Sec-WebSocket-Protocol (RFC 6455, section 1.9). The browser lists the versions
it supports in preference order, the server picks one and echoes it, and a server that supports none returns the
handshake without a subprotocol (browsers then fail the connection). The v field in every envelope repeats the
decision for the benefit of logs, replayed history and proxies that see frames without the handshake.
// browser: offer v2 first, fall back to v1; the server answers with exactly one of them
const ws = new WebSocket("wss://chat.example.test/ws", ["chatterbox.v2", "chatterbox.v1"]);
ws.onopen = () => console.log("negotiated", ws.protocol); // "chatterbox.v1" or "chatterbox.v2"
The rules that keep a protocol evolvable are short:
-
Additive changes stay in v1. New optional payload fields, new error codes and new optional behaviour are fine because receivers ignore what they do not know. A new type is not additive, because an old receiver answers
bad_request; gate new types behind ahellocapability or a new version. -
Breaking changes get a new major version and a new subprotocol token. Run both versions side by side until the old clients are gone, and measure that with the
clientstring inhello. -
Never infer the version from the payload shape. Always read
v. -
Servers advertise limits in
hello(heartbeatMs,maxFrameBytes) so they can change without a client release.
Request/response correlation
WebSocket frames have no built-in reply mechanism. Chatterbox adds one with the envelope id:
-
The client generates a fresh
idand sends the request. -
The server answers with exactly one
ackor oneerrorwhose payload fieldrefequals that id. -
The client keeps a map from pending ids to promises, resolves the promise on the matching
ackand rejects it on the matchingerror, and starts a 10 second ack timeout when it sends.
Everything else is a push: broadcasts such as another user’s message or typing, which no one asked for and which
carry no ref. Two types are deliberately fire-and-forget because losing one is harmless and an ack per keystroke
would double the traffic: typing and receipt (receipts are cumulative, so the next one supersedes a lost one). The
hello request is answered by the hello reply, whose payload also contains ref.
The ack is not a delivery receipt to other users. It means only "the server accepted and stored this envelope, and
assigned seq`". Delivery and read state travel as `receipt frames
(see Presence, Typing and Receipts).
The twelve message types
| Type | Direction | Purpose | Answered by |
|---|---|---|---|
|
client to server first, then server to client |
Authenticate, announce the client, resume a session |
|
|
client to server |
Enter a room, optionally replaying from a cursor |
|
|
client to server |
Leave a room |
|
|
both |
Post a chat message / broadcast it |
|
|
both |
Change the text of your own message |
|
|
both |
Add or remove an emoji reaction |
|
|
both |
Typing indicator (fire-and-forget) |
none |
|
both |
Cumulative delivered or read marker (fire-and-forget) |
none |
|
both |
Own status and application heartbeat / broadcast of someone’s status |
|
|
client to server |
Page backwards through older messages |
|
|
server to client |
Success answer to one request |
- |
|
server to client |
Failure answer to one request, or a connection-level problem |
- |
The next subsections give each type’s payload, one complete example frame and the server behaviour. Every example below is validated against the Zod schemas of The Zod schema module by the conformance script, so they cannot drift from the code.
hello
Direction: client to server as the first frame, and server to client as the reply. The server must receive hello
within 5 seconds of the socket opening, else it closes with code 4408.
Client payload:
| Field | Type | Meaning |
|---|---|---|
|
string, optional |
A one-time ticket minted by an authenticated HTTP call. Omitted when the connection is authenticated by a cookie. |
|
string |
Client name and version such as |
|
object, optional |
|
{
"v": 1, "type": "hello", "id": "01JAB3W6A1B2C3D4E5F6G7H8J9", "room": null, "ts": 0,
"payload": {
"token": "ticket-change-me",
"client": "web/1.4",
"resume": { "sessionId": "sess_01JAB3V0000000000000000000", "rooms": {"general":41} }
}
}
Source: RFC 8259, section 4: Objects
Server payload (the reply):
| Field | Type | Meaning |
|---|---|---|
|
string |
The id of the client |
|
string |
The authenticated identity. |
|
string |
Pass it back in |
|
integer |
Heartbeat interval, 20000 by default. |
|
integer |
Largest client frame the server accepts, 16384. |
|
boolean |
|
|
array of strings |
Rooms the session is currently a member of. |
{
"v": 1, "type": "hello", "id": "s_01JAB3W7QA0000000000000000", "room": null, "ts": 1761300000000,
"payload": {
"ref": "01JAB3W6A1B2C3D4E5F6G7H8J9",
"userId": "alice",
"name": "Alice",
"sessionId": "sess_01JAB3V0000000000000000000",
"heartbeatMs": 20000,
"maxFrameBytes": 16384,
"resumed": true,
"rooms": ["general"]
}
}
Source: RFC 8259, section 5: Arrays
Server behaviour: validate the ticket (one-time, expires after 30 seconds, an invalid or replayed ticket gets
unauthorized and close 4401), create or resume the session, reply, and when resuming replay the message, edit and
react envelopes with seq greater than the client’s lastSeq for each room it names. Any frame other than hello
before the handshake completes is answered with unauthorized and close 4401; a second hello gets conflict.
join
Direction: client to server. Payload { "since": <seq or null> }. The ack result is { "members": [{ "userId", "name", "status" }], "lastSeq": integer }. The server creates the room if it does not exist, adds the user, acks, replays
envelopes newer than since when it is not null, and broadcasts the user’s presence to the other members. Direct
message rooms (dm- prefix) are open only to the two users named in the room, otherwise the answer is forbidden.
Joining a room you are already in is allowed and has no further effect.
{
"v": 1, "type": "join", "id": "01JAB3W6B1B2C3D4E5F6G7H8J9", "room": "general", "ts": 0,
"payload": { "since": 41 }
}
Source: RFC 8259, section 3: Values
leave
Direction: client to server. Payload {}, ack result {}. Leaving a room you are not in answers not_found.
{
"v": 1, "type": "leave", "id": "01JAB3W6C1B2C3D4E5F6G7H8J9", "room": "general", "ts": 0,
"payload": { }
}
Source: RFC 8259, section 4: Objects
message
Direction: client to server to post, and server to client to broadcast to every member of the room including the
sender. The envelope id is the client’s id and stays the message id forever.
Client payload:
| Field | Type | Meaning |
|---|---|---|
|
string |
1 to 4000 UTF-16 code units (the JavaScript |
|
string or null |
The id of the message this one answers, which must exist in the same room. |
|
array |
0 to 5 objects |
{
"v": 1, "type": "message", "id": "01JAB3W7Q8Z4N5R6T7V8X9Y0ZA", "room": "general", "ts": 0,
"payload": {
"text": "Hello, Chatterbox",
"replyTo": null,
"attachments": [
{
"url": "https://chat.example.test/files/menu.pdf",
"name": "menu.pdf",
"mime": "application/pdf",
"size": 48213
}
]
}
}
Source: RFC 8259, section 7: Strings
Broadcast payload: from ({ userId, name }), text, replyTo, attachments, seq and edited (initially
false).
{
"v": 1, "type": "message", "id": "01JAB3W7Q8Z4N5R6T7V8X9Y0ZA", "room": "general", "ts": 1761300000000,
"payload": {
"from": { "userId": "alice", "name": "Alice" },
"text": "Hello, Chatterbox",
"replyTo": null,
"attachments": [
{
"url": "https://chat.example.test/files/menu.pdf",
"name": "menu.pdf",
"mime": "application/pdf",
"size": 48213
}
],
"seq": 42,
"edited": false
}
}
Source: RFC 8259, section 6: Numbers
Server behaviour: the sender must be a member (else forbidden); a replyTo that is not a message of this room is
not_found; the server assigns the next seq, stores the message, answers the sender with an ack carrying seq, and
broadcasts. A duplicate id returns the original ack with dup: true and does nothing else (see Idempotency).
edit
Direction: client to server { "target": messageId, "text": string }; server to client broadcast
{ "target", "text", "seq", "editedAt" }. Only the author may edit (forbidden otherwise), the target must exist (not_found), and the edit
consumes a new seq so that every client applies edits in the same order.
{
"v": 1, "type": "edit", "id": "01JAB3WAE1B2C3D4E5F6G7H8J9", "room": "general", "ts": 0,
"payload": { "target": "01JAB3W7Q8Z4N5R6T7V8X9Y0ZA", "text": "Hello, Chatterbox!" }
}
Source: RFC 8259, section 4: Objects
{
"v": 1, "type": "edit", "id": "s_01JAB3W7QB0000000000000000", "room": "general", "ts": 1761300005000,
"payload": {
"target": "01JAB3W7Q8Z4N5R6T7V8X9Y0ZA",
"text": "Hello, Chatterbox!",
"seq": 43,
"editedAt": 1761300005000
}
}
Source: RFC 8259, section 6: Numbers
react
Direction: client to server { "target": messageId, "emoji": string, "op": "add" or "remove" }, where emoji is exactly
one grapheme cluster; server to client broadcast { "target", "emoji", "op", "from": { "userId" }, "seq" }. Adding a
reaction the user already added is conflict; removing one that is not there is not_found.
{
"v": 1, "type": "react", "id": "01JAB3WAF1B2C3D4E5F6G7H8J9", "room": "general", "ts": 0,
"payload": { "target": "01JAB3W7Q8Z4N5R6T7V8X9Y0ZA", "emoji": "👍", "op": "add" }
}
{
"v": 1, "type": "react", "id": "s_01JAB3W7QC0000000000000000", "room": "general", "ts": 1761300006000,
"payload": {
"target": "01JAB3W7Q8Z4N5R6T7V8X9Y0ZA",
"emoji": "👍",
"op": "add",
"from": { "userId": "bob" },
"seq": 44
}
}
Source: RFC 8259, section 7: Strings
typing
Direction: client to server { "state": "start" or "stop" }, fire-and-forget, throttled by the client to one frame per
3 seconds; server to the other members { "userId", "state" }. Receivers expire a start after 5 seconds, so a lost
stop never leaves an indicator stuck.
{
"v": 1, "type": "typing", "id": "01JAB3WAG1B2C3D4E5F6G7H8J9", "room": "general", "ts": 0,
"payload": { "state": "start" }
}
Source: RFC 8259, section 4: Objects
{
"v": 1, "type": "typing", "id": "s_01JAB3W7QD0000000000000000", "room": "general", "ts": 1761300007000,
"payload": { "userId": "bob", "state": "start" }
}
Source: RFC 8259, section 3: Values
receipt
Direction: client to server { "kind": "delivered" or "read", "upTo": seq }, fire-and-forget and cumulative (everything up
to and including upTo); server to the other members { "userId", "kind", "upTo" }. An upTo beyond the room’s last
seq is bad_request.
{
"v": 1, "type": "receipt", "id": "01JAB3WAH1B2C3D4E5F6G7H8J9", "room": "general", "ts": 0,
"payload": { "kind": "read", "upTo": 44 }
}
Source: RFC 8259, section 6: Numbers
{
"v": 1, "type": "receipt", "id": "s_01JAB3W7QE0000000000000000", "room": "general", "ts": 1761300008000,
"payload": { "userId": "bob", "kind": "read", "upTo": 44 }
}
Source: RFC 8259, section 4: Objects
presence
Direction: client to server { "status": "online" or "away" } with room set to null; server to room members
{ "userId", "name", "status": "online", "away" or "offline" }. This frame doubles as the application heartbeat: the
client re-sends its current status every heartbeatMs, the server acks it, and the client measures the round trip.
The server broadcasts only changes. A user becomes offline for everyone 15 seconds after their last connection
closes.
{
"v": 1, "type": "presence", "id": "01JAB3WAJ1B2C3D4E5F6G7H8J9", "room": null, "ts": 0,
"payload": { "status": "away" }
}
{
"v": 1, "type": "presence", "id": "s_01JAB3W7QF0000000000000000", "room": "general", "ts": 1761300009000,
"payload": { "userId": "alice", "name": "Alice", "status": "away" }
}
Source: RFC 8259, section 7: Strings
history
Direction: client to server { "before": seq or null, "limit": 1 to 100 } (default 50). before is exclusive and null
means "newest". The answer is an ack whose result is { "messages": [message envelopes, ascending seq], "hasMore": boolean, "next": seq or null }. Page back by sending next as the next before. Because edits and reactions
consume sequence numbers too, the message seq values on a page have gaps; that is normal.
{
"v": 1, "type": "history", "id": "01JAB3WAK1B2C3D4E5F6G7H8J9", "room": "general", "ts": 0,
"payload": { "before": 42, "limit": 2 }
}
Source: RFC 8259, section 5: Arrays
The ack that answers it:
{
"v": 1, "type": "ack", "id": "s_01JAB3W7QJ0000000000000000", "room": "general", "ts": 1761300013000,
"payload": {
"ref": "01JAB3WAK1B2C3D4E5F6G7H8J9",
"ok": true,
"seq": null,
"dup": false,
"result": {
"messages": [
{
"v": 1,
"type": "message",
"id": "01JAB3W9D2F6G8H0J1K3M5N7P9",
"room": "general",
"ts": 1761299940000,
"payload": {
"from": { "userId": "bob", "name": "Bob" },
"text": "Morning!",
"replyTo": null,
"attachments": [],
"seq": 40,
"edited": false
}
}
],
"hasMore": true,
"next": 40
}
}
}
Source: RFC 8259, section 4: Objects
Acknowledgements, errors and the code table
ack payload: { "ref": id, "ok": true, "seq": integer or null, "dup": boolean, "result": object }. It is sent for every
client envelope except typing, receipt and hello. seq is the assigned sequence for message, edit and
react, and null otherwise. dup is true when the id was already processed. result is type specific: empty for
a message, the member list for join, the page for history.
{
"v": 1, "type": "ack", "id": "s_01JAB3W7QG0000000000000000", "room": "general", "ts": 1761300000000,
"payload": { "ref": "01JAB3W7Q8Z4N5R6T7V8X9Y0ZA", "ok": true, "seq": 42, "dup": false, "result": {} }
}
Source: RFC 8259, section 4: Objects
error payload: { "ref": id or null, "code": string, "message": string, "retryable": boolean }. ref is null when the
frame was unparseable or the failure is not tied to a request. The message is for logs and developers, never for
users: show localised text chosen by code. Exactly one of ack or error answers each acked request.
{
"v": 1, "type": "error", "id": "s_01JAB3W7QK0000000000000000", "room": "general", "ts": 1761300014000,
"payload": {
"ref": "01JAB3WAE1B2C3D4E5F6G7H8J9",
"code": "forbidden",
"message": "only the author can edit a message",
"retryable": false
}
}
Source: RFC 8259, section 3: Values
| Code | Status | Retryable | When |
|---|---|---|---|
|
400 |
no |
Not JSON, unknown |
|
401 |
no |
No |
|
403 |
no |
Not a member of the room, not the author of the message, or a direct-message room that is not yours. |
|
404 |
no |
Unknown message, reaction or |
|
409 |
no |
A reaction already added, a second |
|
413 |
no |
The frame exceeds |
|
426 |
no |
|
|
429 |
yes |
More than 20 envelopes per second sustained (burst 40). Retry the same id after a delay. |
|
500 |
yes |
Unexpected server failure. Retry with the same id. |
|
503 |
yes |
Dependency down or shutting down. Retry with the same id and backoff. |
The retryable flag exists so that clients do not need a table of codes: retry with the same id when it is true,
surface the failure when it is false. The status numbers are borrowed from HTTP only as a familiar vocabulary; they
never appear on the wire as a status line.
Idempotency
Networks lose acknowledgements. When an ack is lost the client cannot tell "the server never saw my message" from "the
server saw it and I never heard back", and the only safe client policy is to retry. The only safe server policy is to
make the retry harmless. That is what an idempotency key does: the same operation, identified by the same key, has the
same effect no matter how often it is delivered. The envelope id is that key. The same idea protects HTTP payments
(Stripe’s Idempotency-Key header, and the IETF HTTP API working group’s draft of the same name, which is still an
Internet-Draft).
The Chatterbox rules:
-
The server keeps a table from
(userId, id)to the original ack for at least 24 hours and for at least the last 1000 ids per user. -
A repeated id gets that stored ack again with
dup: true. No newseqis assigned, nothing is stored and nothing is re-broadcast. -
Only acks are remembered. An error is not cached, so a retry after
rate_limitedorinternalis processed afresh (and a deterministic error such asforbiddensimply repeats). -
The rate limiter runs before deduplication, so a retry storm is still limited.
-
The client retries unacknowledged envelopes after a reconnect, with the same ids, in their original order, and after an ack timeout of 10 seconds on a live connection.
-
The client never reuses an id for a different operation, and never regenerates the id on retry.
The reference server of this page implements the table in a few lines (see reply() and the early return in onFrame()
in the listing), and the client scenario proves the behaviour three ways: an immediate resend, a resend after a dropped
socket, and no second broadcast to the other member.
The same mechanism gives exactly-once effect out of at-least-once delivery. It does not give exactly-once delivery, which does not exist; it makes duplicate delivery invisible. Keys that expire need a deployment story: if your retention is 24 hours, a client that retries after two days may create a duplicate, so client queues must also expire.
Ordering
TCP, and therefore a single WebSocket, delivers frames in order. Order breaks when you add anything else: several sockets per user (a phone and a laptop), a server cluster where two nodes broadcast the same room, a reconnect that replays history while new events arrive, or optimistic UI that renders a local echo before the ack. Wall-clock timestamps cannot fix it, because clocks disagree and two events in the same millisecond have no order.
Per-room sequence numbers
Chatterbox gives each room one counter, assigned by the server when it accepts a message, edit or react. Rules:
-
seqis a strictly increasing integer starting at 1; there is exactly one writer per room, which is what makes it cheap and gapless in a single process. In a cluster, route all writes for a room to one owner (consistent hashing) or allocate the sequence in a shared store such as a RedisINCR(see Scaling WebSockets). -
Clients render by
seq, not by arrival time orts. An optimistic local echo is shown in a "sending" state at the bottom and re-sorted when the ack arrives. -
A client tracks
lastSeqper room. A jump (it holds 41 and receives 44) means a gap. It fills the gap with ahistoryrequest or withjoinandsince, and never invents the missing envelopes. -
The cursor for reconnects, history and receipts is always a
seq, never a timestamp.
Lamport timestamps as background
A single seq per room is the right tool when one server owns the room. Peer-to-peer and multi-writer designs (data
channels between browsers, offline-first editing, replicated stores) cannot assume one writer, and use logical clocks
instead. Leslie Lamport’s 1978 paper defined the simplest one: every process keeps a counter, increments it before it
sends, stamps the message, and on receipt sets its counter to the maximum of its own and the stamp, plus one. The result
is a partial order consistent with causality: if event A could have influenced event B, then A’s stamp is smaller. The
converse does not hold (a smaller stamp does not prove causality), so concurrent events are ordered arbitrarily, and
adding the process id as a tiebreaker gives every participant the same total order. Vector clocks go one step further and
can detect concurrency, at the price of one counter per participant.
Chatterbox does not need Lamport clocks, because the server is the single sequencer, but the idea explains why seq works
and when you must graduate from it:
/** A Lamport clock: a counter that never goes backwards and jumps ahead of anything it has seen. */
export class LamportClock {
private time = 0;
/** Call before sending: advances the clock and returns the timestamp to attach. */
tick(): number { return ++this.time; }
/** Call on receive: move past the sender's timestamp so causality is preserved. */
observe(remote: number): number { this.time = Math.max(this.time, remote) + 1; return this.time; }
}
// Alice writes, Bob answers after reading it, Carol writes without having seen either.
const alice = new LamportClock(), bob = new LamportClock(), carol = new LamportClock();
const a1 = alice.tick(); // 1: "Shall we ship?"
bob.observe(a1); // Bob's clock becomes 2
const b1 = bob.tick(); // 3: "Yes" (a reply: causally after a1)
const c1 = carol.tick(); // 1: concurrent with a1, unrelated to the other two
console.log({ a1, b1, c1 });
if (!(a1 < b1)) throw new Error("a reply must carry a larger Lamport timestamp than its cause");
// Equal or unordered stamps are concurrent: break ties with (timestamp, userId) to get one total order everyone agrees on.
const order = [{ t: a1, u: "alice" }, { t: b1, u: "bob" }, { t: c1, u: "carol" }]
.sort((x, y) => x.t - y.t || x.u.localeCompare(y.u)).map((e) => `${e.t}:${e.u}`);
console.log(order.join(" < "));
Heartbeat, resume, rate limits, size limits and close codes
Heartbeat. There are two layers and Chatterbox uses both. The server sends a WebSocket protocol ping frame every
heartbeatMs (20 seconds); browsers answer with a pong automatically and JavaScript cannot see either. The client
sends a presence envelope every heartbeatMs, which the server acks, so the page can measure round-trip time and
notice a half-open connection that the operating system has not yet reported. Either side closes with code 4408 after
2 x heartbeatMs of silence. The interval must be shorter than every idle timeout on the path (an AWS Application Load
Balancer defaults to 60 seconds); the full lifecycle is in
Connection Lifecycle, Heartbeats and Reconnection.
Resume. After a drop the client reconnects on a new socket and sends hello with resume: { sessionId, rooms: { room: lastSeq } }.
If the session is still held (the server keeps it for 120 seconds) the reply has resumed: true and the server replays
the missed message, edit and react envelopes with their original seq. If not, the reply has resumed: false and
the client re-joins each room with since set to its lastSeq. Either way the client then resends its unacknowledged
envelopes, and idempotency makes that safe.
Rate limits. 20 envelopes per second per connection with a burst of 40 (a token bucket). Beyond it the server answers
rate_limited (retryable); ten of those within ten seconds close the socket with code 4429.
Size limits. 16384 bytes per client frame; larger frames get too_large and close 4413 (the transport guard may close
with the standard code 1009 first). Text is 4000 code units and attachments are limited to five references. Frames the
server sends may be larger, because a history page is not a client frame, but the server keeps them under 262144 bytes
by returning fewer messages and hasMore: true.
Close codes. Browsers may send only 1000 and codes from 3000 to 4999, so application codes live in the 4xxx range; 1005, 1006 and 1015 are reserved and never sent on the wire (1006 is what JavaScript reports for an abnormal drop).
| Code | Name | Client reaction |
|---|---|---|
1000 |
Normal closure |
Do not reconnect. |
1001 |
Going away |
Reconnect with backoff. |
1006 |
Abnormal closure (never on the wire) |
Reconnect with backoff. |
4400 |
Protocol error (for example a binary frame) |
Fix the bug; do not reconnect in a loop. |
4401 |
Unauthorized |
Get a new ticket or sign in again; then reconnect. |
4403 |
Forbidden |
Stop; the account may not use this service. |
4408 |
Hello or heartbeat timeout |
Reconnect with backoff. |
4413 |
Frame too large |
Fix the bug; do not resend the same frame. |
4426 |
Unsupported version |
Offer another subprotocol or ask the user to update. |
4429 |
Rate limited |
Reconnect with a long delay. |
4503 |
Server restarting |
Reconnect at once with random jitter, because the restart is planned. |
A message round trip
The diagram shows the happy path of one message to a second member, and then the failure that idempotency exists for: the ack is lost on the way back, the client times out after 10 seconds and resends the same envelope, and the server answers from its table without storing or broadcasting a second time.
Choosing an encoding
JSON is the default for a reason: every language parses it, browsers have JSON.parse built in, and a frame can be read
in the browser’s network panel, in a log file and in a bug report. Its costs are size (every field name is repeated in
every frame, and numbers are decimal text), parse time, and a data model of strings, doubles, booleans, null, arrays and
objects, with no bytes, no 64-bit integers, no dates and no undefined, NaN or Infinity. The alternatives trade
readability for bytes:
| Encoding | What it is | Strengths | Costs |
|---|---|---|---|
JSON |
Text, RFC 8259, UTF-8 |
Debuggable everywhere, no tooling, schemas by Zod or JSON Schema |
Verbose; doubles only; field names on every frame |
MessagePack |
Binary, JSON’s data model, a published spec with extension types (timestamp is built in) |
Drop-in for JSON, no schema file, a little smaller, has |
Still carries every field name; needs a library; opaque in the network panel |
Protocol Buffers |
Binary with a compiled schema ( |
Smallest structured option, typed code generation in every language, defined evolution rules |
Build step; the schema must be shared; harder to debug; union of request and reply types is clumsy |
Hand-packed binary ( |
A fixed byte layout you define and document |
Smallest and fastest; no dependency; the only sane choice for 60 Hz game state |
You own every bug: endianness, alignment, versioning and extensibility |
Binary frames are a WebSocket feature, not an extra: the sender uses ws.send(arrayBuffer) and the receiver sees a
binary frame. In the browser set the binaryType property of the socket to the string arraybuffer before the first message, because the default is
blob, which has an asynchronous read API. A server that speaks both encodings must pick one per connection,
usually by subprotocol (chatterbox.v1 for JSON, chatterbox.v1.msgpack for MessagePack), never per frame by sniffing.
Measured sizes
The following table was produced by size-budget.ts (listed below) on Node 24.21 with the library versions given at the
top. All four encodings carry the same logical frame: the server broadcast of a chat message, the typing broadcast,
an ack, and a history page of 50 messages. "Deflate" is the raw DEFLATE size of the encoded bytes, which approximates what
permessage-deflate (RFC 7692) puts on the wire with context takeover disabled, because each frame is compressed on its
own. The hand-packed layout does not carry attachment contents (the sample has none); the Protocol Buffers numbers use
the schema printed below.
| Message | Encoding | Bytes | vs JSON | Deflate | vs JSON deflate |
|---|---|---|---|---|---|
message, 17 characters |
JSON |
230 |
100% |
185 |
100% |
message, 17 characters |
MessagePack |
169 |
73% |
170 |
92% |
message, 17 characters |
Protocol Buffers |
85 |
37% |
85 |
46% |
message, 17 characters |
Hand-packed DataView |
72 |
31% |
73 |
39% |
message, 349 characters |
JSON |
562 |
100% |
376 |
100% |
message, 349 characters |
MessagePack |
503 |
90% |
371 |
99% |
message, 349 characters |
Protocol Buffers |
419 |
75% |
311 |
83% |
message, 349 characters |
Hand-packed DataView |
404 |
72% |
291 |
77% |
typing start |
JSON |
138 |
100% |
109 |
100% |
typing start |
MessagePack |
105 |
76% |
93 |
85% |
typing start |
Protocol Buffers |
58 |
42% |
47 |
43% |
typing start |
Hand-packed DataView |
39 |
28% |
34 |
31% |
ack |
JSON |
182 |
100% |
141 |
100% |
ack |
MessagePack |
132 |
73% |
116 |
82% |
ack |
Protocol Buffers |
81 |
45% |
63 |
45% |
ack |
Hand-packed DataView |
57 |
31% |
49 |
35% |
history page, 50 messages |
JSON |
13865 |
100% |
2358 |
100% |
history page, 50 messages |
MessagePack |
10715 |
77% |
2505 |
106% |
history page, 50 messages |
Protocol Buffers |
6525 |
47% |
2481 |
105% |
history page, 50 messages |
Hand-packed DataView |
5705 |
41% |
2072 |
88% |
How to read the table:
-
For small frames the field names are most of the bytes. A 17-character message is 230 bytes of JSON of which only 17 are the text; MessagePack saves a quarter, Protocol Buffers and the hand-packed layout save about two thirds.
-
The advantage shrinks as the text grows: at 349 characters MessagePack saves only 10 percent, because text dominates. Chat is text, so for Chatterbox the gain is real but modest.
-
Compression closes much of the gap. After deflate, JSON on a history page is as small as MessagePack and Protocol Buffers, because the repeated field names compress away. If your server already runs
permessage-deflate, the remaining win of a binary format is parse speed and the small frames (typing, acks, presence), which are the ones that compression helps least because they are too short to build up a dictionary. -
Compression has a price: per-connection memory for the compression context, and a length oracle if secrets and attacker-controlled text share a context (a CRIME or BREACH style attack). See Security.
-
Size budget. Treat bytes as a budget per feature, not a general goal. Chatterbox: 16 KiB per client frame is generous (a 4000-character message in JSON is about 4.2 KB with its envelope) and keeps one slow message from monopolising the connection; typing and receipts must stay under 150 bytes because they are frequent; one history page should stay under 100 KiB so it parses in one frame budget. Pixel Arena: 20 snapshots per second to 8 players means any per-snapshot overhead is multiplied 160 times per second, which is why its snapshots are hand-packed (see Latency Compensation).
The measuring script, also showing the three binary encoders at work:
import { readFileSync } from "node:fs";
import { deflateRawSync } from "node:zlib";
import { encode as mpEncode, decode as mpDecode } from "@msgpack/msgpack";
import protobuf from "protobufjs";
const root = protobuf.parse(readFileSync(new URL("./chatterbox.proto", import.meta.url), "utf8"), { keepCase: false }).root;
const Envelope = root.lookupType("chatterbox.v1.Envelope");
const CROCKFORD = "0123456789ABCDEFGHJKMNPQRSTVWXYZ";
// ---- the same logical messages in every encoding
const ID = "01JAB3W7Q8Z4N5R6T7V8X9Y0ZA";
const T0 = 1761300000000;
const long = "The kitchen team confirmed the new menu: starters at 12:00, mains at 13:30, dessert and coffee at 15:00. " +
"Vegetarian and vegan options are marked with a leaf, allergens are listed per dish, and the printed copies arrive tomorrow morning. " +
"Please react with a thumbs up when you have read this so that we can close the thread before the end of the day.";
const msg = (text: string, seq: number, id = ID, ts = T0) => ({
v: 1, type: "message", id, room: "general", ts,
payload: { from: { userId: "alice", name: "Alice" }, text, replyTo: null, attachments: [], seq, edited: false },
});
// deterministic pseudo-random chat text so the history page is not unrealistically repetitive
let seed = 7;
const rnd = () => (seed = (seed * 1103515245 + 12345) % 2147483648) / 2147483648;
const words = "lunch meeting tomorrow deploy staging release fixed review thanks coffee later build green failing retry cache design ticket sprint demo budget invoice train late sorry done agreed maybe monday".split(" ");
const chatter = (i: number) => Array.from({ length: 3 + Math.floor(rnd() * (i % 5 === 0 ? 22 : 9)) }, () => words[Math.floor(rnd() * words.length)]).join(" ");
const historyId = (i: number) => "01JAB3W7Q8" + Array.from({ length: 16 }, () => CROCKFORD[Math.floor(rnd() * 32)]).join("");
const samples: Record<string, any> = {
"message, 17 characters": msg("Hello, Chatterbox", 42),
[`message, ${long.length} characters`]: msg(long, 42),
"typing start": { v: 1, type: "typing", id: "s_01JAB3W7QD".padEnd(28, "0"), room: "general", ts: T0, payload: { userId: "bob", state: "start" } },
"ack": { v: 1, type: "ack", id: "s_01JAB3W7QG".padEnd(28, "0"), room: "general", ts: T0, payload: { ref: ID, ok: true, seq: 42, dup: false, result: {} } },
"history page, 50 messages": {
v: 1, type: "ack", id: "s_01JAB3W7QJ".padEnd(28, "0"), room: "general", ts: T0,
payload: { ref: ID, ok: true, seq: null, dup: false, result: {
messages: Array.from({ length: 50 }, (_, i) => msg(chatter(i), 100 + i, historyId(i), T0 - (50 - i) * 41000)),
hasMore: true, next: 100 } },
},
};
// ---- protobuf mapping (a real implementation would generate this with protoc / ts-proto)
function toProto(f: any): any {
const base = { v: f.v, id: f.id, room: f.room ?? "", ts: f.ts };
const p = f.payload;
switch (f.type) {
case "message": return { ...base, message: { text: p.text, from: { userId: p.from.userId, name: p.from.name }, seq: p.seq, edited: p.edited, attachments: p.attachments } };
case "typing": return { ...base, typing: { start: p.state === "start", userId: p.userId } };
case "ack": return { ...base, ack: { ref: p.ref, seq: p.seq ?? undefined, dup: p.dup, historyMessages: p.result.messages?.map(toProto), hasMore: p.result.hasMore ?? false, next: p.result.next } };
}
}
// ---- hand-packed binary with DataView (fixed layout, no field names, no extensibility)
function ulidBytes(id: string): Uint8Array {
const body = id.replace(/^s_/, "");
let n = 0n;
for (const ch of body) n = (n << 5n) | BigInt(CROCKFORD.indexOf(ch));
const out = new Uint8Array(16);
for (let i = 15; i >= 0; i--) { out[i] = Number(n & 0xffn); n >>= 8n; }
return out;
}
const enc = new TextEncoder();
const dec = new TextDecoder();
const TYPE_CODE = { hello: 1, join: 2, leave: 3, message: 4, edit: 5, react: 6, typing: 7, receipt: 8, presence: 9, history: 10, ack: 11, error: 12 } as const;
class Writer {
buf = new Uint8Array(65536); view = new DataView(this.buf.buffer); pos = 0;
u8(v: number) { this.view.setUint8(this.pos, v); this.pos += 1; }
u16(v: number) { this.view.setUint16(this.pos, v); this.pos += 2; }
u32(v: number) { this.view.setUint32(this.pos, v); this.pos += 4; }
f64(v: number) { this.view.setFloat64(this.pos, v); this.pos += 8; }
bytes(b: Uint8Array) { this.buf.set(b, this.pos); this.pos += b.length; }
str8(s: string) { const b = enc.encode(s); this.u8(b.length); this.bytes(b); }
str16(s: string) { const b = enc.encode(s); this.u16(b.length); this.bytes(b); }
done() { return this.buf.slice(0, this.pos); }
}
function packFrame(w: Writer, f: any): void {
w.u8(f.v); w.u8(TYPE_CODE[f.type as keyof typeof TYPE_CODE]);
w.bytes(ulidBytes(f.id));
w.str8(f.room ?? ""); w.f64(f.ts);
const p = f.payload;
if (f.type === "message") { w.str8(p.from.userId); w.str8(p.from.name); w.u32(p.seq); w.u8(p.edited ? 1 : 0); w.u8(p.replyTo ? 1 : 0); w.u8(p.attachments.length); w.str16(p.text); }
else if (f.type === "typing") { w.str8(p.userId); w.u8(p.state === "start" ? 1 : 0); }
else if (f.type === "ack") {
w.bytes(ulidBytes(p.ref)); w.u8((p.ok ? 1 : 0) | (p.dup ? 2 : 0)); w.u32(p.seq ?? 0);
const msgs = p.result.messages;
if (msgs) { w.u16(msgs.length); for (const m of msgs) packFrame(w, m); w.u32(p.result.next ?? 0); w.u8(p.result.hasMore ? 1 : 0); } else w.u16(0);
}
}
function unpackMessageText(bin: Uint8Array): string { // enough of a decoder to prove the layout round-trips
const v = new DataView(bin.buffer, bin.byteOffset, bin.byteLength);
let pos = 2 + 16; pos += 1 + v.getUint8(pos); pos += 8;
pos += 1 + v.getUint8(pos); pos += 1 + v.getUint8(pos); pos += 4 + 3;
const len = v.getUint16(pos);
return dec.decode(bin.subarray(pos + 2, pos + 2 + len));
}
// ---- measure
const rows: string[][] = [];
const pct = (n: number, base: number) => `${Math.round((n / base) * 100)}%`;
console.log("| Message | Encoding | Bytes | vs JSON | deflate (raw) | vs JSON deflate |");
console.log("|---|---|---:|---:|---:|---:|");
for (const [name, frame] of Object.entries(samples)) {
const json = new TextEncoder().encode(JSON.stringify(frame));
const mp = mpEncode(frame);
const results: [string, Uint8Array][] = [["JSON", json], ["MessagePack", mp]];
const pf = toProto(frame);
const pb = Envelope.encode(Envelope.fromObject(pf)).finish();
results.push(["Protocol Buffers", pb]);
const w = new Writer(); packFrame(w, frame); const hp = w.done();
results.push(["Hand-packed DataView", hp]);
// correctness: every encoding decodes back to the same text
const text = frame.payload.text ?? null;
if (text) {
if ((mpDecode(mp) as any).payload.text !== text) throw new Error("msgpack round trip");
if ((Envelope.toObject(Envelope.decode(pb), { longs: Number }) as any).message.text !== text) throw new Error("protobuf round trip");
if (unpackMessageText(hp) !== text) throw new Error("hand-packed round trip");
}
const jsonDeflate = deflateRawSync(json).length;
for (const [enc_, bytes] of results) {
const d = deflateRawSync(bytes).length;
console.log(`| ${name} | ${enc_} | ${bytes.length} | ${pct(bytes.length, json.length)} | ${d} | ${pct(d, jsonDeflate)} |`);
}
}
The Protocol Buffers schema it loads. The type string disappears (the populated oneof member is the type), field
numbers 1 to 15 take a single tag byte, which is why the hot fields have low numbers, and field numbers must never be
reused once published. A production schema would split requests and replies instead of sharing messages as this
size-measurement schema does.
syntax = "proto3";
package chatterbox.v1;
// The envelope fields v, id, room and ts are the same as in JSON. The "type" field disappears: the
// populated member of the oneof IS the type. An empty room string stands for JSON null.
message Envelope {
uint32 v = 1;
string id = 2;
string room = 3;
uint64 ts = 4;
oneof payload {
Hello hello = 10;
Join join = 11;
Leave leave = 12;
Message message = 13;
Edit edit = 14;
React react = 15;
Typing typing = 16;
Receipt receipt = 17;
Presence presence = 18;
History history = 19;
Ack ack = 20;
Error error = 21;
}
}
message User { string user_id = 1; string name = 2; }
message Attachment { string url = 1; string name = 2; string mime = 3; uint32 size = 4; }
message Hello {
string token = 1; // client -> server
string client = 2; // client -> server
string resume_session = 3; // client -> server
map<string, uint64> resume_rooms = 4;
string ref = 5; // server -> client from here on
string user_id = 6;
string name = 7;
string session_id = 8;
uint32 heartbeat_ms = 9;
uint32 max_frame_bytes = 10;
bool resumed = 11;
repeated string rooms = 12;
}
message Join { optional uint64 since = 1; }
message Leave {}
message Message {
string text = 1;
optional string reply_to = 2;
repeated Attachment attachments = 3;
User from = 4; // server -> client
uint64 seq = 5; // server -> client
bool edited = 6; // server -> client
}
message Edit { string target = 1; string text = 2; uint64 seq = 3; uint64 edited_at = 4; }
message React { string target = 1; string emoji = 2; bool remove = 3; string from_user_id = 4; uint64 seq = 5; }
message Typing { bool start = 1; string user_id = 2; }
message Receipt { bool read = 1; uint64 up_to = 2; string user_id = 3; }
message Presence { enum Status { ONLINE = 0; AWAY = 1; OFFLINE = 2; } Status status = 1; string user_id = 2; string name = 3; }
message History { optional uint64 before = 1; uint32 limit = 2; }
message Ack {
string ref = 1;
optional uint64 seq = 2;
bool dup = 3;
repeated Envelope history_messages = 4; // result of a history request
bool has_more = 5;
optional uint64 next = 6;
}
message Error { string ref = 1; string code = 2; string message = 3; bool retryable = 4; }
The hand-packed message layout used in the measurement is, in order: one byte v, one byte type code, 16 bytes of id
(the 26-character ULID decoded from base 32), a length-prefixed room (one length byte), the server time as a big-endian
64-bit float, length-prefixed userId and name, the seq as an unsigned 32-bit integer, three flag bytes (edited,
has-reply, attachment count) and finally the text with a 16-bit length. A DataView reads and writes it with explicit
offsets and explicit endianness; the default of the typed arrays (Uint32Array and friends) is the platform’s
endianness, which is why binary protocols use DataView and name the byte order:
const buf = new ArrayBuffer(8);
const view = new DataView(buf);
view.setUint32(0, 42); // big-endian (network order) by default
view.setUint32(4, 42, true); // little-endian only when the second argument is true
console.log(new Uint8Array(buf)); // Uint8Array(8) [0, 0, 0, 42, 42, 0, 0, 0]
Source: MDN: DataView
Which to pick. Use JSON for control-plane traffic such as Chatterbox, and for anything a human will ever need to
read. Switch to MessagePack when profiling shows JSON parsing or bandwidth on mobile as a real cost and you want no
schema tooling. Choose Protocol Buffers when many languages share the contract and you want generated types and
enforced evolution rules. Hand-pack only the hot data path, as a game does, and keep the control plane in JSON on the
same socket (a text frame for the envelope, a binary frame for state). Optional: an edit could be expressed as a JSON
Merge Patch (RFC 7396) of the message object, which is compact for large documents, but for a chat message replacing
the whole text is simpler, and merge patch cannot express "set to null" distinctly from "remove".
Schema validation on both ends
The WebSocket gives you bytes from a stranger. JSON.parse returns any, so TypeScript types alone prove nothing at the
boundary: a cast tells the compiler to stop checking, and the first malformed frame surfaces as an exception deep inside a
handler. Validate at the edge, once, and let the rest of the code trust the types.
Server side (hostile input). Parse inside try, check v, validate the envelope and the payload for the declared
type, and answer a failure with bad_request carrying the ref when it can be recovered. Reject, do not repair.
Unknown fields are ignored (and stripped, so they cannot leak into your storage), strings and arrays have maximum
lengths, numbers are integers within a range, and ids and rooms are matched against a regular expression with no
nested quantifiers. A size check before JSON.parse stops memory exhaustion: the 16384-byte limit is applied on the
raw text first.
Client side (a server you do not fully control). The same schemas, run over every inbound frame. A server bug, a
proxy that mangles a frame, an old cached client talking to a new server, or a compromised server must not be able to
inject a value that the UI then trusts. The reference server also validates its own outbound frames in sendRaw(), so a
server bug throws in tests rather than reaching a client.
Zod 4 expresses the protocol as data: one object schema per type, a discriminated union on type so a failure points
at one branch, and z.infer giving the TypeScript types for free. Two notes: z.int() rejects fractions and unsafe
integers, and z.toJSONSchema() exports the same schema as JSON Schema (draft 2020-12) so that non-TypeScript clients
and tools such as Ajv can enforce the contract. The conversion cannot express JavaScript refinements, such as the
grapheme-count check on emoji, so the JSON Schema is the looser of the two: keep Zod authoritative where it
exists, and use the JSON Schema to generate documentation, client validators in other languages and API tooling.
The Zod schema module
All twelve types, with the two directions of the types that travel both ways, and the parseClientFrame() helper the
server uses. Save as chatterbox-schema.ts.
import { z } from "zod";
export const PROTOCOL_VERSION = 1;
export const SUBPROTOCOL = "chatterbox.v1";
export const MAX_FRAME_BYTES = 16384;
// ---------------------------------------------------------------- primitives
export const ROOM_RE = /^[a-z0-9-]{1,64}$/;
const Room = z.string().regex(ROOM_RE);
/** Client ids: UUIDv7 or ULID, 26-36 characters. */
const ClientId = z.string().regex(/^[0-9A-Za-z-]{26,36}$/);
/** Server ids: "s_" followed by a ULID. */
const ServerId = z.string().regex(/^s_[0-9A-HJKMNP-TV-Z]{26}$/);
const Seq = z.int().min(1);
const Text = z.string().min(1).max(4000); // UTF-16 code units, i.e. String.length
const UserId = z.string().min(1).max(64);
const Emoji = z.string().refine((s) => [...new Intl.Segmenter().segment(s)].length === 1, "exactly one grapheme");
const Status = z.enum(["online", "away", "offline"]);
const Attachment = z.object({
url: z.url().regex(/^https:\/\//),
name: z.string().min(1).max(255),
mime: z.string().min(3).max(127),
size: z.int().min(0),
});
// ---------------------------------------------------------------- envelope factory
function envelope<T extends string, I extends z.ZodType, R extends z.ZodType, P extends z.ZodType>(
type: T,
id: I,
room: R,
payload: P,
) {
return z.object({ v: z.literal(PROTOCOL_VERSION), type: z.literal(type), id, room, ts: z.int().min(0), payload });
}
const clientFrame = <T extends string, R extends z.ZodType, P extends z.ZodType>(t: T, room: R, p: P) =>
envelope(t, ClientId, room, p);
const serverFrame = <T extends string, R extends z.ZodType, P extends z.ZodType>(t: T, room: R, p: P) =>
envelope(t, ServerId, room, p);
const Empty = z.object({});
const Member = z.object({ userId: UserId, name: z.string(), status: Status });
// ---------------------------------------------------------------- client -> server
export const HelloC = clientFrame("hello", z.null(), z.object({
token: z.string().min(1).optional(), // omitted when the connection is authenticated by a cookie
client: z.string().min(1).max(64),
resume: z.object({ sessionId: z.string().min(1), rooms: z.record(Room, Seq.or(z.literal(0))) }).optional(),
}));
export const Join = clientFrame("join", Room, z.object({ since: Seq.or(z.literal(0)).nullable() }));
export const Leave = clientFrame("leave", Room, Empty);
export const MessageC = clientFrame("message", Room, z.object({
text: Text,
replyTo: ClientId.nullable(),
attachments: z.array(Attachment).max(5),
}));
export const Edit = clientFrame("edit", Room, z.object({ target: ClientId, text: Text }));
export const React = clientFrame("react", Room, z.object({
target: ClientId,
emoji: Emoji,
op: z.enum(["add", "remove"]),
}));
export const TypingC = clientFrame("typing", Room, z.object({ state: z.enum(["start", "stop"]) }));
export const ReceiptC = clientFrame("receipt", Room, z.object({
kind: z.enum(["delivered", "read"]),
upTo: Seq,
}));
export const PresenceC = clientFrame("presence", z.null(), z.object({ status: z.enum(["online", "away"]) }));
export const History = clientFrame("history", Room, z.object({
before: Seq.nullable(),
limit: z.int().min(1).max(100).default(50),
}));
export const ClientFrame = z.discriminatedUnion("type", [
HelloC, Join, Leave, MessageC, Edit, React, TypingC, ReceiptC, PresenceC, History,
]);
// ---------------------------------------------------------------- server -> client
export const HelloS = serverFrame("hello", z.null(), z.object({
ref: ClientId,
userId: UserId,
name: z.string(),
sessionId: z.string().min(1),
heartbeatMs: z.int().min(1),
maxFrameBytes: z.int().min(1),
resumed: z.boolean(),
rooms: z.array(Room),
}));
// A broadcast message keeps the CLIENT id, because the message id is the envelope id forever.
export const MessageS = clientFrame("message", Room, z.object({
from: z.object({ userId: UserId, name: z.string() }),
text: Text,
replyTo: ClientId.nullable(),
attachments: z.array(Attachment).max(5),
seq: Seq,
edited: z.boolean(),
}));
export const EditS = serverFrame("edit", Room, z.object({
target: ClientId, text: Text, seq: Seq, editedAt: z.int().min(0),
}));
export const ReactS = serverFrame("react", Room, z.object({
target: ClientId,
emoji: Emoji,
op: z.enum(["add", "remove"]),
from: z.object({ userId: UserId }),
seq: Seq,
}));
export const TypingS = serverFrame("typing", Room, z.object({
userId: UserId, state: z.enum(["start", "stop"]),
}));
export const ReceiptS = serverFrame("receipt", Room, z.object({
userId: UserId, kind: z.enum(["delivered", "read"]), upTo: Seq,
}));
export const PresenceS = serverFrame("presence", Room, z.object({
userId: UserId, name: z.string(), status: Status,
}));
export const Ack = serverFrame("ack", Room.nullable(), z.object({
ref: ClientId,
ok: z.literal(true),
seq: Seq.nullable(),
dup: z.boolean(),
result: z.record(z.string(), z.unknown()),
}));
/** Error codes with their HTTP-style status and whether a retry with the same id can succeed. */
export const ERROR_CODES = {
bad_request: { status: 400, retryable: false },
unauthorized: { status: 401, retryable: false },
forbidden: { status: 403, retryable: false },
not_found: { status: 404, retryable: false },
conflict: { status: 409, retryable: false },
too_large: { status: 413, retryable: false },
unsupported_version: { status: 426, retryable: false },
rate_limited: { status: 429, retryable: true },
internal: { status: 500, retryable: true },
unavailable: { status: 503, retryable: true },
} as const;
export type ErrorCode = keyof typeof ERROR_CODES;
export const ErrorFrame = serverFrame("error", Room.nullable(), z.object({
ref: ClientId.nullable(),
code: z.enum(Object.keys(ERROR_CODES) as [ErrorCode, ...ErrorCode[]]),
message: z.string(),
retryable: z.boolean(),
}));
export const ServerFrame = z.discriminatedUnion("type", [
HelloS, MessageS, EditS, ReactS, TypingS, ReceiptS, PresenceS, Ack, ErrorFrame,
]);
// ---------------------------------------------------------------- inferred types
export type ClientFrameT = z.infer<typeof ClientFrame>;
export type ServerFrameT = z.infer<typeof ServerFrame>;
export type MessageFrame = z.infer<typeof MessageS>;
export type AckFrame = z.infer<typeof Ack>;
export type ErrorFrameT = z.infer<typeof ErrorFrame>;
/** Parse one inbound text frame. Never throws: callers answer failures with a bad_request error. */
export function parseClientFrame(text: string):
| { ok: true; frame: ClientFrameT }
| { ok: false; code: "bad_request" | "unsupported_version"; message: string; ref: string | null } {
let raw: unknown;
try {
raw = JSON.parse(text);
} catch {
return { ok: false, code: "bad_request", message: "frame is not valid JSON", ref: null };
}
const obj = typeof raw === "object" && raw !== null ? (raw as Record<string, unknown>) : {};
const ref = typeof obj.id === "string" && ClientId.safeParse(obj.id).success ? obj.id : null;
if (obj.v !== PROTOCOL_VERSION) {
return { ok: false, code: "unsupported_version", message: `unsupported protocol version ${String(obj.v)}`, ref };
}
const parsed = ClientFrame.safeParse(raw);
if (!parsed.success) {
const issue = parsed.error.issues[0];
return { ok: false, code: "bad_request", message: `${issue.path.join(".") || "frame"}: ${issue.message}`, ref };
}
return { ok: true, frame: parsed.data };
}
Example frames and the conformance script
frames.ts holds one valid frame per type and direction (the same frames printed in the sections above) and a list of
frames every conforming receiver must reject. The conformance script validates all of them and checks that each
invalid frame produces the documented error code. Use the same file as a test fixture in every other language.
/** One valid example frame per type and direction. The documentation page prints these exact frames. */
export interface Example { name: string; direction: "c2s" | "s2c"; frame: Record<string, unknown> }
const M1 = "01JAB3W7Q8Z4N5R6T7V8X9Y0ZA";
const M2 = "01JAB3W9D2F6G8H0J1K3M5N7P9";
const SID = (n: string) => `s_01JAB3W7Q${n.padEnd(17, "0")}`;
const T0 = 1761300000000;
export const examples: Example[] = [
{ name: "hello (client)", direction: "c2s", frame: {
v: 1, type: "hello", id: "01JAB3W6A1B2C3D4E5F6G7H8J9", room: null, ts: 0,
payload: { token: "ticket-change-me", client: "web/1.4",
resume: { sessionId: "sess_01JAB3V0000000000000000000", rooms: { general: 41 } } } } },
{ name: "hello (server)", direction: "s2c", frame: {
v: 1, type: "hello", id: SID("A"), room: null, ts: T0,
payload: { ref: "01JAB3W6A1B2C3D4E5F6G7H8J9", userId: "alice", name: "Alice", sessionId: "sess_01JAB3V0000000000000000000",
heartbeatMs: 20000, maxFrameBytes: 16384, resumed: true, rooms: ["general"] } } },
{ name: "join", direction: "c2s", frame: {
v: 1, type: "join", id: "01JAB3W6B1B2C3D4E5F6G7H8J9", room: "general", ts: 0, payload: { since: 41 } } },
{ name: "leave", direction: "c2s", frame: {
v: 1, type: "leave", id: "01JAB3W6C1B2C3D4E5F6G7H8J9", room: "general", ts: 0, payload: {} } },
{ name: "message (client)", direction: "c2s", frame: {
v: 1, type: "message", id: M1, room: "general", ts: 0,
payload: { text: "Hello, Chatterbox", replyTo: null,
attachments: [{ url: "https://chat.example.test/files/menu.pdf", name: "menu.pdf", mime: "application/pdf", size: 48213 }] } } },
{ name: "message (server)", direction: "s2c", frame: {
v: 1, type: "message", id: M1, room: "general", ts: T0,
payload: { from: { userId: "alice", name: "Alice" }, text: "Hello, Chatterbox", replyTo: null,
attachments: [{ url: "https://chat.example.test/files/menu.pdf", name: "menu.pdf", mime: "application/pdf", size: 48213 }],
seq: 42, edited: false } } },
{ name: "edit (client)", direction: "c2s", frame: {
v: 1, type: "edit", id: "01JAB3WAE1B2C3D4E5F6G7H8J9", room: "general", ts: 0,
payload: { target: M1, text: "Hello, Chatterbox!" } } },
{ name: "edit (server)", direction: "s2c", frame: {
v: 1, type: "edit", id: SID("B"), room: "general", ts: T0 + 5000,
payload: { target: M1, text: "Hello, Chatterbox!", seq: 43, editedAt: T0 + 5000 } } },
{ name: "react (client)", direction: "c2s", frame: {
v: 1, type: "react", id: "01JAB3WAF1B2C3D4E5F6G7H8J9", room: "general", ts: 0,
payload: { target: M1, emoji: "\u{1F44D}", op: "add" } } },
{ name: "react (server)", direction: "s2c", frame: {
v: 1, type: "react", id: SID("C"), room: "general", ts: T0 + 6000,
payload: { target: M1, emoji: "\u{1F44D}", op: "add", from: { userId: "bob" }, seq: 44 } } },
{ name: "typing (client)", direction: "c2s", frame: {
v: 1, type: "typing", id: "01JAB3WAG1B2C3D4E5F6G7H8J9", room: "general", ts: 0, payload: { state: "start" } } },
{ name: "typing (server)", direction: "s2c", frame: {
v: 1, type: "typing", id: SID("D"), room: "general", ts: T0 + 7000, payload: { userId: "bob", state: "start" } } },
{ name: "receipt (client)", direction: "c2s", frame: {
v: 1, type: "receipt", id: "01JAB3WAH1B2C3D4E5F6G7H8J9", room: "general", ts: 0, payload: { kind: "read", upTo: 44 } } },
{ name: "receipt (server)", direction: "s2c", frame: {
v: 1, type: "receipt", id: SID("E"), room: "general", ts: T0 + 8000, payload: { userId: "bob", kind: "read", upTo: 44 } } },
{ name: "presence (client)", direction: "c2s", frame: {
v: 1, type: "presence", id: "01JAB3WAJ1B2C3D4E5F6G7H8J9", room: null, ts: 0, payload: { status: "away" } } },
{ name: "presence (server)", direction: "s2c", frame: {
v: 1, type: "presence", id: SID("F"), room: "general", ts: T0 + 9000,
payload: { userId: "alice", name: "Alice", status: "away" } } },
{ name: "history (client)", direction: "c2s", frame: {
v: 1, type: "history", id: "01JAB3WAK1B2C3D4E5F6G7H8J9", room: "general", ts: 0, payload: { before: 42, limit: 2 } } },
{ name: "ack (message)", direction: "s2c", frame: {
v: 1, type: "ack", id: SID("G"), room: "general", ts: T0,
payload: { ref: M1, ok: true, seq: 42, dup: false, result: {} } } },
{ name: "ack (duplicate retry)", direction: "s2c", frame: {
v: 1, type: "ack", id: SID("H"), room: "general", ts: T0 + 12000,
payload: { ref: M1, ok: true, seq: 42, dup: true, result: {} } } },
{ name: "ack (history)", direction: "s2c", frame: {
v: 1, type: "ack", id: SID("J"), room: "general", ts: T0 + 13000,
payload: { ref: "01JAB3WAK1B2C3D4E5F6G7H8J9", ok: true, seq: null, dup: false,
result: { messages: [{ v: 1, type: "message", id: M2, room: "general", ts: T0 - 60000,
payload: { from: { userId: "bob", name: "Bob" }, text: "Morning!", replyTo: null, attachments: [], seq: 40, edited: false } }],
hasMore: true, next: 40 } } } },
{ name: "error", direction: "s2c", frame: {
v: 1, type: "error", id: SID("K"), room: "general", ts: T0 + 14000,
payload: { ref: "01JAB3WAE1B2C3D4E5F6G7H8J9", code: "forbidden", message: "only the author can edit a message", retryable: false } } },
];
/** Frames that every conforming receiver must reject, with the error code the server answers. */
export const invalid: { name: string; text: string; code: string }[] = [
{ name: "not JSON", text: "{nope", code: "bad_request" },
{ name: "unknown version", text: JSON.stringify({ v: 2, type: "message", id: "01JAB3W7Q8Z4N5R6T7V8X9Y0ZA", room: "general", ts: 0, payload: {} }), code: "unsupported_version" },
{ name: "unknown type", text: JSON.stringify({ v: 1, type: "shout", id: "01JAB3W7Q8Z4N5R6T7V8X9Y0ZA", room: "general", ts: 0, payload: {} }), code: "bad_request" },
{ name: "empty text", text: JSON.stringify({ v: 1, type: "message", id: "01JAB3W7Q8Z4N5R6T7V8X9Y0ZA", room: "general", ts: 0, payload: { text: "", replyTo: null, attachments: [] } }), code: "bad_request" },
{ name: "id too short", text: JSON.stringify({ v: 1, type: "leave", id: "abc", room: "general", ts: 0, payload: {} }), code: "bad_request" },
{ name: "room with capitals", text: JSON.stringify({ v: 1, type: "leave", id: "01JAB3W7Q8Z4N5R6T7V8X9Y0ZA", room: "General", ts: 0, payload: {} }), code: "bad_request" },
{ name: "http attachment", text: JSON.stringify({ v: 1, type: "message", id: "01JAB3W7Q8Z4N5R6T7V8X9Y0ZA", room: "general", ts: 0, payload: { text: "x", replyTo: null, attachments: [{ url: "http://insecure.example.test/a.png", name: "a.png", mime: "image/png", size: 1 }] } }), code: "bad_request" },
{ name: "two graphemes in react", text: JSON.stringify({ v: 1, type: "react", id: "01JAB3W7Q8Z4N5R6T7V8X9Y0ZA", room: "general", ts: 0, payload: { target: "01JAB3W7Q8Z4N5R6T7V8X9Y0ZA", emoji: "\u{1F44D}\u{1F44D}", op: "add" } }), code: "bad_request" },
{ name: "null payload", text: JSON.stringify({ v: 1, type: "leave", id: "01JAB3W7Q8Z4N5R6T7V8X9Y0ZA", room: "general", ts: 0, payload: null }), code: "bad_request" },
{ name: "history limit 500", text: JSON.stringify({ v: 1, type: "history", id: "01JAB3W7Q8Z4N5R6T7V8X9Y0ZA", room: "general", ts: 0, payload: { before: null, limit: 500 } }), code: "bad_request" },
{ name: "ack sent by a client", text: JSON.stringify({ v: 1, type: "ack", id: "01JAB3W7Q8Z4N5R6T7V8X9Y0ZA", room: null, ts: 0, payload: {} }), code: "bad_request" },
];
import { ClientFrame, ServerFrame, parseClientFrame } from "./chatterbox-schema.ts";
import { examples, invalid } from "./frames.ts";
let failures = 0;
const check = (ok: boolean, label: string, detail = ""): void => {
console.log(`${ok ? "PASS" : "FAIL"} ${label}${detail ? ` ${detail}` : ""}`);
if (!ok) failures++;
};
const types = new Set<string>();
for (const { name, direction, frame } of examples) {
const schema = direction === "c2s" ? ClientFrame : ServerFrame;
const result = schema.safeParse(frame);
check(result.success, `${direction} ${name}`, result.success ? "" : result.error.issues[0].message);
types.add(String(frame.type));
// a frame must survive a JSON round trip unchanged
check(JSON.stringify(JSON.parse(JSON.stringify(frame))) === JSON.stringify(frame), `${name} round trip`);
}
check(types.size === 12, "all twelve types covered", [...types].sort().join(","));
for (const { name, text, code } of invalid) {
const r = parseClientFrame(text);
check(!r.ok && r.code === code, `reject ${name}`, r.ok ? "accepted!" : `${r.code}: ${r.message}`);
}
// directions are enforced: an ack is never accepted from a client, a hello from a server needs the s_ id prefix
check(!ClientFrame.safeParse(examples.find((e) => e.name === "ack (message)")!.frame).success, "client schema rejects ack");
check(!ServerFrame.safeParse(examples.find((e) => e.name === "join")!.frame).success, "server schema rejects join");
console.log(failures === 0 ? "\nconformance: OK" : `\nconformance: ${failures} failure(s)`);
process.exit(failures === 0 ? 0 : 1);
The same contract in JSON Schema with Ajv
The script below generates JSON Schema from the Zod module, compiles it with Ajv, checks that Ajv accepts all example frames, and that Ajv and Zod agree on every invalid frame except the one that needs a JavaScript refinement.
import { z } from "zod";
import { Ajv2020 } from "ajv/dist/2020.js";
import addFormatsModule from "ajv-formats";
import { ClientFrame, ServerFrame } from "./chatterbox-schema.ts";
import { examples, invalid } from "./frames.ts";
// Zod 4 can emit JSON Schema (draft 2020-12), so one source of truth feeds both validators.
const clientSchema = z.toJSONSchema(ClientFrame, { io: "input" });
const serverSchema = z.toJSONSchema(ServerFrame, { io: "input" });
const addFormats = addFormatsModule as unknown as (a: Ajv2020) => void;
const ajv = new Ajv2020({ allErrors: true, strict: false });
addFormats(ajv);
const validateClient = ajv.compile(clientSchema);
const validateServer = ajv.compile(serverSchema);
let failures = 0;
for (const { name, direction, frame } of examples) {
const ok = (direction === "c2s" ? validateClient : validateServer)(frame);
console.log(`${ok ? "PASS" : "FAIL"} ajv accepts ${direction} ${name}`);
if (!ok) failures++;
}
for (const { name, text } of invalid) {
let parsed: unknown;
try { parsed = JSON.parse(text); } catch { console.log(`SKIP ajv ${name} (not JSON)`); continue; }
const jsonSchemaSays = validateClient(parsed);
const zodSays = ClientFrame.safeParse(parsed).success;
// Known gap: the Intl.Segmenter refinement cannot be expressed in JSON Schema
const expectedGap = name === "two graphemes in react";
const agree = jsonSchemaSays === zodSays || expectedGap;
console.log(`${agree ? "PASS" : "FAIL"} ajv ${jsonSchemaSays ? "accepts" : "rejects"} ${name}${expectedGap ? " (refinement only Zod can check)" : ""}`);
if (!agree) failures++;
}
console.log(`\nGenerated JSON Schema: ${(clientSchema.oneOf ?? clientSchema.anyOf)?.length} client variants, ${(serverSchema.oneOf ?? serverSchema.anyOf)?.length} server variants`);
process.exit(failures === 0 ? 0 : 1);
Source: Zod documentation: JSON Schema
For a hand-written JSON Schema, the two keywords that matter most for protocol schemas are additionalProperties: false
(reject unknown fields, which suits an authentication payload but not an evolving protocol) and a oneOf on a
const discriminator (what Zod generates for a discriminated union). Compile each schema once at startup; Ajv
generates JavaScript for each validator, so compiling per frame is the classic performance bug.
Source: Ajv: Getting started
The reference server
The server below implements all twelve types in memory: handshake with Origin allow-list and subprotocol negotiation,
ticket authentication, hello timeout, rate limiting, size limits, idempotent acknowledgement, per-room seq, history
paging, resume with replay, presence with a grace period, WebSocket-level heartbeats and the close codes of the table.
It uses the ws library (version 8.22.0, the server that Node.js itself does not provide) and runs on Node 24 directly,
because Node strips the TypeScript types by itself. It is deliberately a single process: stacks, scaling and persistence
are the subject of the server pages, which keep this exact wire behaviour.
Project files. The engines field documents the Node version that can run .ts files without a build step, and
erasableSyntaxOnly in tsconfig.json makes the compiler reject the TypeScript features that Node cannot strip.
{
"name": "chatterbox-reference",
"version": "1.0.0",
"private": true,
"type": "module",
"engines": { "node": ">=22.18" },
"scripts": {
"typecheck": "tsc --noEmit",
"conformance": "node conformance.ts",
"ajv": "node ajv-check.ts",
"sizes": "node size-budget.ts",
"edge": "node edge-cases.ts",
"server": "node server.ts",
"client": "node client.ts"
},
"dependencies": {
"@msgpack/msgpack": "3.1.3",
"ajv": "8.20.0",
"ajv-formats": "3.0.1",
"protobufjs": "8.8.1",
"ws": "8.22.0",
"zod": "4.6.7"
},
"devDependencies": {
"@types/node": "24.19.2",
"@types/ws": "8.18.2",
"typescript": "7.0.2"
}
}
{
"compilerOptions": {
"target": "ES2023",
"module": "NodeNext",
"moduleResolution": "NodeNext",
"strict": true,
"noEmit": true,
"allowImportingTsExtensions": true,
"erasableSyntaxOnly": true,
"verbatimModuleSyntax": true,
"skipLibCheck": true,
"types": ["node"]
},
"include": ["*.ts"]
}
The id helpers (a ULID generator, with the s_ prefix for server ids):
import { randomBytes } from "node:crypto";
const CROCKFORD = "0123456789ABCDEFGHJKMNPQRSTVWXYZ";
/** 26-character ULID: 48-bit millisecond timestamp followed by 80 random bits. */
export function ulid(now: number = Date.now()): string {
let time = "";
let t = now;
for (let i = 0; i < 10; i++) {
time = CROCKFORD[t % 32] + time;
t = Math.floor(t / 32);
}
const rnd = randomBytes(10);
let bits = 0n;
for (const byte of rnd) bits = (bits << 8n) | BigInt(byte);
let tail = "";
for (let i = 0; i < 16; i++) {
tail = CROCKFORD[Number(bits & 31n)] + tail;
bits >>= 5n;
}
return time + tail;
}
/** Server-originated envelope ids carry the "s_" prefix so they never collide with client ids. */
export const serverId = (): string => `s_${ulid()}`;
/** Client-generated envelope id; it becomes the message id forever when the envelope is a message. */
export const clientId = (): string => ulid();
Source: ULID specification
The server:
import { createServer, type IncomingMessage, type Server } from "node:http";
import { fileURLToPath } from "node:url";
import { WebSocketServer, WebSocket } from "ws";
import {
ERROR_CODES, MAX_FRAME_BYTES, PROTOCOL_VERSION, ServerFrame, SUBPROTOCOL, parseClientFrame,
type ClientFrameT, type ErrorCode,
} from "./chatterbox-schema.ts";
import { serverId, ulid } from "./ids.ts";
export interface Options {
port?: number;
allowedOrigins?: string[];
heartbeatMs?: number;
helloTimeoutMs?: number;
presenceGraceMs?: number;
sessionKeepMs?: number;
/** Largest server-to-client frame; history pages are shortened to fit. */
maxOutboundBytes?: number;
}
type Envelope = Record<string, any>;
interface Member { userId: string; name: string }
interface Room {
name: string;
seq: number;
members: Set<string>;
log: Envelope[]; // every message/edit/react broadcast, in seq order, for replay
messages: Map<string, Envelope>; // current state of each message envelope, keyed by message id
reactions: Map<string, Set<string>>; // messageId -> "userId|emoji"
}
interface Session {
id: string; userId: string; rooms: Set<string>; status: "online" | "away" | "offline";
conns: Set<Conn>; detachedAt: number | null; offlineTimer?: NodeJS.Timeout;
acks: Map<string, { payload: Envelope; room: string | null; at: number }>; // idempotency table
}
interface Conn {
ws: WebSocket; session: Session | null; lastSeen: number; tokens: number; refilledAt: number; limited: number[];
helloTimer?: NodeJS.Timeout;
}
const ack24h = 24 * 60 * 60 * 1000;
export function startServer(opts: Options = {}) {
const o = {
port: 18080,
allowedOrigins: ["https://chat.example.test"],
heartbeatMs: 20000,
helloTimeoutMs: 5000,
presenceGraceMs: 15000,
sessionKeepMs: 120000,
maxOutboundBytes: 262144,
...opts,
};
const tickets = new Map<string, { userId: string; expires: number }>(); // one-time tickets
const users = new Map<string, Member>(); // userId -> identity
const sessions = new Map<string, Session>(); // sessionId -> session
const byUser = new Map<string, Session>(); // userId -> live or detached session
const rooms = new Map<string, Room>();
const conns = new Set<Conn>();
// ------------------------------------------------------------ http: login stand-in and upgrade
const http: Server = createServer((req, res) => {
const url = new URL(req.url ?? "/", "http://localhost");
if (req.method === "POST" && url.pathname === "/ticket") {
// Stand-in for "the login system": a real server mints this behind the session cookie.
const userId = (url.searchParams.get("user") ?? "").toLowerCase();
if (!/^[a-z0-9]{1,32}$/.test(userId)) { res.writeHead(400).end(); return; }
res.writeHead(200, { "content-type": "application/json" }).end(JSON.stringify({ ticket: issueTicket(userId) }));
return;
}
res.writeHead(404).end();
});
const wss = new WebSocketServer({
noServer: true,
maxPayload: 65536, // transport guard; the 16384-byte protocol limit is enforced below with close code 4413
handleProtocols: (protocols) => (protocols.has(SUBPROTOCOL) ? SUBPROTOCOL : false),
});
http.on("upgrade", (req: IncomingMessage, socket, head) => {
const origin = req.headers.origin; // browsers always send it; non-browser clients may omit it
if (origin !== undefined && !o.allowedOrigins.includes(origin)) {
socket.write("HTTP/1.1 403 Forbidden\r\nConnection: close\r\n\r\n");
socket.destroy();
return;
}
wss.handleUpgrade(req, socket, head, (ws) => onConnection(ws));
});
function issueTicket(userId: string): string {
const t = `tk_${ulid()}`;
tickets.set(t, { userId, expires: Date.now() + 30000 });
return t;
}
// ------------------------------------------------------------ sending
const sendRaw = (ws: WebSocket, env: Envelope): void => {
const check = ServerFrame.safeParse(env); // validate outbound too: a server bug must not reach the wire
if (!check.success) throw new Error(`invalid outbound frame: ${check.error.issues[0].message}`);
if (ws.readyState === WebSocket.OPEN) ws.send(JSON.stringify(env));
};
const envelope = (type: string, room: string | null, payload: Envelope, id = serverId()): Envelope =>
({ v: PROTOCOL_VERSION, type, id, room, ts: Date.now(), payload });
const sendToSession = (s: Session, env: Envelope): void => s.conns.forEach((c) => sendRaw(c.ws, env));
const sendError = (c: Conn, ref: string | null, code: ErrorCode, message: string, room: string | null = null): void =>
sendRaw(c.ws, envelope("error", room, { ref, code, message, retryable: ERROR_CODES[code].retryable }));
const closeWith = (c: Conn, code: number, reason: string): void => {
c.ws.close(code, reason);
setTimeout(() => c.ws.terminate(), 1000).unref(); // do not wait for a dead peer to answer the close frame
};
const broadcast = (room: Room, env: Envelope, except?: string): void => {
for (const userId of room.members) {
if (userId === except) continue;
const s = byUser.get(userId);
if (s) sendToSession(s, env);
}
};
const roomOf = (name: string): Room => {
let r = rooms.get(name);
if (!r) rooms.set(name, (r = { name, seq: 0, members: new Set(), log: [], messages: new Map(), reactions: new Map() }));
return r;
};
const who = (userId: string): Member => users.get(userId)!;
// ------------------------------------------------------------ connection lifecycle
function onConnection(ws: WebSocket): void {
const c: Conn = { ws, session: null, lastSeen: Date.now(), tokens: 40, refilledAt: Date.now(), limited: [] };
conns.add(c);
c.helloTimer = setTimeout(() => { if (!c.session) closeWith(c, 4408, "hello timeout"); }, o.helloTimeoutMs);
ws.on("pong", () => { c.lastSeen = Date.now(); });
ws.on("message", (data, isBinary) => {
c.lastSeen = Date.now();
if (isBinary) return closeWith(c, 4400, "binary frames are not part of chatterbox.v1");
const text = data.toString("utf8");
if (Buffer.byteLength(text) > MAX_FRAME_BYTES) {
sendError(c, null, "too_large", `frames are limited to ${MAX_FRAME_BYTES} bytes`);
return closeWith(c, 4413, "frame too large");
}
onFrame(c, text);
});
ws.on("close", () => onClose(c));
ws.on("error", () => {});
}
function onClose(c: Conn): void {
conns.delete(c);
clearTimeout(c.helloTimer);
const s = c.session;
if (!s) return;
s.conns.delete(c);
if (s.conns.size > 0) return;
s.detachedAt = Date.now();
// After the grace period without a reconnect the user becomes offline for everyone in their rooms.
s.offlineTimer = setTimeout(() => { if (s.conns.size === 0) setStatus(s, "offline"); }, o.presenceGraceMs);
setTimeout(() => {
if (s.conns.size === 0 && s.detachedAt && Date.now() - s.detachedAt >= o.sessionKeepMs) {
sessions.delete(s.id);
if (byUser.get(s.userId) === s) byUser.delete(s.userId);
}
}, o.sessionKeepMs + 50).unref();
}
function setStatus(s: Session, status: Session["status"]): void {
if (s.status === status) return;
s.status = status;
for (const name of s.rooms) {
const r = rooms.get(name)!;
broadcast(r, envelope("presence", name, { userId: s.userId, name: who(s.userId).name, status }), s.userId);
}
}
// ------------------------------------------------------------ rate limit: 20 envelopes/s, burst 40
function allow(c: Conn): boolean {
const now = Date.now();
c.tokens = Math.min(40, c.tokens + ((now - c.refilledAt) / 1000) * 20);
c.refilledAt = now;
if (c.tokens >= 1) { c.tokens -= 1; return true; }
c.limited = c.limited.filter((t) => now - t < 10000).concat(now);
return false;
}
// ------------------------------------------------------------ frame dispatch
function onFrame(c: Conn, text: string): void {
if (!allow(c)) {
sendError(c, null, "rate_limited", "slow down: 20 envelopes per second, burst 40");
if (c.limited.length >= 10) closeWith(c, 4429, "rate limited");
return;
}
const parsed = parseClientFrame(text);
if (!parsed.ok) {
sendError(c, parsed.ref, parsed.code, parsed.message);
if (parsed.code === "unsupported_version") closeWith(c, 4426, "unsupported version");
return;
}
const f = parsed.frame;
if (f.type === "hello") return onHello(c, f);
const s = c.session;
if (!s) {
sendError(c, f.id, "unauthorized", "send hello first");
return closeWith(c, 4401, "hello required");
}
// Idempotency: a repeated id gets the ORIGINAL ack again, with dup:true, and has no other effect.
const prior = s.acks.get(f.id);
if (prior) {
sendRaw(c.ws, envelope("ack", prior.room, { ...prior.payload, dup: true }));
return;
}
try {
handlers[f.type](c, s, f as never);
} catch (e) {
console.error(e);
sendError(c, f.id, "internal", "unexpected server error", f.room);
}
}
function reply(c: Conn, s: Session, f: ClientFrameT, seq: number | null, result: Envelope = {}): void {
const payload = { ref: f.id, ok: true, seq, dup: false, result };
s.acks.set(f.id, { payload, room: f.room, at: Date.now() });
if (s.acks.size > 1000) { // keep at least the last 1000 ids, drop anything older than 24 hours
for (const [id, a] of s.acks) { if (s.acks.size <= 1000 || Date.now() - a.at < ack24h) break; s.acks.delete(id); }
}
sendRaw(c.ws, envelope("ack", f.room, payload));
}
function onHello(c: Conn, f: Extract<ClientFrameT, { type: "hello" }>): void {
if (c.session) return sendError(c, f.id, "conflict", "hello already completed");
const token = f.payload.token ?? ""; // this reference server accepts tickets only, not cookies
const ticket = tickets.get(token);
tickets.delete(token); // one-time
if (!ticket || ticket.expires < Date.now()) {
sendError(c, f.id, "unauthorized", "invalid or expired ticket");
return closeWith(c, 4401, "unauthorized");
}
clearTimeout(c.helloTimer);
const userId = ticket.userId;
users.set(userId, { userId, name: userId[0].toUpperCase() + userId.slice(1) });
const resume = f.payload.resume;
let session = resume && sessions.get(resume.sessionId);
let resumed = false;
if (session && session.userId === userId && Date.now() - (session.detachedAt ?? Date.now()) < o.sessionKeepMs) {
resumed = true;
clearTimeout(session.offlineTimer);
session.detachedAt = null;
} else {
session = { id: `sess_${ulid()}`, userId, rooms: new Set(), status: "online", conns: new Set(), detachedAt: null, acks: new Map() };
sessions.set(session.id, session);
byUser.set(userId, session);
}
session.conns.add(c);
c.session = session;
sendRaw(c.ws, envelope("hello", null, {
ref: f.id, userId, name: who(userId).name, sessionId: session.id, heartbeatMs: o.heartbeatMs,
maxFrameBytes: MAX_FRAME_BYTES, resumed, rooms: [...session.rooms],
}));
if (resumed) {
for (const [name, last] of Object.entries(resume!.rooms)) if (session.rooms.has(name)) replay(c, rooms.get(name)!, last);
setStatus(session, "online");
}
}
const replay = (c: Conn, room: Room, after: number): void => {
for (const env of room.log) if (env.payload.seq > after) sendRaw(c.ws, env);
};
// ------------------------------------------------------------ per-type handlers
type H<T extends ClientFrameT["type"]> = (c: Conn, s: Session, f: Extract<ClientFrameT, { type: T }>) => void;
const member = (c: Conn, s: Session, f: ClientFrameT): Room | null => {
const r = f.room ? rooms.get(f.room) : undefined;
if (!r || !r.members.has(s.userId)) { sendError(c, f.id, "forbidden", "join the room first", f.room); return null; }
return r;
};
const dmAllowed = (room: string, userId: string): boolean => !room.startsWith("dm-") || room.split("-").slice(1).includes(userId);
const handlers: { [K in ClientFrameT["type"]]: H<K> } = {
hello: () => {},
join(c, s, f) {
if (!dmAllowed(f.room, s.userId)) return sendError(c, f.id, "forbidden", "not your direct-message room", f.room);
const r = roomOf(f.room);
const wasMember = r.members.has(s.userId);
r.members.add(s.userId);
s.rooms.add(r.name);
const members = [...r.members].map((id) => ({ userId: id, name: who(id).name, status: byUser.get(id)?.status ?? "offline" }));
reply(c, s, f, null, { members, lastSeq: r.seq });
if (f.payload.since !== null) replay(c, r, f.payload.since);
if (!wasMember) broadcast(r, envelope("presence", r.name, { userId: s.userId, name: who(s.userId).name, status: s.status }), s.userId);
},
leave(c, s, f) {
const r = rooms.get(f.room);
if (!r || !r.members.delete(s.userId)) return sendError(c, f.id, "not_found", "not a member of that room", f.room);
s.rooms.delete(r.name);
reply(c, s, f, null);
},
message(c, s, f) {
const r = member(c, s, f); if (!r) return;
const { text, replyTo, attachments } = f.payload;
if (replyTo && !r.messages.has(replyTo)) return sendError(c, f.id, "not_found", "replyTo does not exist in this room", r.name);
if (r.messages.has(f.id)) return sendError(c, f.id, "conflict", "message id already used", r.name);
const seq = ++r.seq;
const env = envelope("message", r.name, {
from: { userId: s.userId, name: who(s.userId).name }, text, replyTo, attachments, seq, edited: false,
}, f.id); // the message id IS the client's envelope id
r.messages.set(f.id, env);
r.log.push(env);
reply(c, s, f, seq);
broadcast(r, env);
},
edit(c, s, f) {
const r = member(c, s, f); if (!r) return;
const target = r.messages.get(f.payload.target);
if (!target) return sendError(c, f.id, "not_found", "no such message", r.name);
if (target.payload.from.userId !== s.userId) return sendError(c, f.id, "forbidden", "only the author can edit a message", r.name);
const seq = ++r.seq;
const editedAt = Date.now();
target.payload.text = f.payload.text;
target.payload.edited = true;
const env = envelope("edit", r.name, { target: f.payload.target, text: f.payload.text, seq, editedAt });
r.log.push(env);
reply(c, s, f, seq);
broadcast(r, env);
},
react(c, s, f) {
const r = member(c, s, f); if (!r) return;
const { target, emoji, op } = f.payload;
if (!r.messages.has(target)) return sendError(c, f.id, "not_found", "no such message", r.name);
const set = r.reactions.get(target) ?? new Set<string>();
const key = `${s.userId}|${emoji}`;
if (op === "add" && set.has(key)) return sendError(c, f.id, "conflict", "reaction already added", r.name);
if (op === "remove" && !set.has(key)) return sendError(c, f.id, "not_found", "reaction not present", r.name);
op === "add" ? set.add(key) : set.delete(key);
r.reactions.set(target, set);
const seq = ++r.seq;
const env = envelope("react", r.name, { target, emoji, op, from: { userId: s.userId }, seq });
r.log.push(env);
reply(c, s, f, seq);
broadcast(r, env);
},
typing(c, s, f) { // fire-and-forget: no ack, only an error when the sender is not a member
const r = member(c, s, f); if (!r) return;
broadcast(r, envelope("typing", r.name, { userId: s.userId, state: f.payload.state }), s.userId);
},
receipt(c, s, f) { // fire-and-forget and cumulative
const r = member(c, s, f); if (!r) return;
if (f.payload.upTo > r.seq) return sendError(c, f.id, "bad_request", "upTo is beyond the last seq", r.name);
broadcast(r, envelope("receipt", r.name, { userId: s.userId, kind: f.payload.kind, upTo: f.payload.upTo }), s.userId);
},
presence(c, s, f) { // doubles as the application heartbeat; the ack lets the client measure RTT
reply(c, s, f, null);
setStatus(s, f.payload.status);
},
history(c, s, f) {
const r = member(c, s, f); if (!r) return;
const { before, limit } = f.payload;
const all = [...r.messages.values()].filter((m) => before === null || m.payload.seq < before);
let page = all.slice(-limit);
const size = (p: Envelope[]) => Buffer.byteLength(JSON.stringify(p));
while (page.length > 1 && size(page) > o.maxOutboundBytes - 1024) page = page.slice(1); // keep the newest messages that fit
const hasMore = page.length < all.length;
reply(c, s, f, null, { messages: page, hasMore, next: hasMore ? page[0].payload.seq : null });
},
};
// ------------------------------------------------------------ heartbeat: ping every heartbeatMs, close after 2x silence
const timer = setInterval(() => {
const now = Date.now();
for (const c of conns) {
if (now - c.lastSeen > 2 * o.heartbeatMs) { closeWith(c, 4408, "heartbeat timeout"); continue; }
if (c.ws.readyState === WebSocket.OPEN) c.ws.ping();
}
}, Math.max(10, o.heartbeatMs / 2));
timer.unref();
return new Promise<{ port: number; issueTicket: typeof issueTicket; close: (code?: number) => Promise<void> }>((resolve) => {
http.listen(o.port, "127.0.0.1", () => {
const port = (http.address() as { port: number }).port;
resolve({
port,
issueTicket,
close: (code = 4503) => new Promise((done) => {
clearInterval(timer);
for (const c of conns) c.ws.close(code, "server restarting");
setTimeout(() => { for (const c of conns) c.ws.terminate(); http.close(() => done()); http.closeAllConnections(); }, 100);
}),
});
});
});
}
if (process.argv[1] === fileURLToPath(import.meta.url)) {
const srv = await startServer({ port: Number(process.env.PORT ?? 18080) });
console.log(`chatterbox reference server on ws://127.0.0.1:${srv.port}/ (POST /ticket?user=alice)`);
process.on("SIGTERM", () => void srv.close().then(() => process.exit(0)));
process.on("SIGINT", () => void srv.close().then(() => process.exit(0)));
}
Points worth noticing in the listing:
-
The Origin check happens in the
upgradehandler, beforehandleUpgrade. Browsers apply no same-origin policy to WebSocket handshakes and attach cookies, so a page on any site can open a socket to your server as the logged-in user (Cross-Site WebSocket Hijacking) unless the server comparesOriginwith an allow-list. Requests without anOrigin(not a browser) are admitted here because they must still present a one-time ticket inhello. -
Authentication is in the first frame, not the URL. The browser
WebSocketconstructor cannot set headers. A one-time ticket in the first message never lands in access logs orRefererheaders the way a query-string token does, expires in 30 seconds and cannot be replayed; the 5-secondhellodeadline keeps unauthenticated sockets cheap. -
reply()is the idempotency table, and the early return at the top ofonFrame()is the deduplication. -
sendRaw()validates outbound frames with the same Zod schema the client uses. -
closeWith()terminates after 1 second, because a peer that never answers the close frame would otherwise hold the socket for the library’s close timeout.
The client and the scenario script
The client class below validates inbound frames, correlates acks with promises, keeps unacknowledged envelopes in
order, resends them with the same ids after a reconnect, and resumes with lastSeq per room. It uses the global
WebSocket of Node 22 and later, which is the same API a browser has, so the class runs in a page unchanged apart from
the ticket fetch. The scenario at the bottom exercises everything and prints PASS or FAIL for 21 checks, including the
duplicate-id retry three ways.
import { fileURLToPath } from "node:url";
import { ServerFrame, SUBPROTOCOL, type ServerFrameT } from "./chatterbox-schema.ts";
import { clientId } from "./ids.ts";
type Listener = (frame: ServerFrameT) => void;
interface Pending { frame: Record<string, unknown>; resolve: (f: ServerFrameT) => void; timer?: ReturnType<typeof setTimeout> }
/** Minimal Chatterbox client: validates every inbound frame, correlates acks and retries with the same id. */
export class ChatterboxClient {
ws: WebSocket | null = null;
sessionId: string | null = null;
lastSeq: Record<string, number> = {};
readonly pending = new Map<string, Pending>(); // insertion order = original send order
readonly sent = new Map<string, Record<string, unknown>>(); // every request frame, kept so tests can replay it
readonly seen: ServerFrameT[] = [];
private listeners = new Set<Listener>();
private base: string;
private userId: string;
private ackTimeoutMs: number;
constructor(base: string, userId: string, ackTimeoutMs = 10000) {
this.base = base;
this.userId = userId;
this.ackTimeoutMs = ackTimeoutMs;
}
on(l: Listener): () => void { this.listeners.add(l); return () => this.listeners.delete(l); }
waitFor(pred: (f: ServerFrameT) => boolean, ms = 3000): Promise<ServerFrameT> {
return new Promise((resolve, reject) => {
const t = setTimeout(() => { off(); reject(new Error("waitFor timed out")); }, ms);
const off = this.on((f) => { if (pred(f)) { clearTimeout(t); off(); resolve(f); } });
});
}
async connect(): Promise<ServerFrameT & { type: "hello" }> {
const res = await fetch(`http://${this.base}/ticket?user=${this.userId}`, { method: "POST" });
const { ticket } = (await res.json()) as { ticket: string };
const ws = new WebSocket(`ws://${this.base}/`, SUBPROTOCOL); // browser-compatible global client, Node >= 22
this.ws = ws;
ws.onmessage = (ev) => this.onMessage(String(ev.data));
await new Promise<void>((ok, bad) => { ws.onopen = () => ok(); ws.onerror = () => bad(new Error("connect failed")); });
const resume = this.sessionId ? { sessionId: this.sessionId, rooms: this.lastSeq } : undefined;
const hello = this.waitFor((f) => f.type === "hello");
this.write({ v: 1, type: "hello", id: clientId(), room: null, ts: 0, payload: { token: ticket, client: "node/1.0", ...(resume && { resume }) } });
const h = (await hello) as ServerFrameT & { type: "hello" };
this.sessionId = h.payload.sessionId;
for (const p of this.pending.values()) this.write(p.frame); // retry unacked envelopes, same ids, original order
return h;
}
/** Send an envelope that expects an ack or an error and resolve with whichever arrives. */
request(type: string, room: string | null, payload: Record<string, unknown>, id = clientId()): Promise<ServerFrameT> {
const frame = { v: 1, type, id, room, ts: 0, payload };
this.sent.set(id, frame);
return new Promise((resolve) => {
const p: Pending = { frame, resolve };
this.pending.set(id, p);
p.timer = setTimeout(() => { if (this.ws?.readyState === WebSocket.OPEN) this.write(frame); }, this.ackTimeoutMs);
if (this.ws?.readyState === WebSocket.OPEN) this.write(frame);
});
}
/** Fire-and-forget envelopes (typing, receipt). */
notify(type: string, room: string, payload: Record<string, unknown>): void {
this.write({ v: 1, type, id: clientId(), room, ts: 0, payload });
}
/** Re-send a frame that was already sent, byte for byte, as a lost-ack retry would. */
resend(id: string): void { this.write(this.sent.get(id)!); }
private write(frame: Record<string, unknown>): void { this.ws?.send(JSON.stringify(frame)); }
private onMessage(text: string): void {
const parsed = ServerFrame.safeParse(JSON.parse(text));
if (!parsed.success) { console.error("dropping invalid server frame:", parsed.error.issues[0]); return; }
const f = parsed.data;
this.seen.push(f);
if (f.type === "message" || f.type === "edit" || f.type === "react") {
this.lastSeq[f.room] = Math.max(this.lastSeq[f.room] ?? 0, f.payload.seq);
}
if (f.type === "ack" || f.type === "error") {
const ref = f.payload.ref;
const p = ref && this.pending.get(ref);
if (p && (f.type === "ack" || !f.payload.retryable)) {
clearTimeout(p.timer);
this.pending.delete(ref);
p.resolve(f);
}
}
this.listeners.forEach((l) => l(f));
}
close(): void { this.ws?.close(1000, "bye"); }
drop(): void { this.ws?.close(1000, "simulated network loss"); }
}
// ---------------------------------------------------------------- demo scenario
if (process.argv[1] === fileURLToPath(import.meta.url)) {
const base = process.env.CHATTERBOX ?? "127.0.0.1:18080";
let failed = 0;
const check = (ok: boolean, label: string): void => { console.log(`${ok ? "PASS" : "FAIL"} ${label}`); if (!ok) failed++; };
const ackOf = (f: ServerFrameT) => (f.type === "ack" ? f.payload : null);
const sleep = (ms: number) => new Promise((r) => setTimeout(r, ms));
const alice = new ChatterboxClient(base, "alice");
const bob = new ChatterboxClient(base, "bob");
const h1 = await alice.connect();
await bob.connect();
check(h1.payload.userId === "alice" && !h1.payload.resumed, "hello: alice authenticated with a one-time ticket");
check(ackOf(await alice.request("join", "general", { since: null }))?.ok === true, "join general (alice)");
const bobJoin = ackOf(await bob.request("join", "general", { since: null }));
check((bobJoin?.result.members as unknown[]).length === 2, "join general (bob) lists two members");
// 1. a normal message: ack with the assigned seq, broadcast to both members
const id = clientId();
const bobSees = bob.waitFor((f) => f.type === "message" && f.id === id);
const first = ackOf(await alice.request("message", "general", { text: "Hello, Chatterbox", replyTo: null, attachments: [] }, id));
check(first?.ok === true && first.dup === false && typeof first.seq === "number", `message acked with seq ${first?.seq}`);
await bobSees;
// 2. lost-ack retry: the SAME id is sent again, the server answers with the original ack and dup:true
const retry = alice.waitFor((f) => f.type === "ack" && f.payload.ref === id && f.payload.dup);
alice.resend(id);
const dup = ackOf(await retry);
await sleep(200); // give a (wrong) second broadcast time to arrive
check(dup?.dup === true && dup.seq === first?.seq, "duplicate id: same seq, dup:true");
check(bob.seen.filter((f) => f.type === "message" && f.id === id).length === 1, "duplicate id: not broadcast twice");
// 3. network loss: alice drops, bob keeps talking, alice resumes and gets the missed envelope with its original seq
alice.drop();
await sleep(100);
const missedId = clientId();
ackOf(await bob.request("message", "general", { text: "You missed this", replyTo: null, attachments: [] }, missedId));
const again = alice.waitFor((f) => f.type === "message" && f.id === missedId);
const h2 = await alice.connect();
check(h2.payload.resumed === true && h2.payload.sessionId === h1.payload.sessionId, "resume: same session, resumed:true");
const replayed = await again;
check(replayed.type === "message" && replayed.payload.seq === (first!.seq! + 1), "resume: missed message replayed with original seq");
const dupAfterReconnect = ackOf(await alice.request("message", "general", { text: "Hello, Chatterbox", replyTo: null, attachments: [] }, id));
check(dupAfterReconnect?.dup === true && dupAfterReconnect.seq === first?.seq, "retry after reconnect: still deduplicated");
// 4. edit, react, typing, receipt, presence
const edit = ackOf(await alice.request("edit", "general", { target: id, text: "Hello, Chatterbox!" }));
check(edit?.ok === true && edit.seq! > first!.seq!, "edit acked with a newer seq");
const forbidden = await bob.request("edit", "general", { target: id, text: "hijack" });
check(forbidden.type === "error" && forbidden.payload.code === "forbidden", "edit by a non-author -> forbidden");
const react = ackOf(await bob.request("react", "general", { target: id, emoji: "\u{1F44D}", op: "add" }));
check(react?.ok === true, "react add acked");
const conflict = await bob.request("react", "general", { target: id, emoji: "\u{1F44D}", op: "add" });
check(conflict.type === "error" && conflict.payload.code === "conflict", "react add twice -> conflict");
const typing = alice.waitFor((f) => f.type === "typing" && f.payload.userId === "bob");
bob.notify("typing", "general", { state: "start" });
await typing;
check(true, "typing broadcast reaches the other member");
const receipt = alice.waitFor((f) => f.type === "receipt");
bob.notify("receipt", "general", { kind: "read", upTo: react!.seq! });
await receipt;
check(true, "receipt broadcast reaches the other member");
const away = bob.waitFor((f) => f.type === "presence" && f.payload.status === "away");
void alice.request("presence", null, { status: "away" });
await away;
check(true, "presence change broadcast to room members");
// 5. history paging backwards by seq
for (let i = 0; i < 4; i++) await alice.request("message", "general", { text: `filler ${i}`, replyTo: null, attachments: [] });
const page1 = ackOf(await alice.request("history", "general", { before: null, limit: 3 }))!;
const r1 = page1.result as { messages: { payload: { seq: number } }[]; hasMore: boolean; next: number | null };
check(r1.messages.length === 3 && r1.hasMore && r1.next !== null, "history page 1: 3 messages, hasMore");
const page2 = ackOf(await alice.request("history", "general", { before: r1.next, limit: 3 }))!;
const r2 = page2.result as { messages: { payload: { seq: number } }[] };
check(r2.messages.length >= 1 && r2.messages.at(-1)!.payload.seq < r1.next!, "history page 2 continues before next");
// 6. error paths
const notMember = await alice.request("message", "secret-room", { text: "hi", replyTo: null, attachments: [] });
check(notMember.type === "error" && notMember.payload.code === "forbidden", "message in a room you have not joined -> forbidden");
const dm = await alice.request("join", "dm-bob-carol", { since: null });
check(dm.type === "error" && dm.payload.code === "forbidden", "joining someone else's direct-message room -> forbidden");
const leave = ackOf(await alice.request("leave", "general", {}));
check(leave?.ok === true, "leave general");
alice.close(); bob.close();
await sleep(100);
console.log(failed === 0 ? "\nclient scenario: OK" : `\nclient scenario: ${failed} failure(s)`);
process.exit(failed === 0 ? 0 : 1);
}
Source: MDN: WebSocket
Edge cases: handshake, limits and close codes
edge-cases.ts starts the server in the same process with short timeouts and drives it with raw ws sockets to
prove each limit and close code of the specification.
import { WebSocket } from "ws";
import { startServer } from "./server.ts";
import { clientId } from "./ids.ts";
let failed = 0;
const check = (ok: boolean, label: string): void => { console.log(`${ok ? "PASS" : "FAIL"} ${label}`); if (!ok) failed++; };
const srv = await startServer({ port: 0, helloTimeoutMs: 300, heartbeatMs: 200, presenceGraceMs: 300 });
const base = `127.0.0.1:${srv.port}`;
/** Open a raw socket, collect frames and the close code. */
async function open(origin?: string, autoPong = true) {
const ws = new WebSocket(`ws://${base}/`, "chatterbox.v1", { ...(origin && { origin }), autoPong });
const frames: any[] = [];
const closed = new Promise<number>((r) => ws.on("close", (code) => r(code)));
ws.on("message", (d) => frames.push(JSON.parse(String(d))));
await new Promise<void>((ok, bad) => { ws.on("open", () => ok()); ws.on("error", bad); });
return { ws, frames, closed };
}
const sleep = (ms: number) => new Promise((r) => setTimeout(r, ms));
const hello = async (user: string) => {
const { ticket } = await (await fetch(`http://${base}/ticket?user=${user}`, { method: "POST" })).json() as { ticket: string };
return JSON.stringify({ v: 1, type: "hello", id: clientId(), room: null, ts: 0, payload: { token: ticket, client: "edge/1" } });
};
// 1. Origin allow-list: a browser from another site is refused during the handshake
const bad = new WebSocket(`ws://${base}/`, "chatterbox.v1", { origin: "https://evil.example.test" });
const badResult = await new Promise<string>((r) => { bad.on("unexpected-response", (_q, res) => r(String(res.statusCode))); bad.on("open", () => r("open")); bad.on("error", () => r("error")); });
check(badResult === "403", "Origin not on the allow-list -> HTTP 403 handshake");
const good = await open("https://chat.example.test");
check(good.ws.protocol === "chatterbox.v1", "allowed Origin accepted, subprotocol chatterbox.v1 negotiated");
good.ws.close();
// 2. no hello within the deadline -> 4408
const silent = await open();
check((await silent.closed) === 4408, "no hello within helloTimeoutMs -> close 4408");
// 3. first frame is not hello -> unauthorized + 4401
const nohello = await open();
nohello.ws.send(JSON.stringify({ v: 1, type: "join", id: clientId(), room: "general", ts: 0, payload: { since: null } }));
check((await nohello.closed) === 4401 && nohello.frames[0].payload.code === "unauthorized", "frame before hello -> error unauthorized, close 4401");
// 4. unknown version -> error then 4426
const v2 = await open();
v2.ws.send(JSON.stringify({ v: 2, type: "hello", id: clientId(), room: null, ts: 0, payload: {} }));
check((await v2.closed) === 4426 && v2.frames[0].payload.code === "unsupported_version", "v:2 -> error unsupported_version, close 4426");
// 5. one-time ticket cannot be replayed
const t1 = await open();
const h = await hello("alice");
t1.ws.send(h);
await sleep(100);
const t2 = await open();
t2.ws.send(h);
check((await t2.closed) === 4401, "replayed ticket -> close 4401");
// 6. oversized frame -> too_large then 4413
const big = await open();
big.ws.send(await hello("bob"));
await sleep(100);
big.ws.send(JSON.stringify({ v: 1, type: "message", id: clientId(), room: "general", ts: 0, payload: { text: "x".repeat(20000), replyTo: null, attachments: [] } }));
check((await big.closed) === 4413 && big.frames.some((f) => f.payload.code === "too_large"), "frame over 16384 bytes -> too_large, close 4413");
// 7. malformed payload -> bad_request carrying the ref, connection stays open
const mal = await open();
mal.ws.send(await hello("carol"));
await sleep(100);
const badId = clientId();
mal.ws.send(JSON.stringify({ v: 1, type: "join", id: badId, room: "NOT VALID", ts: 0, payload: { since: null } }));
await sleep(100);
const err = mal.frames.find((f) => f.type === "error");
check(err?.payload.code === "bad_request" && err.payload.ref === badId && mal.ws.readyState === WebSocket.OPEN, "invalid room -> bad_request with ref, socket stays open");
// 8. rate limit: burst of 40 passes, the rest is rate_limited, persistent abuse closes with 4429
const rl = await open();
rl.ws.send(await hello("dave"));
await sleep(100);
for (let i = 0; i < 120; i++) rl.ws.send(JSON.stringify({ v: 1, type: "presence", id: clientId(), room: null, ts: 0, payload: { status: "online" } }));
const code = await rl.closed;
check(rl.frames.some((f) => f.payload?.code === "rate_limited" && f.payload.retryable === true) && code === 4429, "burst of 120 envelopes -> rate_limited (retryable), then close 4429");
// 9. silence for 2 x heartbeatMs: a peer that never answers pings is closed with 4408
const dead = await open(undefined, false); // autoPong:false -> never answers the server's pings
dead.ws.send(await hello("erin"));
check((await dead.closed) === 4408, "no pong and no frames for 2 x heartbeatMs -> close 4408");
// 10. presence goes offline after the grace period
const a = await open();
a.ws.send(await hello("fred"));
const b = await open();
b.ws.send(await hello("gina"));
await sleep(100);
for (const [w, room] of [[a.ws, "lobby"], [b.ws, "lobby"]] as const) w.send(JSON.stringify({ v: 1, type: "join", id: clientId(), room, ts: 0, payload: { since: null } }));
await sleep(100);
a.ws.close();
await sleep(600);
check(b.frames.some((f) => f.type === "presence" && f.payload.userId === "fred" && f.payload.status === "offline"), "closed connection -> presence offline after the grace period");
b.ws.close();
// 11. graceful shutdown tells clients to reconnect: 4503
const live = await open();
live.ws.send(await hello("hank"));
await sleep(100);
const shutdown = srv.close();
check((await live.closed) === 4503, "server restart -> close 4503");
await shutdown;
console.log(failed === 0 ? "\nedge cases: OK" : `\nedge cases: ${failed} failure(s)`);
process.exit(failed === 0 ? 0 : 1);
Running everything
mkdir chatterbox-reference && cd chatterbox-reference
# save the files above, then:
npm install
npm run typecheck # tsc --noEmit, no output means success
npm run conformance # every example frame validates; every invalid frame is rejected
npm run ajv # the generated JSON Schema agrees with Zod
npm run sizes # prints the table of the previous section
node lamport.ts # prints the Lamport ordering
npm run edge # handshake, limits, close codes (starts its own server)
PORT=18080 npm run server & # the reference server; POST /ticket?user=alice mints a demo ticket
CHATTERBOX=127.0.0.1:18080 npm run client
Excerpt of the output of the last command (21 checks in total):
PASS duplicate id: same seq, dup:true
PASS duplicate id: not broadcast twice
PASS resume: same session, resumed:true
PASS resume: missed message replayed with original seq
PASS retry after reconnect: still deduplicated
...
PASS leave general
client scenario: OK
Security considerations
-
Treat every frame as hostile input. Check the byte length before parsing, validate the envelope and payload with a schema, and cap strings, arrays, and depth. Deeply nested JSON and huge arrays are cheap to send and costly to parse.
JSON.parseitself is safe with a__proto__key (it creates an ordinary own property), but merging parsed objects into others with a recursive merge helper can pollute prototypes: copy only the fields the schema returned. -
Authorise on every frame, not once at connect. Membership, message authorship and DM ownership are checked per request (see
member()anddmAllowed()), because permissions change while sockets stay open (see Authentication and Authorization). -
Never trust
from,userIdortssent by a client. The server derives identity from the session and stamps time itself; the broadcastfromcomes from the session, never from the payload. -
Escape on render.
textis user content. Insert it withtextContent, neverinnerHTML, and never interpolate it into markup, a URL or a log line without encoding. Attachments arehttpsURLs only, validated by the schema, and should be loaded from a separate origin with a content-security policy. -
Limit everything that scales with the attacker: frame size, frame rate, rooms per user, members per room, history page size, pending resend queue. The protocol advertises the first three so the client can behave.
-
Do not log payloads or tickets. Log type, id, room, size and outcome. Tickets in the first frame stay out of access logs, which is their point.
-
Idempotency tables are state an attacker can fill. Bound them per user (this server keeps the last 1000 ids at least) and expire them.
-
Compression and secrets. If you enable
permessage-deflate, do not put secrets in frames that also contain attacker text; see Security.
Common errors and pitfalls
-
Regenerating the id on retry. The retry then creates a second message. Generate the id once, store it with the queued envelope.
-
Treating the ack as delivery. It means "stored". A user who sees a double tick because of an ack is misled.
-
Assuming ack and broadcast arrive in a fixed order. The sender may see its own broadcast before the ack. Match by id.
-
Ordering by
tsor arrival time. Useseq. -
Using an unbounded
seqin a 32-bit field. The hand-packed layout above holds 4.29 billion messages per room; a busy public room can exceed that. Choose the width on purpose. -
JSON.stringifysurprises.undefinedfields vanish,NaNandInfinitybecomenull,Datebecomes an ISO string,BigIntthrows, and aMapbecomes{}. Build frames from plain objects and validate them before sending too. -
Large integers as numbers. Ids above 2^53 corrupt silently in JavaScript. Send them as strings.
-
Mixed text and binary without a plan. Decide per connection, and set
binaryTypetoarraybufferin browsers. -
Message and field names that leak the implementation.
postgres_row_idin a payload is a protocol you cannot change. -
Errors without
ref. A client that cannot correlate a failure to a request can only guess; always include the id when it could be parsed. -
Ignoring
retryable. Retrying aforbiddenforever, or never retrying arate_limited. -
Replaying a mutation on a new connection without the same id. The most common cause of double-posted messages after a flaky mobile network.
-
One giant union with no discriminator. Validation errors then list every branch and are useless.
-
Forgetting the other direction. Servers validate clients, but clients must validate servers too; the second is where version skew shows up first.
Version and specification notes
-
JSON is specified by RFC 8259 (December 2017, Internet Standard 90), which obsoletes RFC 7159 and requires UTF-8 for JSON exchanged between systems. RFC 7493 (the I-JSON profile) adds the interoperability rules this page leans on: unique member names, numbers that fit an IEEE double, and strings for anything larger. RFC 7396 (JSON Merge Patch) is the optional alternative to full-replacement edits.
-
UUIDv7 is time-ordered and was standardised in RFC 9562 (May 2024), which obsoletes RFC 4122. ULID is a community specification with the same time-ordered idea in a 26-character Crockford base32 string.
-
MessagePack is a community specification (versioned by its spec document, with the timestamp extension type defined in it);
@msgpack/msgpack3.1.3 is the current JavaScript implementation. -
Protocol Buffers:
proto3is the schema syntax used here; the encoding rules (varints, tags, wire types) are stable and documented separately, and the field-number reuse rule is the one rule that cannot be broken. -
Zod 4 (2025) added the built-in
z.toJSONSchema()and the top-levelz.int()andz.url()forms that this page uses; Zod 3 code usesz.number().int()andz.string().url(). -
Node.js: native TypeScript type stripping is on by default in Node 24 (and 22.18 and later); the global
WebSocketclient is stable since 22.4.0; there is still no built-in server, so servers usewsor another library. -
The Chatterbox protocol itself is versioned by
vand thechatterbox.v1subprotocol; this page is its single definition, and the stack pages must not change its semantics.
Related pages
-
The WebSocket Protocol for the frame format, opcodes and close handshake.
-
The Browser WebSocket API for
binaryType,bufferedAmountand events. -
Connection Lifecycle, Heartbeats and Reconnection for backoff, jitter and visibility handling.
-
Choosing a WebSocket Stack and the stack pages, which implement this protocol.
-
Chat Architecture and History, Offline Delivery and Moderation for what sits behind these messages.
-
WebRTC & WebSockets for the section overview.