Designing a Chat/Messaging App
Real-time delivery and never losing a message pull the design in two directions — the interesting part is reconciling them.
Advanced
TopNotchNote
Notes that matter
requirements
A chat app — call it Threadline — lets users send real-time direct messages. This case study is a good stress test for this track's protocol and ordering pages, because "real-time" and "never lose a message" are two requirements that pull the design in different directions.
| Functional | Non-functional |
|---|---|
| Send and receive messages in near real time | Low latency for delivery to an online recipient |
| Deliver missed messages when a user comes back online | No message loss, even if the recipient is offline for days |
| Show delivery status (sent / delivered / read) | Messages within one conversation must stay in order |
why plain HTTP request-response doesn't fit
HTTP is client-driven — the server can't push a new message to a recipient without the recipient asking first. See Communication Protocols: this is exactly the scenario WebSocket exists for. Threadline keeps a persistent WebSocket connection open for every currently-online user, letting the server push a message the instant it arrives instead of the recipient having to poll for it.
high-level design
| Component | Role |
|---|---|
| Connection gateway | Holds the open WebSocket connection per online user; maintains a mapping of user ID → which gateway instance holds their connection (since a large user base is spread across many gateway machines). |
| Message service | Receives a send request, persists the message, and looks up whether the recipient is currently online. |
| Message store | The durable record of every message — a natural fit for a database that handles high write volume and mostly-sequential access per conversation. |
| Presence service | Tracks who's currently online and which gateway instance holds their connection — commonly backed by a fast key-value store given the simple, high-frequency read/write pattern. |
the send path
1. User A sends a message via their open WebSocket connection
2. Message Service persists it to the Message Store (durability first)
3. Message Service checks Presence Service: is User B online?
4. If yes: forward the message to the gateway instance holding User B's connection, push it immediately
5. If no: leave it durably stored; User B receives it on next connect
Persisting the message before attempting real-time delivery is the key ordering decision here — it guarantees a message survives even if the delivery push itself fails, which mirrors the same "durability before best-effort delivery" pattern from Distributed Messaging Queues.
ordering messages within a conversation
Two messages in the same conversation, sent moments apart, need to display in the order they were actually sent — a case where causal ordering genuinely matters (see Consistency Models). Rather than relying on each client's own clock (unreliable across devices), each message is assigned a monotonically increasing sequence number scoped to its conversation, generated server-side at write time — a narrower, per-conversation version of the same unique-ID-generation problem covered in Unique ID Generation at Scale.
the distinctive component: offline delivery without polling
The hard part of this design isn't sending a message to someone who's online — it's guaranteeing delivery to someone who's been offline for three days without making every client poll constantly in the meantime.
The fix: every message write also updates a per-user "undelivered count" or cursor. When a client reconnects, before opening its live WebSocket feed it makes one request: "give me everything after sequence number N" for each conversation it's part of. This turns a potentially large backlog into a single bounded catch-up query, after which the client switches to the live push path — rather than either replaying the client's entire history or requiring it to poll repeatedly while offline.
evaluation
| Requirement | How it's met |
|---|---|
| Real-time delivery when online | WebSocket push via the connection gateway holding the recipient's live connection. |
| No message loss when offline | Messages persist to durable storage before any delivery attempt; a reconnect triggers a bounded catch-up query. |
| Correct ordering | Server-assigned, per-conversation monotonic sequence numbers instead of trusting client clocks. |
| Scales with user count | Presence lookups and connection routing are key-value-backed, and the message store scales the same way any high-write dataset does — see Data Partitioning & Sharding Strategies. |