· 10 min read
Getting WebSockets to 100k concurrent connections
What we changed when a client's real-time collaboration platform grew from 2,000 to 40,000 daily users: Redis pub/sub, sticky sessions, backpressure and monitoring.
The problem with WebSockets at scale
WebSockets look simple. A single Node.js process can comfortably hold 10-15k concurrent connections. Beyond that, memory pressure and event loop latency start to hurt every connected user. We found this out when a client's real-time collaboration platform went from 2,000 daily active users to 40,000 in three months.
The hard parts are keeping connections alive, routing messages efficiently and failing gracefully when the infrastructure misbehaves.
Architecture
We split the production system into three layers:
| Layer | Responsibility | Technology |
|---|---|---|
| Edge | TLS termination, sticky routing | HAProxy / AWS NLB |
| Connection | Socket lifecycle, heartbeats | Node.js + ws |
| Messaging | Pub/sub fanout, presence | Redis Cluster |
With this split, we can scale connection servers horizontally without worrying about message delivery semantics, and scale the messaging layer separately based on channel volume.
Connection servers
Each connection server is stateless from the application's perspective. It holds socket references in memory but leaves all business logic to the messaging layer. Our connection handler, stripped down:
import { WebSocketServer, WebSocket } from 'ws';
import { RedisClient } from './redis';
import { metrics } from './monitoring';
const HEARTBEAT_INTERVAL = 30_000;
const MAX_BACKPRESSURE = 1024 * 1024; // 1MB
interface ClientSocket extends WebSocket {
isAlive: boolean;
userId: string;
channels: Set<string>;
}
export function createConnectionHandler(wss: WebSocketServer, redis: RedisClient) {
wss.on('connection', (ws: ClientSocket, req) => {
ws.isAlive = true;
ws.channels = new Set();
metrics.connectionsActive.inc();
ws.on('pong', () => { ws.isAlive = true; });
ws.on('message', async (raw) => {
// Backpressure check — drop messages if the client
// can't keep up rather than buffering unboundedly
if (ws.bufferedAmount > MAX_BACKPRESSURE) {
metrics.messagesDropped.inc({ reason: 'backpressure' });
return;
}
try {
const msg = JSON.parse(raw.toString());
await handleMessage(ws, msg, redis);
} catch (err) {
ws.send(JSON.stringify({ error: 'invalid_message' }));
}
});
ws.on('close', () => {
metrics.connectionsActive.dec();
cleanupSubscriptions(ws, redis);
});
});
// Heartbeat sweep — terminates zombies
setInterval(() => {
wss.clients.forEach((ws: ClientSocket) => {
if (!ws.isAlive) {
metrics.connectionsTerminated.inc({ reason: 'heartbeat_timeout' });
return ws.terminate();
}
ws.isAlive = false;
ws.ping();
});
}, HEARTBEAT_INTERVAL);
}Horizontal scaling with Redis pub/sub
The core scaling mechanism is Redis pub/sub. When a user sends a message to a channel, their connection server publishes it to Redis. Every connection server subscribed to that channel receives the message and fans it out to its own connected clients.
package fanout
import (
"context"
"encoding/json"
"log/slog"
"sync"
"github.com/redis/go-redis/v9"
)
type Fanout struct {
rdb *redis.Client
mu sync.RWMutex
listeners map[string][]chan<- []byte
}
func New(rdb *redis.Client) *Fanout {
return &Fanout{
rdb: rdb,
listeners: make(map[string][]chan<- []byte),
}
}
func (f *Fanout) Subscribe(ctx context.Context, channel string, ch chan<- []byte) {
f.mu.Lock()
f.listeners[channel] = append(f.listeners[channel], ch)
needsSubscribe := len(f.listeners[channel]) == 1
f.mu.Unlock()
if needsSubscribe {
go f.redisSubscribe(ctx, channel)
}
}
func (f *Fanout) redisSubscribe(ctx context.Context, channel string) {
sub := f.rdb.Subscribe(ctx, channel)
defer sub.Close()
for msg := range sub.Channel() {
f.mu.RLock()
listeners := f.listeners[channel]
f.mu.RUnlock()
for _, ch := range listeners {
select {
case ch <- []byte(msg.Payload):
default:
slog.Warn("listener backpressure, dropping message",
"channel", channel)
}
}
}
}Why we did not use Redis Streams
We evaluated Redis Streams for guaranteed delivery, but the overhead was not justified for our use case. Pub/sub's fire-and-forget model is fine once you accept that WebSocket messages are best-effort anyway. If a client misses a message during reconnection, it fetches the current state over HTTP. That is simpler and more resilient than building exactly-once delivery over WebSockets.
Sticky sessions and load balancing
WebSocket connections are long-lived, so you need sticky routing at the load balancer layer. We use source-IP hashing at the network load balancer:
| Strategy | Pros | Cons |
|---|---|---|
| Source IP hash | Simple, no cookies needed | Uneven distribution behind corporate NATs |
| Cookie-based | Even distribution | Requires HTTP upgrade path |
| Connection ID header | Precise control | Custom client logic required |
For most deployments, source IP hashing with a fallback rebalance mechanism is enough. We run a background process that monitors per-server connection counts and starts a gradual drain when the imbalance exceeds 20%.
Backpressure and graceful degradation
Server-side backpressure
The bufferedAmount property on a WebSocket is the number to watch. When a client cannot consume messages fast enough, the kernel send buffer fills up and bufferedAmount grows. We enforce a hard cap: any client with more than 1MB of buffered data is deprioritized and receives only critical messages until the buffer drains.
Degrading under load
When connection servers approach capacity, we degrade in three tiers:
Reduce broadcast frequency
Non-critical updates (typing indicators, cursor positions) are throttled from real-time to 2-second batches. Users barely notice, and message volume drops by 60-70%.
Shed non-essential connections
Read-only spectators on collaborative documents move to long-polling at 5-second intervals. This frees socket slots for active editors.
Reject new connections with retry-after
New connection attempts receive a 503 with a Retry-After header. Clients back off exponentially with jitter, which prevents a thundering herd when the servers recover.
What we monitor
After two years of running this stack, our dashboard is down to the metrics that predict incidents:
- Event loop lag (p99). Above 100ms, message processing is too slow. It is the first metric to spike, before users notice latency.
- Connections per server, current vs. capacity. We alert at 70% capacity to leave time for scaling.
- Redis pub/sub subscriber count. A mismatch between expected and actual subscribers means a connection server has silently lost its connection to Redis.
- Message fanout ratio. Messages published vs. messages delivered. A ratio that stays below expected means dropped messages.
import { Histogram, Gauge, Counter } from 'prom-client';
export const metrics = {
connectionsActive: new Gauge({
name: 'ws_connections_active',
help: 'Currently active WebSocket connections',
labelNames: ['server_id'],
}),
messageLatency: new Histogram({
name: 'ws_message_latency_seconds',
help: 'End-to-end message delivery latency',
buckets: [0.001, 0.005, 0.01, 0.05, 0.1, 0.5, 1],
}),
messagesDropped: new Counter({
name: 'ws_messages_dropped_total',
help: 'Messages dropped due to backpressure or errors',
labelNames: ['reason'],
}),
connectionsTerminated: new Counter({
name: 'ws_connections_terminated_total',
help: 'Connections terminated by the server',
labelNames: ['reason'],
}),
};Lessons from production
Connection storms
When a connection server restarts, all of its clients reconnect at once. Without jitter in the reconnection logic, you get a thundering herd that can cascade across the cluster. We enforce client-side reconnection delays of min(30s, baseDelay * 2^attempt + random(0, 1000)ms).
Memory leaks are subtle
Each WebSocket connection carries per-socket buffers, event listeners and application state. A single event listener that never gets removed and holds a closure over a large object will slowly eat the server's memory over days. We take weekly heap snapshots in staging under synthetic load to catch these early.
Redis pub/sub has limits
A single Redis instance handles roughly 500k pub/sub messages per second, but subscriber fan-out is the bottleneck. With 50 connection servers each subscribed to 10,000 channels, Redis spends most of its time iterating over subscriber lists. We shard channels across a Redis Cluster and hash channel names to spread them evenly.
Written by the MiraiTech team in Tokyo
If something here is wrong, or you are working on a similar problem, write to us at contact@mirai-tech.net.