Redis Essentials for Backend Developers

Redis is an in-memory data store that backend developers use as a cache, a session store, a rate limiter and a lightweight message broker. Its Pub/Sub feature is the simplest way to push real-time events between services: one client runs PUBLISH to send a message to a channel, and every client that ran SUBSCRIBE on that channel receives it at once. Redis keeps no copy, so a subscriber that is offline misses the message. This post covers Pub/Sub properly, then the five core data types and when to use each, TTL, cache-aside, rate limiting, RDB vs AOF persistence, and how Redis compares with Memcached.

What makes Redis fast

Redis keeps its dataset in RAM instead of reading from disk on every request, so a typical lookup takes well under a millisecond and costs far less than a database query. Over a network both still pay the round trip, so the big win is taking repeated read load off your database. At its core it's a key-value store: SET user:1:name "Arun" stores a value, GET user:1:name retrieves it, and the value isn't limited to plain strings. The mental model worth keeping: PostgreSQL is your long-term source of truth; Redis is the fast-access layer sitting in front of it, reducing load and cutting response times for anything read repeatedly.

A note on licensing: starting with Redis 7.4, Redis moved from the BSD license to source-available licenses (RSALv2 and SSPLv1), and Redis 8 added AGPLv3 as an open-source option. Valkey is a Linux Foundation fork of Redis 7.2 under the BSD license. It speaks the same protocol, so everything in this post works on either.

Good key naming, upfront

Redis has no concept of tables or schemas, so key naming is your only structure. The standard pattern is colon-separated namespaces: user:1, user:1:session, cache:blog:post:123, rate_limit:login:192.168.1.5. Consistent naming isn't cosmetic; it's what makes debugging and scanning keys in production actually manageable months later, when "temp" or "data1" would tell you nothing.

The five core data structures, and what each is for

Strings are the simplest type: text, numbers, JSON blobs, serialized objects. INCR and DECR atomically bump a numeric string up or down, which is exactly what you want for view counters and rate limits where two simultaneous requests must never both read the same starting number. Good fits: API response caching, OTP (one-time password) codes, feature flags, JWT blacklist entries.

Hashes group multiple fields under one key (HSET user:1 name "Arun" email "arun@example.com" role "admin"), instead of scattering user:1:name, user:1:email, user:1:role as three separate keys. Good fits: user profiles, product metadata, session data, or anywhere you'd otherwise reach for a small object.

Lists are ordered collections: LPUSH and RPUSH add to either end, LRANGE reads a slice. Good fits: simple FIFO queues, recent activity feeds, notification stacks, or anywhere order matters and you're mostly adding to one end and reading from the other.

Sets store unique, unordered values: SADD post:123:tags "redis" "backend" adds tags, and adding "redis" again is a no-op since sets can't hold duplicates. Good fits: tags, unique visitor tracking, permission sets, deduplication.

Sorted sets are sets where every member also carries a numeric score, and Redis keeps the whole set ordered by that score automatically: ZADD leaderboard 150 "meera", then ZREVRANGE leaderboard 0 9 WITHSCORES instantly returns the top 10, already sorted, no separate ORDER BY query needed. Good fits: leaderboards, trending content rankings, anything priority-ordered.

# String
SET user:1:name "Arun"
GET user:1:name

# Hash
HSET user:1 name "Arun" email "arun@example.com" role "admin"
HGETALL user:1

# List
RPUSH activity:feed "signed_up" "viewed_pricing"
LRANGE activity:feed 0 -1

# Set
SADD post:123:tags "redis" "backend"
SMEMBERS post:123:tags

# Sorted set
ZADD leaderboard 150 "meera" 90 "arun"
ZREVRANGE leaderboard 0 9 WITHSCORES

Redis TTL: keys that clean themselves up

Any key can be given a TTL (time-to-live): EXPIRE otp:user:42 300 means Redis deletes that key automatically 300 seconds from now, no background cleanup job required. TTL returns tell you the state: a positive number is seconds remaining, -1 means the key exists but never expires, and -2 means the key doesn't exist at all. This one feature is what makes Redis the natural home for OTPs, password reset tokens, session tokens, and rate-limit windows. Anything that should only be valid for a limited time cleans itself up.

# Generate an OTP that expires in 5 minutes
SET otp:user:42 "483920" EX 300

# Check how much time is left
TTL otp:user:42
# -> 287 (seconds remaining)

# A key with no expiry set
TTL user:1
# -> -1 (exists, never expires)

# A key that's already gone
TTL otp:user:99
# -> -2 (doesn't exist)

Redis TTL gotchas

• –A plain SET on an existing key wipes its TTL. Update a session that way and it never expires. Pass EX again, or use SET ... KEEPTTL (Redis 6.0+) to keep the current expiry.
• –Commands that change a value without replacing it keep the TTL: INCR, HSET, LPUSH. That's why the rate limiter further down can set the expiry once and keep incrementing.
• –Since Redis 7.0, EXPIRE takes NX, XX, GT and LT: set the TTL only if there isn't one, only if there is one, or only if the new TTL is longer or shorter. A key with no TTL counts as infinite for GT and LT, so EXPIRE key 50 GT on it does nothing and returns 0.
• –PERSIST removes a TTL. PEXPIRE and PTTL do the same jobs in milliseconds.
• –Expired keys are removed when something tries to read them, plus by a background job that tests a few random keys with a TTL at a time. Keys nobody reads are cleaned up gradually, not at the exact second. The EXPIRE docs cover the rest.

SET session:abc "v1" EX 100
SET session:abc "v2"            # TTL is gone
TTL session:abc                 # -> -1 (never expires)

SET session:abc "v1" EX 100
SET session:abc "v2" KEEPTTL    # TTL survives
TTL session:abc                 # -> 100 (or a little less)

The cache-aside pattern

This is the pattern behind almost every Redis cache: check Redis first; if the data's there, return it immediately (a cache hit); if not, fetch from the database, store the result in Redis with a TTL, then return it (a cache miss). Next time the same request comes in within the TTL window, it's served straight from memory. The part beginners skip, and shouldn't: cache invalidation. When the underlying data changes (say, a user updates their profile), the cached copy has to be deleted (or updated) in the same operation, or Redis will happily keep serving the old version until the TTL naturally expires.

// Uses the ioredis client. With node-redis v4+, the TTL is written as
// redis.set(key, value, { EX: 300 }).
import { Redis } from 'ioredis';

const redis = new Redis();

type UserProfile = { id: number; name: string; email: string };

async function getUserProfile(userId: number): Promise<UserProfile | null> {
  const cacheKey = `cache:user:${userId}`;

  // 1. Check Redis first
  const cached = await redis.get(cacheKey);
  if (cached !== null) {
    return JSON.parse(cached); // cache hit (can be a cached null, see step 3)
  }

  // 2. Miss — fall back to the database. Select only the fields you want
  // cached, so a password hash never ends up in Redis.
  const user = await User.findByPk(userId, {
    attributes: ['id', 'name', 'email'],
  });
  const profile = user ? (user.toJSON() as UserProfile) : null;

  // 3. Store it for next time, with a TTL. Caching null is deliberate:
  // it stops repeated database lookups for a user that doesn't exist.
  await redis.set(cacheKey, JSON.stringify(profile), 'EX', 300);

  return profile; // same plain-object shape on a hit or a miss
}

async function updateUserProfile(userId: number, data: Partial<UserProfile>) {
  const user = await User.findByPk(userId);
  if (!user) return null;
  await user.update(data);

  // Invalidate the cache so the next read isn't stale
  await redis.del(`cache:user:${userId}`);

  return getUserProfile(userId);
}

Sessions and rate limiting

Server-side sessions fit Redis naturally: create a session ID, store session data in a Redis hash (HSET session:xyz userId 42 role admin), attach a TTL, send the session ID to the client as a cookie. It's fast and self-expiring, and unlike sessions kept in one server's memory, it works correctly across multiple server instances behind a load balancer, since every instance reads from the same Redis.

Rate limiting uses the same counter idea: one key per IP or user with a 60-second expiry, incremented on every request, and the request gets rejected once the count crosses your threshold. Send the increment and the expiry together in one transaction (MULTI, INCR, EXPIRE ... NX, EXEC). If you run INCR and then EXPIRE as two separate commands and the app crashes between them, the key never expires and that IP stays blocked for good. Creating the key first with SET ... EX 60 NX and then calling INCR has the same hole: if the key expires between the two commands, INCR creates a fresh key with no expiry. A few lines get you "5 login attempts per IP per minute" with no extra infrastructure.

# Count this request and start the 60-second window in one atomic step.
# EXPIRE ... NX (Redis 7+) only sets a TTL when the key has none,
# so later requests in the same window don't push the expiry back.
MULTI
INCR rate_limit:login:192.168.1.5
EXPIRE rate_limit:login:192.168.1.5 60 NX
EXEC
# -> the first reply is the new count. If it's > 5, reject the request (HTTP 429).

# Behind a load balancer, key on the real client IP (from a trusted
# X-Forwarded-For header), or every user shares the balancer's IP.

Redis Pub/Sub: channels, patterns and delivery

Redis Pub/Sub lets one part of your system PUBLISH a message to a channel and any number of subscribers SUBSCRIBE to receive it in real time. It's handy for chat notifications or lightweight event broadcasting between services. The catch: it's at-most-once delivery. If a subscriber isn't connected at the moment a message is published, that message is simply gone. Nothing stores it for later. That makes Pub/Sub great for live, transient events and a bad fit for anything that must never be dropped, like a durable job queue (which is exactly why BullMQ, a popular Node.js job queue library, is built as its own layer on top of Redis, rather than just using Pub/Sub directly). If you need Redis itself to keep messages for consumers that were offline, use Redis Streams: messages stay in the stream until you trim or delete them, so a consumer that reconnects can read what it missed and acknowledge each one.

SUBSCRIBE, PUBLISH and PSUBSCRIBE

Channels are just names. They don't live in the keyspace and they ignore database numbers, so a message published on db 10 still reaches a subscriber on db 1. If you need to separate environments, put it in the channel name (prod:notifications:user:42). PUBLISH returns how many clients received the message, pattern subscribers included, and 0 means nobody was listening. PSUBSCRIBE takes a glob pattern like notifications:* and delivers matching messages with the pattern attached. A client subscribed to both a channel and a pattern that matches it gets the same message twice, once for each subscription. The Redis Pub/Sub docs show the exact wire format.

# Terminal 1: subscribe to one channel
SUBSCRIBE notifications:user:42

# Terminal 2: subscribe to every notifications channel
PSUBSCRIBE notifications:*

# Terminal 3: publish
PUBLISH notifications:user:42 '{"type":"comment","postId":7}'
# -> (integer) 2   one channel subscriber + one pattern subscriber

# Terminal 1 receives: message  notifications:user:42  {"type":"comment","postId":7}
# Terminal 2 receives: pmessage notifications:*  notifications:user:42  {"type":"comment","postId":7}

A working Node.js publisher and subscriber with ioredis

The rule that trips people up in code: on the default RESP2 protocol, a connection that has run SUBSCRIBE can only run subscription commands, plus PING, RESET and QUIT. So the subscriber and the publisher each get their own connection. I ran this file with Node.js and ioredis 6 against a local Redis 8; it prints the three lines at the bottom.

import { Redis } from 'ioredis';

// A connection that runs SUBSCRIBE can't run normal commands,
// so the subscriber and the publisher each get their own.
const sub = new Redis(); // localhost:6379 by default
const pub = new Redis();

type Notification = { userId: number; type: 'comment' | 'like'; postId: number };

// Exact channel names arrive as 'message'
sub.on('message', (channel, message) => {
  const event = JSON.parse(message) as Notification;
  console.log(`[${channel}] user ${event.userId} got a ${event.type}`);
});

// Pattern subscriptions arrive as 'pmessage'
sub.on('pmessage', (pattern, channel, message) => {
  console.log(`[${pattern} matched ${channel}] ${message}`);
});

await sub.subscribe('notifications:user:42');
await sub.psubscribe('audit:*');

const event: Notification = { userId: 42, type: 'comment', postId: 7 };
const received = await pub.publish('notifications:user:42', JSON.stringify(event));
console.log(`delivered to ${received} subscriber(s)`); // 0 means nobody was listening

await pub.publish('audit:login', JSON.stringify({ userId: 42 }));

// Give the subscriber a moment to print, then close both connections.
setTimeout(() => {
  sub.quit();
  pub.quit();
}, 200);

// Output:
// delivered to 1 subscriber(s)
// [notifications:user:42] user 42 got a comment
// [audit:* matched audit:login] {"userId":42}

What at-most-once delivery means in practice

• –A subscriber that is disconnected, restarting or not yet subscribed misses everything published in that window. ioredis resubscribes to its channels after a reconnect (the autoResubscribe option, on by default), but it can't recover the messages sent while it was gone.
• –A slow subscriber gets cut off. Redis buffers outgoing messages per client, and the default client-output-buffer-limit for Pub/Sub clients is 32 MB, or 8 MB sustained for 60 seconds. Past that, Redis closes the connection and the buffered messages are lost.
• –There's no acknowledgement and no replay. If your message handler throws, that message is gone.
• –Order does hold: subscribers receive messages in the order they were published.

Sharded Pub/Sub in Redis Cluster (SSUBSCRIBE and SPUBLISH)

In Redis Cluster, a classic PUBLISH is forwarded to every node over the cluster bus, so adding nodes doesn't add Pub/Sub capacity, and its return value only counts subscribers on the node you published to. Redis 7.0 added sharded Pub/Sub: SSUBSCRIBE, SUNSUBSCRIBE and SPUBLISH hash a shard channel to a slot the same way keys are hashed, and the message only travels within the shard that owns that slot. Subscribers can connect to that shard's primary or any of its replicas. In ioredis, sharded subscriptions need a Cluster client created with shardedSubscribers: true, messages arrive on the smessage event, and all channels in one ssubscribe call must hash to the same slot (use a hash tag, like orders{eu}:created). On a single Redis server you don't need any of this.

Redis Pub/Sub vs Streams: which one to use

Use Pub/Sub when a missed message doesn't matter, because the next one replaces it or the client reloads state anyway: typing indicators, "new comment" toasts, presence, telling every app server to drop a cached value. Use Redis Streams when every message has to be processed. XADD appends an entry that stays in the stream, consumer groups split entries between consumers with XREADGROUP, and each entry stays in the group's pending list until a consumer runs XACK, so another consumer can claim the work of one that crashed. For background jobs with retries and scheduling, a queue library on top of Redis, like BullMQ, saves you building all of that yourself.

# Create a consumer group ($ = only entries added from now on)
XGROUP CREATE events mailers $ MKSTREAM

# Producer: append an entry (it stays until you trim it)
XADD events * type signup userId 42

# Consumer: read up to 10 new entries for this group, waiting up to 5 seconds
XREADGROUP GROUP mailers worker-1 COUNT 10 BLOCK 5000 STREAMS events >

# After handling an entry, acknowledge it using the id XREADGROUP returned
XACK events mailers 1792608037930-0

Persistence and eviction: the parts people forget

Redis is in-memory, but it can still persist to disk. RDB takes periodic point-in-time snapshots. They're compact and fast to restore from, but anything written after the last snapshot is lost if Redis crashes. AOF logs every write operation and replays them on recovery. It's more durable, at the cost of larger files. Many production setups run both. The practical rule: pure cache data can usually skip persistence entirely (losing it just means more cache misses for a while); sessions, rate-limit state, or anything closer to "real" data deserves AOF or both.

The defaults matter here. Out of the box (no redis.conf overrides), Redis has RDB on (snapshot after an hour if at least 1 key changed, after 5 minutes if 100 changed, after 60 seconds if 10,000 changed) and AOF off (appendonly no). Turn AOF on and the default appendfsync everysec means a crash costs you about one second of writes; with RDB alone you lose everything since the last snapshot, which is usually minutes. When both are on, Redis rebuilds from the AOF on restart because it's the more complete copy. Managed Redis services pick their own defaults, so check yours. The Redis persistence docs go through the trade-offs in detail.

# RDB snapshots (these are the built-in defaults)
save 3600 1 300 100 60 10000

# Turn on the append-only file
appendonly yes
appendfsync everysec   # the default: fsync once a second

Separately, when Redis hits its configured maxmemory limit, an eviction policy decides what happens next: noeviction simply rejects new writes with an error, while other policies evict existing keys to make room. If Redis is a pure cache, unexpected eviction is fine, because the data was disposable by design. If Redis is holding sessions or workflow state, an aggressive eviction policy quietly breaking your application is a real risk, so memory limits deserve actual planning, not defaults left untouched. If the same Redis also runs BullMQ, it must use noeviction (the default): any other policy can silently delete queue data.

Redis vs Memcached

Both keep data in memory, and both are fast enough that the network round trip is usually the slow part. The difference is what else you get. Memcached stores opaque values under string keys, with an expiry; it's multithreaded, it has no persistence or built-in replication, and by default an item can be at most 1 MB. Redis adds the data types above, persistence, replication, Pub/Sub, Streams, transactions and Lua scripting, and a single string value can be up to 512 MB. Redis runs commands on one main thread (Redis 6 added optional I/O threads for network reads and writes), which is also why each command is atomic.

Memcached still makes sense for a plain string cache on a large multi-core machine. Pick Redis when the cache may grow into sessions, counters, rate limits or queues, which for most Node.js backends is sooner than you'd expect.

Mistakes worth avoiding

• –Treating Redis as your only copy of important data without a persistence plan. Fine for cache, dangerous for anything you can't afford to lose.
• –Storing everything as one giant JSON string in a plain string key when a hash, set, or sorted set would be a better structural fit.
• –Forgetting TTL on temporary data. Sessions and OTPs that never expire just grow memory usage forever.
• –Assuming Pub/Sub stores messages for offline subscribers. It doesn't.
• –Publishing and subscribing on the same connection. Once a connection subscribes, normal commands on it fail, so give subscribers their own.
• –Updating a key with a plain SET and silently dropping its TTL.

Common questions

Does Redis Pub/Sub store messages?

No. Redis delivers each message to the clients subscribed at that moment and keeps no copy. If messages have to survive a disconnect, use Redis Streams.

Can one Redis connection publish and subscribe?

Not on the default RESP2 protocol: a subscribed connection only accepts subscription commands plus PING, RESET and QUIT. Use one connection to subscribe and another for everything else. Over RESP3, a subscribed client can run any command.

What's the difference between Redis Pub/Sub and Streams?

Pub/Sub is a fire-and-forget broadcast with at-most-once delivery. Streams are an append-only log with consumer groups and acknowledgements, so consumers can catch up on entries they missed and you can get at-least-once processing.

Does Redis lose data on restart?

Only what wasn't persisted. With the default RDB snapshots, you lose the writes since the last snapshot. With AOF and appendfsync everysec, about one second. With persistence turned off, everything.

What happens to a key's TTL when I update it?

It depends on the command. SET replaces the value and clears the TTL unless you pass EX or KEEPTTL. INCR, HSET, LPUSH and similar commands change the value in place and leave the TTL alone.

What to read next

Next in the series: BullMQ Essentials, a job queue built on top of Redis. For where Redis fits alongside PostgreSQL, queues and search, see how I'm learning backend development. I'm a frontend engineer who builds the UI on top of APIs like this one. If you need that, see my work or get in touch.