Skip to content

Redis

An in-memory data store that answers in microseconds. Redis is not for analytics — it's for caching, queues, rate limiting, sessions, and low-latency lookups that keep ML systems fast.

Table of Contents

Installation / Setup

docker run -d -p 6379:6379 redis:7

pip install redis        # or: uv add redis

import redis
r = redis.Redis(host="localhost", port=6379, decode_responses=True)
r.ping()                 # True — connection ok

redis-cli                # CLI: set/get/... (commands below work identically)

Core Syntax

Strings (the atomic value)

r.set("model:version", "v3")        # SET
r.get("model:version")              # 'v3' (GET)
r.setex("cache:key", 300, payload)  # SET with 300s expiry (TTL)
r.incr("request:count")             # atomic increment
r.delete("model:version")           # DEL
r.exists("cache:key")               # 0/1

Data types quick table

Type Python Use
String set/get scalars, counters, JSON blobs
Hash hset/hget/hgetall objects with fields (a dict)
List lpush/rpush/lrange queues, recent items
Set sadd/smembers/sismember unique sets, memberships
Sorted set zadd/zrangebyscore leaderboards, ranges by score
Stream xadd/xread append-only event log

Hashes (nested-ish objects)

r.hset("user:42", mapping={"name": "Ada", "tier": "pro"})
r.hget("user:42", "name")             # 'Ada'
r.hgetall("user:42")                  # {'name': 'Ada', 'tier': 'pro'}
r.hincrby("user:42", "logins", 1)

TTL (time to live)

r.setex("cache:pred_42", 600, predictions_json)   # expires in 10 min
r.expire("user:42", 3600)                         # set TTL on existing
r.ttl("cache:pred_42")                            # seconds remaining

Common Operations

Caching with fallback (cache-aside)

def get_predictions(user_id):
    key = f"pred:{user_id}"
    cached = r.get(key)
    if cached is not None:
        return json.loads(cached)                 # cache hit (~1ms)
    pred = model.predict(...)                     # cache miss (~100ms+)
    r.setex(key, 600, json.dumps(pred.tolist()))
    return pred

Rate limiting (sliding window via INCR + EXPIRE)

def allow(user_id, limit=10, window=60):
    key = f"rate:{user_id}:{int(time.time() // window)}"
    n = r.incr(key)
    if n == 1:
        r.expire(key, window + 1)
    return n <= limit

Queues (Redis as message broker)

# producer
r.lpush("jobs:inference", json.dumps({"id": 42}))
# consumer
job = json.dumps(r.brpop("jobs:inference", timeout=5)[1])   # blocking pop

Sorted sets (leaderboards / thresholds)

r.zadd("model:leaderboard", {"model_a": 0.91, "model_b": 0.88})
r.zrevrange("model:leaderboard", 0, 2, withscores=True)   # top 3
r.zrangebyscore("features:recent", 100, 200)              # score range

Pub/Sub (notifications between services)

pub = r.pubsub()                       # subscriber
pub.subscribe("events")
r.publish("events", "model_updated")   # publisher

Practical Examples

1. Deduplicate requests / idempotency

if r.set(f"processed:{event_id}", "1", nx=True, ex=3600):
    process(event)                     # first time only — atomic SETNX

2. Feature-cache for a RAG/LLM service

# cache expensive embedding lookups
emb = r.get(f"emb:{text_hash}")
if emb is None:
    emb = embed(text)
    r.setex(f"emb:{text_hash}", 86400, emb)

3. In-memory session / experiment flags

r.hset(f"session:{session_id}", mapping={"user": "42", "variant": "control"})
variant = r.hget(f"session:{session_id}", "variant")

Common Patterns

# JSON blobs in strings
r.set("doc:123", json.dumps({"title": "x", "tags": ["a"]}))

# Cache stampede protection (single-flight)
# (distributed lock is a larger pattern; nx=True is the building block)
lock_ok = r.set("lock:refresh", "1", nx=True, ex=30)
if lock_ok:
    refresh(); r.delete("lock:refresh")

# Counter resetting daily
r.setex("api:quota:today", 86400, 0)
r.incr("api:quota:today")

Best Practices

  • Use Redis for speed paths, not source of truth — it's in-memory; data can be lost on restart (persistence options exist but have tradeoffs).
  • Always set TTLs on caches — without them, memory grows forever.
  • Store serialized JSON in strings for complex values; hashes when you mutate fields.
  • One key per entity with a prefix namespace: user:42, cache:page, rate:u42:60s.
  • Use nx=True for atomic create-if-absent (locks, idempotency) instead of check-then-set.
  • Batch with pipelines for many ops: pipe = r.pipeline() … pipe.execute().
  • Keyspace sizing: estimate memory; Redis must fit working set in RAM.
  • For durable analytical workloads, use a real database (PostgreSQL/MongoDB) and keep Redis as the accelerator.

Common Mistakes

Mistake Consequence Fix
No TTL on cache keys OOM over time setex/expire everywhere
Storing full DataFrames as one string Slow, bloated Store JSON or derived scalars; shard keys
Check-then-set for idempotency Races duplicate work SET NX
Using Redis for analytics queries Memory-heavy scans Let SQL/warehouse own analytics
decode_responses=False bytes everywhere decode_responses=True in python clients
One giant key per user with all fields Hot keys, contention Hash fields / smaller keys
Ignoring eviction policy Silent key loss when memory full Configure maxmemory-policy deliberately

Troubleshooting

Symptom Cause Fix
ConnectionError Server down / wrong port Check 6379; docker ps
OOM command not allowed maxmemory reached TTLs, eviction policy, bigger instance
Slow reads suddenly Large keys / many keys in one scan Hash sharding; avoid KEYS * (use SCAN)
WRONGTYPE Operation Key exists with another type Flush or rename; check r.type(key)
Pub/Sub messages lost No subscriber at publish time Use Streams (xadd) for durable events
Data gone after restart No persistence (AOF/RDB off) Enable appendonly yes if needed

Official Documentation

Last Updated

2026-09-06 — Initial version.