Redis for System Design

Redis (REmote DIctionary Server) is an open source, in memory (RAM) data structure used as a high performance database, cache and message broker. ### What is Redis? Redis is a super-fast network accessible data structure server. It lives in memory, meaning that it can have microsecond level response times. Additionally, it's single threaded providing predictable performance and no race conditions. Optionally, it can be persisted on disk and can be used as infrastructure. Redis has lots of use cases and functionality including Redis Cluster and Redis Pub/Sub. The key strength of Redis is that since it's fast and single threaded, it's **predictable** and useful for real-time systems. Note that since Redis commands are atomic this eliminates race conditions within Redis itself. ### How is Redis useful? Redis is useful throughout large distributed systems. Firstly, Redis is a great **cache**, since it's single threaded (no race conditions) and very fast. It is typically placed between the server and database layer within large distributed systems. This reduces the time it takes to query data, and since large, amalgamated queries can be saved within Redis this reduces the number of queries and server time taken to process these. Note that for caches you can have TTL's (Time To Live) and eviction policies including LRU (Least Recently Used), FIFO (First In, First Out), and LFU (Least Frequently Used). *** Another use case for Redis is as a **job queue**. Although Redis is a key-value data structure, the values that Redis can contain can be complex data structures. These include: strings, hashes, *lists*, sets, sorted sets, and streams. Importantly, for this use case, you can have the key identify the queue and the value be the queue itself. Because Redis is single threaded & in memory, this makes it fast with no race conditions. For a job queue this means predictable and consistent behavior. *** Redis **Pub/Sub** is a lightweight message broadcast system built into Redis. It works since Redis can fan out messages to many subscribers in real time with its event loop and in-memory data structures. The basic model is as follows: - Publishers send messages to a channel - Subscribers listen to one or more channels - Messages are delivered to all active subscribers There is no queue and no persistence, i.e. if you're not subscribed at publish time, you miss the message. Pub/Sub messages aren't stored as keys or values, they live only in memory during delivery. More specifically it works like this: 1. Subscriber opens long-lived TCP connection 2. Redis registers connection under a channel name 3. Publisher sends *Publish* 4. Redis synchronously - Finds all subscribers for the channel - Writes message to each socket's output buffer 5. Message delete *** Redis can also be used to implement a **distributed lock**. Redis executes commands atomically, which allows multiple processes to coordinate access to a shared resource using a single key. Locks are implemented using conditional key creation with an expiration time. This prevents deadlocks if a client crashes while holding the lock. The expiration makes Redis locks time bounded leases, not permanent mutexes. This makes this sensitive to the TTL value set. Locks must include a unique token so that a client can verify ownership and avoid releasing someone else's lock. Redis locks are safe because of atomic ownership checks, not necessarily because Redis is single-threaded. *** Optionally, you can **persist Redis to disk** using RDB (Redis Database) snapshots and/or append only files (AOF). Redis Database snapshots are point-in-time compact binary backups of an entire Redis database, captured by a single file (dump.rdb) on disk. This enables fast recovery after failures or restarts.

🎥 Video