We are now, simply, Redis
If you’re confused because you can’t remember reading about publish or subscribe yet, don’t be — this is the first time we’ve talked about it. Generally, the concept of publish/ subscribe, also known as pub/sub, is characterized by listeners subscribing to channels, with publishers sending binary string messages to channels. Anyone listening to a given channel will receive all messages sent to that channel while they’re connected and listening. You can think of it like a radio station, where subscribers can listen to multiple radio stations at the same time, and publishers can send messages on any radio station. In this section, we’ll discuss and use operations involving publish and subscribe.
When finished with this section, you’ll know how to use these commands, and why we use other similar solutions in later chapters.
In Redis, the pub/sub concept has been included through the use of a collection of the five commands shown in table 3.11.
|Command||Example use and description|
|SUBSCRIBE||SUBSCRIBE channel [channel …] — Subscribes to the given channels|
|UNSUBSCRIBE||UNSUBSCRIBE [channel [channel …]] — Unsubscribes from the provided channels, or unsubscribes all channels if no channel is given|
|PUBLISH||PUBLISH channel message — Publishes a message to the given channel|
|PSUBSCRIBE||PSUBSCRIBE pattern [pattern …] — Subscribes to messages broadcast to channels that match the given pattern|
|PUNSUBSCRIBE||PUNSUBSCRIBE [pattern [pattern …]] — Unsubscribes from the provided patterns, or unsubscribes from all subscribed patterns if none are given|
With the way the PUBLISH and SUBSCRIBE commands are implemented on the Python side of things, it’s easier to demonstrate the feature if we use a helper thread to handle the PUBLISHing. You can see an example of PUBLISH/SUBSCRIBE in the next listing.3
The publish/subscribe pattern and its Redis implementation can be useful. If you skip ahead and scan around other chapters, you’ll notice that we only use publish/ subscribe in one other section, section 8.5. If PUBLISH and SUBSCRIBE are so useful, why don’t we use them very much? There are two reasons.
One reason is because of Redis system reliability. In older versions of Redis, a client that had subscribed to channels but didn’t read sent messages fast enough could cause Redis itself to keep a large outgoing buffer. If this outgoing buffer grew too large, it could cause Redis to slow down drastically or crash, could cause the operating system to kill Redis, and could even cause the operating system itself to become unusable. Modern versions of Redis don’t have this issue, and will disconnect subscribed clients that are unable to keep up with the client-output-buffer-limit pubsub configuration option (which we’ll talk about in chapter 8).
The second reason is for data transmission reliability. Within any sort of networked system, you must operate under the assumption that your connection could fail at some point. Typically, this is handled by one side or the other reconnecting as a result of a connection error. Our Python Redis client will normally handle connection issues well by automatically reconnecting on failure, automatically handling connection pooling (we’ll talk about this more in chapter 4), and more. But in the case of clients that have subscribed, if the client is disconnected and a message is sent before it can reconnect, the client will never see the message. When you’re relying on receiving messages over a channel, the semantics of PUBLISH/SUBSCRIBE in Redis may let you down.
It’s for these two reasons that we write two different methods to handle reliable message delivery in chapter 6, which works in the face of network disconnections, and which won’t cause Redis memory to grow (even in older versions of Redis) unless you want it to.
If you like the simplicity of using PUBLISH/SUBSCRIBE, and you’re okay with the chance that you may lose a little data, then feel free to use pub/sub instead of our methods, as we also do in section 8.5; just remember to configure client-output-buffer-limit pubsub reasonably before starting.
At this point, we’ve covered the majority of commands that you’ll use on a regular basis that are related to individual data types. There are a few more commands that you’ll also likely use, which don’t fit into our nice five structures-plus-pub/sub theme.
3If you’d like to run this code yourself, you can: I included the publisher() and run_pubsub() functions in the source code for this chapter.