3.3 Sets

  • Redis in Action – Home
  • Foreword
  • Preface
  • Part 1: Getting Started
  • Part 2: Core concepts
  • 1.3.1 Voting on articles
  • 1.3.2 Posting and fetching articles
  • 1.3.3 Grouping articles
  • 4.2.1 Configuring Redis for replication
  • 4.2.2 Redis replication startup process
  • 4.2.3 Master/slave chains
  • 4.2.4 Verifying disk writes
  • 5.1 Logging to Redis
  • 5.2 Counters and statistics
  • 5.3 IP-to-city and -country lookup
  • 5.4 Service discovery and configuration
  • 5.1.1 Recent logs
  • 5.1.2 Common logs
  • 5.2.2 Storing statistics in Redis
  • 5.3.1 Loading the location tables
  • 5.3.2 Looking up cities
  • 5.4.1 Using Redis to store configuration information
  • 5.4.2 One Redis server per application component
  • 5.4.3 Automatic Redis connection management
  • 8.1.1 User information
  • 8.1.2 Status messages
  • 9.1.1 The ziplist representation
  • 9.1.2 The intset encoding for SETs
  • Chapter 11: Scripting Redis with Lua
  • 11.1.1 Loading Lua scripts into Redis
  • 11.1.2 Creating a new status message
  • 11.2 Rewriting locks and semaphores with Lua
  • 11.3 Doing away with WATCH/MULTI/EXEC
  • 11.4 Sharding LISTs with Lua
  • 11.5 Summary
  • 11.2.1 Why locks in Lua?
  • 11.2.2 Rewriting our lock
  • 11.2.3 Counting semaphores in Lua
  • 11.4.1 Structuring a sharded LIST
  • 11.4.2 Pushing items onto the sharded LIST
  • 11.4.4 Performing blocking pops from the sharded LIST
  • A.1 Installation on Debian or Ubuntu Linux
  • A.2 Installing on OS X
  • B.1 Forums for help
  • B.4 Data visualization and recording
  • Buy the paperback
  • Redis in Action – Home
  • Foreword
  • Preface
  • Part 1: Getting Started
  • Part 2: Core concepts
  • 1.3.1 Voting on articles
  • 1.3.2 Posting and fetching articles
  • 1.3.3 Grouping articles
  • 4.2.1 Configuring Redis for replication
  • 4.2.2 Redis replication startup process
  • 4.2.3 Master/slave chains
  • 4.2.4 Verifying disk writes
  • 5.1 Logging to Redis
  • 5.2 Counters and statistics
  • 5.3 IP-to-city and -country lookup
  • 5.4 Service discovery and configuration
  • 5.1.1 Recent logs
  • 5.1.2 Common logs
  • 5.2.2 Storing statistics in Redis
  • 5.3.1 Loading the location tables
  • 5.3.2 Looking up cities
  • 5.4.1 Using Redis to store configuration information
  • 5.4.2 One Redis server per application component
  • 5.4.3 Automatic Redis connection management
  • 8.1.1 User information
  • 8.1.2 Status messages
  • 9.1.1 The ziplist representation
  • 9.1.2 The intset encoding for SETs
  • Chapter 11: Scripting Redis with Lua
  • 11.1.1 Loading Lua scripts into Redis
  • 11.1.2 Creating a new status message
  • 11.2 Rewriting locks and semaphores with Lua
  • 11.3 Doing away with WATCH/MULTI/EXEC
  • 11.4 Sharding LISTs with Lua
  • 11.5 Summary
  • 11.2.1 Why locks in Lua?
  • 11.2.2 Rewriting our lock
  • 11.2.3 Counting semaphores in Lua
  • 11.4.1 Structuring a sharded LIST
  • 11.4.2 Pushing items onto the sharded LIST
  • 11.4.4 Performing blocking pops from the sharded LIST
  • A.1 Installation on Debian or Ubuntu Linux
  • A.2 Installing on OS X
  • B.1 Forums for help
  • B.4 Data visualization and recording
  • Buy the paperback

    3.3 Sets

    You’ll remember from chapter 1 that SETs hold unique items in an unordered fashion. You can quickly add, remove, and determine whether an item is in the SET. Among the many uses of SETs are storing who voted for an article and which articles belong to a specific group, as seen in chapter 1.

    In this section, we’ll discuss some of the most frequently used commands that operate on SETs. You’ll learn about the standard operations for inserting, removing, and moving members between SETs, as well as commands to perform intersection, union, and differences on SETs. When finished with this section, you’ll be better prepared to fully understand how our search examples in chapter 7 work.

    Let’s take a look at table 3.5 to see some of the more commonly used set commands.

    Table 3.5 Some commonly used SET commands
    Command Example use and description
    SADD SADD key-name item [item …] — Adds the items to the set and returns the number of items added that weren’t already present
    SREM SREM key-name item [item …] — Removes the items and returns the number of items that were removed
    SISMEMBER SISMEMBER key-name item — Returns whether the item is in the SET
    SCARD SCARD key-name — Returns the number of items in the SET
    SMEMBERS SMEMBERS key-name — Returns all of the items in the SET as a Python set
    SRANDMEMBER SRANDMEMBER key-name [count] — Returns one or more random items from the SET. When count is positive, Redis will return count distinct randomly chosen items, and when count is negative, Redis will return count randomly chosen items that may not be distinct.
    SPOP SPOP key-name — Removes and returns a random item from the SET
    SMOVE SMOVE source-key dest-key item — If the item is in the source, removes the item from the source and adds it to the destination, returning if the item was moved

    Some of those commands should be familiar from chapter 1, so let’s jump to the next listing to see some of these commands in action.

    Listing 3.5 A sample interaction showing some common SET commands in Redis
    >>> conn.sadd('set-key', 'a', 'b', 'c')
    3
    

    Adding items to the SET returns the number of items that weren’t already in the SET.

    >>> conn.srem('set-key', 'c', 'd')
    True
    >>> conn.srem('set-key', 'c', 'd')
    False
    

    Removing items from the SET returns whether an item was removed; note that the client is buggy in that respect — Redis itself returns the total number of items removed.

    >>> conn.scard('set-key')
    2
    

    We can get the number of items in the SET.

    >>> conn.smembers('set-key')
    set(['a', 'b'])
    

    We can also fetch the whole SET.

    >>> conn.smove('set-key', 'set-key2', 'a')
    True
    

    We can easily move items from one SET to another SET.

    >>> conn.smove('set-key', 'set-key2', 'c')
    False
    >>> conn.smembers('set-key2')
    set(['a'])
    

    When an item doesn’t exist in the first set during a SMOVE, it isn’t added to the destination SET.

    Using just these commands, we can keep track of unique events and items like we did in chapter 1 with voting and article groups. But the real power of SETs is in the commands that combine multiple SETs at the same time. Table 3.6 shows some of the ways that you can relate multiple SETs to each other.

    Table 3.6 Operations for combining and manipulating SETs in Redis
    Command Example use and description
    SDIFF SDIFF key-name [key-name …] — Returns the items in the first SET that weren’t in any of the other SETs (mathematical set difference operation)
    SDIFFSTORE SDIFFSTORE dest-key key-name [key-name …] — Stores at the dest-key the items in the first SET that weren’t in any of the other SETs (mathematical set difference operation)
    SINTER SINTER key-name [key-name …] — Returns the items that are in all of the SETs (mathematical set intersection operation)
    SINTERSTORE SINTERSTORE dest-key key-name [key-name …] — Stores at the dest-key the items that are in all of the SETs (mathematical set intersection operation)
    SUNION SUNION key-name [key-name …] — Returns the items that are in at least one of the SETs (mathematical set union operation)
    SUNIONSTORE SUNIONSTORE dest-key key-name [key-name …] — Stores at the dest-key the items that are in at least one of the SETs (mathematical set union operation)

    This group of commands are three fundamental SET operations, with both “return the result” and “store the result” versions. Let’s see a sample of what these commands are able to do.

    Listing 3.6 A sample interaction showing SET difference, intersection, and union in Redis
    >>> conn.sadd('skey1', 'a', 'b', 'c', 'd')
    4
    >>> conn.sadd('skey2', 'c', 'd', 'e', 'f')
    4
    

    First we’ll add a few items to a couple of SETs.

    >>> conn.sdiff('skey1', 'skey2')
    set(['a', 'b'])
    

    We can calculate the result of removing all of the items in the second SET from the first SET.

    >>> conn.sinter('skey1', 'skey2')
    set(['c', 'd'])
    

    We can also find out which items exist in both SETs.

    >>> conn.sunion('skey1', 'skey2')
    set(['a', 'c', 'b', 'e', 'd', 'f'])
    

    And we can find out all of the items that are in either of the SETs.

    If you’re comparing with Python sets, Redis SETs offer many of the same semantics and functionality, but are available remotely to potentially many clients. We’ll dig more deeply into what SETs are capable of in chapter 7, where we build a type of search engine with them.

    Coming up next, we’ll talk about commands that manipulate HASHes, which allow us to group related keys and values together for easy fetching and updating.