6.2.3 Building a lock in Redis

Building a mostly correct lock in Redis is easy. Building a completely correct lock in Redis
isn’t much more difficult, but requires being extra careful about the operations we
use to build it. In this first version, we’re not going to handle the case where a lock
times out, or the case where the holder of the lock crashes and doesn’t release the
lock. Don’t worry; we’ll get to those cases in the next section, but for now, let’s just get
basic locking correct.

The first part of making sure that no other code can run is to acquire the lock. The
natural building block to use for acquiring a lock is the SETNX command, which will
only set a value if the key doesn’t already exist. We’ll set the value to be a unique identifier
to ensure that no other process can get the lock, and the unique identifier we’ll
use is a 128-bit randomly generated UUID.

If we fail to acquire the lock initially, we’ll retry until we acquire the lock, or until a
specified timeout has passed, whichever comes first, as shown here.

def acquire_lock(conn, lockname, acquire_timeout=10):

   identifier = str(uuid.uuid4())

   end = time.time() + acquire_timeout
   while time.time() < end:

      if conn.setnx('lock:' + lockname, identifier):

         return identifier

      time.sleep(.001)

   return False

As described, we’ll attempt to acquire the lock by using SETNX to set the value of the
lock’s key only if it doesn’t already exist. On failure, we’ll continue to attempt this
until we’ve run out of time (which defaults to 10 seconds).

Now that we have the lock, we can perform our buying or selling without WATCH
errors getting in our way. We’ll acquire the lock and, just like before, check the price
of the item, make sure that the buyer has enough money, and if so, transfer the money
and item. When completed, we release the lock. The code for this can be seen next.

def purchase_item_with_lock(conn, buyerid, itemid, sellerid):
   buyer = "users:%s"%buyerid
   seller = "users:%s"%sellerid
   item = "%s.%s"%(itemid, sellerid)
   inventory = "inventory:%s"%buyerid
   end = time.time() + 30

   locked = acquire_lock(conn, market)

      return False

   pipe = conn.pipeline(True)
   try:
      while time.time() < end:
         try:
            pipe.watch(buyer)

            pipe.zscore("market:", item)
            pipe.hget(buyer, 'funds')
            price, funds = pipe.execute()
            if price is None or price > funds:
               pipe.unwatch()
               return None

            pipe.hincrby(seller, int(price))
            pipe.hincrby(buyerid, int(-price))
            pipe.sadd(inventory, itemid)
            pipe.zrem("market:", item)
            pipe.execute()

            return True

         except redis.exceptions.WatchError:
            pass
   finally:

      release_lock(conn, market, locked)

Looking through the code listing, it almost seems like we’re locking the operation. But
don’t be fooled—we’re locking the market data, and the lock must exist while we’re
operating on the data, which is why it surrounds the code performing the operation.
To release the lock, we have to be at least as careful as when acquiring the lock.

Between the time when we acquired the lock and when we’re trying to release it,
someone may have done bad things to the lock. To release the lock, we need to WATCH
the lock key, and then check to make sure that the value is still the same as what we set
it to before we delete it. This also prevents us from releasing a lock multiple times.
The release_lock() function is shown next.

def release_lock(conn, lockname, identifier):
   pipe = conn.pipeline(True)
   lockname = 'lock:' + lockname

   while True:
      try:

         pipe.watch(lockname)
         if pipe.get(lockname) == identifier:

            pipe.multi()
            pipe.delete(lockname)
            pipe.execute()
            return True

         pipe.unwatch()
         break

      except redis.exceptions.WatchError:
         pass
         

   return False

We take many of the same steps to ensure that our lock hasn’t changed as we did with
our money transfer in the first place. But if you think about our release lock function
for long enough, you’ll (reasonably) come to the conclusion that, except in very rare
situations, we don’t need to repeatedly loop. But the next version of the acquire lock
function that supports timeouts, if accidentally mixed with earlier versions (also
unlikely, but anything is possible with code), could cause the release lock transaction
to fail and could leave the lock in the acquired state for longer than necessary. So, just
to be extra careful, and to guarantee correctness in as many situations as possible,
we’ll err on the side of caution.

After we’ve wrapped our calls with locks, we can perform the same simulation of
buying and selling as we did before. In table 6.2, we have new rows that use the lockbased
buying and selling code, which are shown below our earlier rows.

Though we generally have lower total number of items that finished being listed,
we never retry, and our number of listed items compared to our number of purchased

items is close to the ratio of number of listers to buyers. At this point, we’re running at
the limit of contention between the different listing and buying processes.

¹ Having tested a few available Redis lock implementations that include support for timeouts, I was able to
induce lock duplication on at least half of the lock implementations with just five clients acquiring and releasing
the same lock over 10 seconds.