Try Redis Cloud Essentials for Only $5/Month!

Learn More

11.1.1 Loading Lua scripts into Redis

back to home

11.1.1 Loading Lua scripts into Redis

Some older (and still used) Python Redis libraries for Redis 2.6 don’t yet offer the
capability to load or execute Lua scripts directly, so we’ll spend a few moments to create
a loader for the scripts. To load scripts into Redis, there’s a two-part command
called SCRIPT LOAD that, when provided with a string that’s a Lua script, will store the
script for later execution and return the SHA1 hash of the script. Later, when we want
to execute that script, we run the Redis command EVALSHA with the hash that was
returned by Redis, along with any arguments that the script needs.

Our code for doing these operations will be inspired by the current Python Redis
code. (We use our method primarily because it allows for using any connection we
want without having to explicitly create new scripting objects, which can be useful
when dealing with server sharding.) When we pass a string to our script_load()
function, it’ll create a function that can later be called to execute the script in Redis.
When calling the object to execute the script, we must provide a Redis connection,
which will then call SCRIPT LOAD on its first call, followed by EVALSHA for all future
calls. The script_load() function is shown in the following listing.

Listing 11.1A function that loads scripts to be called later
def script_load(script):
    sha = [None]

Store the cached SHA1 hash of the result of SCRIPT LOAD in a list so we can change it later from within the call() function.

    def call(conn, keys=[], args=[], force_eval=False):

When calling the loaded script, we must provide a connection, a set of keys that the script will manipulate, and any other arguments to the function.

            if not force_eval:
                if not sha[0]:

We’ll only try loading the script if the SHA1 hash isn’t cached.

                        sha[0] = conn.execute_command(
                            "SCRIPT", "LOAD", script, parse="LOAD")

                try:
                            return conn.execute_command(
                               "EVALSHA", sha[0], len(keys), *(keys+args))

Execute the command from the cached SHA1.

                        except redis.exceptions.ResponseError as msg:
                            if not msg.args[0].startswith("NOSCRIPT"):
                                raise

If the error was unrelated to a missing script, raise the exception again.
If we received a script-related error, or if we need to force-execute the script, directly execute the script, which will automatically cache the script on the server (with the same SHA1 that we’ve already cached) when done.

                return conn.execute_command(
                    "EVAL", script, len(keys), *(keys+args))
            return call

Return the function that automatically loads and executes scripts when called.

You’ll notice that in addition to our SCRIPT LOAD and EVALSHA calls, we captured an
exception that can happen if we’ve cached a script’s SHA1 hash locally, but the server
doesn’t know about it. This can happen if the server were to be restarted, if someone
had executed the SCRIPT FLUSH command to clean out the script cache, or if we provided
connections to two different Redis servers to our function at different times. If
we discover that the script is missing, we execute the script directly with EVAL, which
caches the script in addition to executing it. Also, we allow clients to directly execute
the script, which can be useful when executing a Lua script as part of a transaction or
other pipelined sequence.

KEYS AND ARGUMENTS TO LUA SCRIPTSBuried inside our script loader, you
may have noticed that calling a script in Lua takes three arguments. The first
is a Redis connection, which should be standard by now. The second argument
is a list of keys. The third is a list of arguments to the function.

The difference between keys and arguments is that you’re supposed to pass
all of the keys that the script will be reading or writing as part of the keys
argument. This is to potentially let other layers verify that all of the keys are
on the same shard if you were using multiserver sharding techniques like
those described in chapter 10.

When Redis cluster is released, which will offer automatic multiserver sharding,
keys will be checked before a script is run, and will return an error if any
keys that aren’t on the same server are accessed.

The second list of arguments has no such limitation and is meant to hold data
to be used inside the Lua call.

Let’s try it out in the console for a simple example to get started.

>>> ret_1 = script_load("return 1")

Most uses will load the script and store a reference to the returned function.

>>> ret_1(conn)

We can then call the function by passing the connection object and any desired arguments.

1L

Results will be returned and converted into the appropriate Python types, when possible.

As you can see in this example, we created a simple script whose only purpose is to
return a value of 1. When we call it with the connection object, the script is loaded
and then executed, resulting in the value 1 being returned.

RETURNING NON-STRING AND NON-INTEGER VALUES FROM LUA

Due to limitations in how Lua allows data to be passed in and out of it, some data types
that are available in Lua aren’t allowed to be passed out, or are altered before being
returned. Table 11.1 shows how this data is altered upon return.

Table 11.1Values returned from Lua and what they’re translated into

Lua value

What happens during conversion to Python

true

Turns into 1

false

Turns into None

nil

Doesn’t turn into anything, and stops remaining values in a table from being returned

1.5 (or any other float)

Fractional part is discarded, turning it into an integer

1e30 (or any other large float)

Is turned into the minimum integer for your version of Python

"strings"

Unchanged

1 (or any other integer +/-253-1)

Integer is returned unchanged

Because of the ambiguity that results when returning a variety of data types, you
should do your best to explicitly return strings whenever possible, and perform any
parsing manually. We’ll only be returning Booleans, strings, integers, and Lua tables
(which are turned into Python lists) for our examples.

Now that we can load and execute scripts, let’s get started with a simple example
from chapter 8, creating a status message.