5.1 Logging to Redis

  • Redis in Action – Home
  • Foreword
  • Preface
  • Part 1: Getting Started
  • 1.1.1 Redis compared to other databases and software
  • 1.1.3 Why Redis?
  • 1.3.2 Posting and fetching articles
  • 1.3.3 Grouping articles
  • 2.1 Login and cookie caching
  • 2.2 Shopping carts in Redis
  • 2.3 Web page caching
  • 2.5 Web page analytics
  • 3.1 Strings
  • 3.2 Lists
  • 3.4 Hashes
  • 3.5 Sorted sets
  • 3.7.1 Sorting
  • 3.7.2 Basic Redis transactions
  • 3.7.3 Expiring keys
  • 4.4 Redis transactions
  • 4.5 Non-transactional pipelines
  • 4.7 Summary
  • 5.1 Logging to Redis
  • 5.2 Counters and statistics
  • 5.2.2 Storing statistics in Redis
  • 5.4.1 Using Redis to store configuration information
  • 5.4.2 One Redis server per application component
  • 5.4.3 Automatic Redis connection management
  • 6.1 Autocomplete
  • 6.2 Distributed locking
  • 6.3 Counting semaphores
  • 6.6 Distributing files with Redis
  • 6.2.1 Why locks are important
  • 6.2.2 Simple locks
  • 6.2.3 Building a lock in Redis
  • 6.2.5 Locks with timeouts
  • 6.3.2 Fair semaphores
  • 6.3.4 Preventing race conditions
  • 6.5.1 Single-recipient publish/subscribe replacement
  • 6.5.2 Multiple-recipient publish/subscribe replacement
  • 6.6.2 Sending files
  • 7.1 Searching in Redis
  • 7.2 Sorted Indexes
  • 7.5 Summary
  • 7.1.2 Sorting search results
  • 8.1.1 User information
  • 8.5.1 Data to be streamed
  • 9.2.2 SETs
  • 9.4 Summary
  • Chapter 11: Scripting Redis with Lua
  • 11.1.1 Loading Lua scripts into Redis
  • 11.2 Rewriting locks and semaphores with Lua
  • 11.5 Summary
  • 11.2.1 Why locks in Lua?
  • 11.2.2 Rewriting our lock
  • B.1 Forums for help
  • Buy the paperback
  • Redis in Action – Home
  • Foreword
  • Preface
  • Part 1: Getting Started
  • 1.1.1 Redis compared to other databases and software
  • 1.1.3 Why Redis?
  • 1.3.2 Posting and fetching articles
  • 1.3.3 Grouping articles
  • 2.1 Login and cookie caching
  • 2.2 Shopping carts in Redis
  • 2.3 Web page caching
  • 2.5 Web page analytics
  • 3.1 Strings
  • 3.2 Lists
  • 3.4 Hashes
  • 3.5 Sorted sets
  • 3.7.1 Sorting
  • 3.7.2 Basic Redis transactions
  • 3.7.3 Expiring keys
  • 4.4 Redis transactions
  • 4.5 Non-transactional pipelines
  • 4.7 Summary
  • 5.1 Logging to Redis
  • 5.2 Counters and statistics
  • 5.2.2 Storing statistics in Redis
  • 5.4.1 Using Redis to store configuration information
  • 5.4.2 One Redis server per application component
  • 5.4.3 Automatic Redis connection management
  • 6.1 Autocomplete
  • 6.2 Distributed locking
  • 6.3 Counting semaphores
  • 6.6 Distributing files with Redis
  • 6.2.1 Why locks are important
  • 6.2.2 Simple locks
  • 6.2.3 Building a lock in Redis
  • 6.2.5 Locks with timeouts
  • 6.3.2 Fair semaphores
  • 6.3.4 Preventing race conditions
  • 6.5.1 Single-recipient publish/subscribe replacement
  • 6.5.2 Multiple-recipient publish/subscribe replacement
  • 6.6.2 Sending files
  • 7.1 Searching in Redis
  • 7.2 Sorted Indexes
  • 7.5 Summary
  • 7.1.2 Sorting search results
  • 8.1.1 User information
  • 8.5.1 Data to be streamed
  • 9.2.2 SETs
  • 9.4 Summary
  • Chapter 11: Scripting Redis with Lua
  • 11.1.1 Loading Lua scripts into Redis
  • 11.2 Rewriting locks and semaphores with Lua
  • 11.5 Summary
  • 11.2.1 Why locks in Lua?
  • 11.2.2 Rewriting our lock
  • B.1 Forums for help
  • Buy the paperback

    5.1 Logging to Redis

    As we build applications and services, being able to discover information about the running system becomes increasingly important. Being able to dig into that information to diagnose problems, discover problems before they become severe, or even just to discover information about users—these all necessitate logging.

    In the world of Linux and Unix, there are two common logging methods. The first is logging to a file, where over time we write individual log lines to a file, and every once in a while, we write to a new file. Many thousands of pieces of software have been written do this (including Redis itself). But this method can run into issues because we have many different services writing to a variety of log files, each with a different way of rolling them over, and no common way of easily taking all of the log files and doing something useful with them.

    Running on TCP and UDP port 514 of almost every Unix and Linux server available is a service called syslog, the second common logging method. Syslog accepts log messages from any program that sends it a message and routes those messages to various on-disk log files, handling rotation and deletion of old logs. With configuration, it can even forward messages to other servers for further processing. As a service, it’s far more convenient than logging to files directly, because all of the special log file rotation and deletion is already handled for us.

    REPLACING SYSLOGWhether you end up using the logging methods described here, you owe it to yourself to consider replacing your current syslog daemon (which is likely Rsyslogd) with syslog-ng. Having used and configured both systems, I find that the configuration language that syslog-ng provides for directing log messages is easier to use. And though I don’t have the space or time to build it in this book, building a service that consumes syslog messages and puts them into Redis is a great way to offer a layer of indirection between what needs to happen now for processing a request, and what can happen later (like logging or updating counters).

    Having logs available in files on a single server (thanks to syslog forwarding) is a great long-term plan with logging (remember to back them up). In this section, we’ll talk about using Redis as a way of keeping more time-sensitive logs, to function as a replacement for syslog messages being stored in the short term. Our first view into changing logs is a continuously updated stream of recent log messages.