XUtils

Redis

Redis client.

Node.js
https://github.com/luin/ioredis

Versions

Version Branch Node.js Version Redis Version
5.x.x (latest) main >= 12 2.6.12 ~ latest
4.x.x v4 >= 6 2.6.12 ~ 7

Refer to CHANGELOG.md for features and bug fixes introduced in v5.

🚀 Upgrading from v4 to v5

Links

Quick Start

Install

npm install ioredis

In a TypeScript project, you may want to add TypeScript declarations for Node.js:

npm install --save-dev @types/node

Connect to Redis

When a new Redis instance is created, a connection to Redis will be created at the same time. You can specify which Redis to connect to by:

new Redis(); // Connect to 127.0.0.1:6379
new Redis(6380); // 127.0.0.1:6380
new Redis(6379, "192.168.1.1"); // 192.168.1.1:6379
new Redis("/tmp/redis.sock");
new Redis({
  port: 6379, // Redis port
  host: "127.0.0.1", // Redis host
  username: "default", // needs Redis >= 6
  password: "my-top-secret",
  db: 0, // Defaults to 0
});

You can also specify connection options as a redis:// URL or rediss:// URL when using TLS encryption:

// Connect to 127.0.0.1:6380, db 4, using password "authpassword":
new Redis("redis://:authpassword@127.0.0.1:6380/4");

// Username can also be passed via URI.
new Redis("redis://username:authpassword@127.0.0.1:6380/4");

See API Documentation for all available options.

Expiration

Redis can set a timeout to expire your key, after the timeout has expired the key will be automatically deleted. (You can find the official Expire documentation to understand better the different parameters you can use), to set your key to expire in 60 seconds, we will have the following code:

redis.set("key", "data", "EX", 60);
// Equivalent to redis command "SET key data EX 60", because on ioredis set method,
// all arguments are passed directly to the redis server.

Pipelining

If you want to send a batch of commands (e.g. > 5), you can use pipelining to queue the commands in memory and then send them to Redis all at once. This way the performance improves by 50%~300% (See benchmark section).

redis.pipeline() creates a Pipeline instance. You can call any Redis commands on it just like the Redis instance. The commands are queued in memory and flushed to Redis by calling the exec method:

const pipeline = redis.pipeline();
pipeline.set("foo", "bar");
pipeline.del("cc");
pipeline.exec((err, results) => {
  // `err` is always null, and `results` is an array of responses
  // corresponding to the sequence of queued commands.
  // Each response follows the format `[err, result]`.
});

// You can even chain the commands:
redis
  .pipeline()
  .set("foo", "bar")
  .del("cc")
  .exec((err, results) => {});

// `exec` also returns a Promise:
const promise = redis.pipeline().set("foo", "bar").get("foo").exec();
promise.then((result) => {
  // result === [[null, 'OK'], [null, 'bar']]
});

Each chained command can also have a callback, which will be invoked when the command gets a reply:

redis
  .pipeline()
  .set("foo", "bar")
  .get("foo", (err, result) => {
    // result === 'bar'
  })
  .exec((err, result) => {
    // result[1][1] === 'bar'
  });

In addition to adding commands to the pipeline queue individually, you can also pass an array of commands and arguments to the constructor:

redis
  .pipeline([
    ["set", "foo", "bar"],
    ["get", "foo"],
  ])
  .exec(() => {
    /* ... */
  });

#length property shows how many commands in the pipeline:

const length = redis.pipeline().set("foo", "bar").get("foo").length;
// length === 2

Dynamic Keys

If the number of keys can’t be determined when defining a command, you can omit the numberOfKeys property and pass the number of keys as the first argument when you call the command:

redis.defineCommand("echoDynamicKeyNumber", {
  lua: "return {KEYS[1],KEYS[2],ARGV[1],ARGV[2]}",
});

// Now you have to pass the number of keys as the first argument every time
// you invoke the `echoDynamicKeyNumber` command:
redis.echoDynamicKeyNumber(2, "k1", "k2", "a1", "a2", (err, result) => {
  // result === ['k1', 'k2', 'a1', 'a2']
});

As Constructor Options

Besides defineCommand(), you can also define custom commands with the scripts constructor option:

const redis = new Redis({
  scripts: {
    myecho: {
      numberOfKeys: 2,
      lua: "return {KEYS[1],KEYS[2],ARGV[1],ARGV[2]}",
    },
  },
});

Transforming Arguments & Replies

Most Redis commands take one or more Strings as arguments, and replies are sent back as a single String or an Array of Strings. However, sometimes you may want something different. For instance, it would be more convenient if the HGETALL command returns a hash (e.g. { key: val1, key2: v2 }) rather than an array of key values (e.g. [key1, val1, key2, val2]).

ioredis has a flexible system for transforming arguments and replies. There are two types of transformers, argument transformer and reply transformer:

const Redis = require("ioredis");

// Here's the built-in argument transformer converting
// hmset('key', { k1: 'v1', k2: 'v2' })
// or
// hmset('key', new Map([['k1', 'v1'], ['k2', 'v2']]))
// into
// hmset('key', 'k1', 'v1', 'k2', 'v2')
Redis.Command.setArgumentTransformer("hmset", (args) => {
  if (args.length === 2) {
    if (args[1] instanceof Map) {
      // utils is a internal module of ioredis
      return [args[0], ...utils.convertMapToArray(args[1])];
    }
    if (typeof args[1] === "object" && args[1] !== null) {
      return [args[0], ...utils.convertObjectToArray(args[1])];
    }
  }
  return args;
});

// Here's the built-in reply transformer converting the HGETALL reply
// ['k1', 'v1', 'k2', 'v2']
// into
// { k1: 'v1', 'k2': 'v2' }
Redis.Command.setReplyTransformer("hgetall", (result) => {
  if (Array.isArray(result)) {
    const obj = {};
    for (let i = 0; i < result.length; i += 2) {
      obj[result[i]] = result[i + 1];
    }
    return obj;
  }
  return result;
});

There are three built-in transformers, two argument transformers for hmset & mset and a reply transformer for hgetall. Transformers for hmset and hgetall were mentioned above, and the transformer for mset is similar to the one for hmset:

redis.mset({ k1: "v1", k2: "v2" });
redis.get("k1", (err, result) => {
  // result === 'v1';
});

redis.mset(
  new Map([
    ["k3", "v3"],
    ["k4", "v4"],
  ])
);
redis.get("k3", (err, result) => {
  // result === 'v3';
});

Another useful example of a reply transformer is one that changes hgetall to return array of arrays instead of objects which avoids an unwanted conversation of hash keys to strings when dealing with binary hash keys:

Redis.Command.setReplyTransformer("hgetall", (result) => {
  const arr = [];
  for (let i = 0; i < result.length; i += 2) {
    arr.push([result[i], result[i + 1]]);
  }
  return arr;
});
redis.hset("h1", Buffer.from([0x01]), Buffer.from([0x02]));
redis.hset("h1", Buffer.from([0x03]), Buffer.from([0x04]));
redis.hgetallBuffer("h1", (err, result) => {
  // result === [ [ <Buffer 01>, <Buffer 02> ], [ <Buffer 03>, <Buffer 04> ] ];
});

Streamify Scanning

Redis 2.8 added the SCAN command to incrementally iterate through the keys in the database. It’s different from KEYS in that SCAN only returns a small number of elements each call, so it can be used in production without the downside of blocking the server for a long time. However, it requires recording the cursor on the client side each time the SCAN command is called in order to iterate through all the keys correctly. Since it’s a relatively common use case, ioredis provides a streaming interface for the SCAN command to make things much easier. A readable stream can be created by calling scanStream:

const redis = new Redis();
// Create a readable stream (object mode)
const stream = redis.scanStream();
stream.on("data", (resultKeys) => {
  // `resultKeys` is an array of strings representing key names.
  // Note that resultKeys may contain 0 keys, and that it will sometimes
  // contain duplicates due to SCAN's implementation in Redis.
  for (let i = 0; i < resultKeys.length; i++) {
    console.log(resultKeys[i]);
  }
});
stream.on("end", () => {
  console.log("all keys have been visited");
});

scanStream accepts an option, with which you can specify the MATCH pattern, the TYPE filter, and the COUNT argument:

const stream = redis.scanStream({
  // only returns keys following the pattern of `user:*`
  match: "user:*",
  // only return objects that match a given type,
  // (requires Redis >= 6.0)
  type: "zset",
  // returns approximately 100 elements per call
  count: 100,
});

Just like other commands, scanStream has a binary version scanBufferStream, which returns an array of buffers. It’s useful when the key names are not utf8 strings.

There are also hscanStream, zscanStream and sscanStream to iterate through elements in a hash, zset and set. The interface of each is similar to scanStream except the first argument is the key name:

const stream = redis.hscanStream("myhash", {
  match: "age:??",
});

You can learn more from the Redis documentation.

Useful Tips It’s pretty common that doing an async task in the data handler. We’d like the scanning process to be paused until the async task to be finished. Stream#pause() and Stream#resume() do the trick. For example if we want to migrate data in Redis to MySQL:

const stream = redis.scanStream();
stream.on("data", (resultKeys) => {
  // Pause the stream from scanning more keys until we've migrated the current keys.
  stream.pause();

  Promise.all(resultKeys.map(migrateKeyToMySQL)).then(() => {
    // Resume the stream here.
    stream.resume();
  });
});

stream.on("end", () => {
  console.log("done migration");
});

Auto-reconnect

By default, ioredis will try to reconnect when the connection to Redis is lost except when the connection is closed manually by redis.disconnect() or redis.quit().

It’s very flexible to control how long to wait to reconnect after disconnection using the retryStrategy option:

const redis = new Redis({
  // This is the default value of `retryStrategy`
  retryStrategy(times) {
    const delay = Math.min(times * 50, 2000);
    return delay;
  },
});

retryStrategy is a function that will be called when the connection is lost. The argument times means this is the nth reconnection being made and the return value represents how long (in ms) to wait to reconnect. When the return value isn’t a number, ioredis will stop trying to reconnect, and the connection will be lost forever if the user doesn’t call redis.connect() manually.

When reconnected, the client will auto subscribe to channels that the previous connection subscribed to. This behavior can be disabled by setting the autoResubscribe option to false.

And if the previous connection has some unfulfilled commands (most likely blocking commands such as brpop and blpop), the client will resend them when reconnected. This behavior can be disabled by setting the autoResendUnfulfilledCommands option to false.

By default, all pending commands will be flushed with an error every 20 retry attempts. That makes sure commands won’t wait forever when the connection is down. You can change this behavior by setting maxRetriesPerRequest:

const redis = new Redis({
  maxRetriesPerRequest: 1,
});

Set maxRetriesPerRequest to null to disable this behavior, and every command will wait forever until the connection is alive again (which is the default behavior before ioredis v4).

Connection Events

The Redis instance will emit some events about the state of the connection to the Redis server.

Event Description
connect emits when a connection is established to the Redis server.
ready If enableReadyCheck is true, client will emit ready when the server reports that it is ready to receive commands (e.g. finish loading data from disk).
Otherwise, ready will be emitted immediately right after the connect event.
error emits when an error occurs while connecting.
However, ioredis emits all error events silently (only emits when there’s at least one listener) so that your application won’t crash if you’re not listening to the error event.
close emits when an established Redis server connection has closed.
reconnecting emits after close when a reconnection will be made. The argument of the event is the time (in ms) before reconnecting.
end emits after close when no more reconnections will be made, or the connection is failed to establish.
wait emits when lazyConnect is set and will wait for the first command to be called before connecting.

You can also check out the Redis#status property to get the current connection status.

Besides the above connection events, there are several other custom events:

Event Description
select emits when the database changed. The argument is the new db number.

Offline Queue

When a command can’t be processed by Redis (being sent before the ready event), by default, it’s added to the offline queue and will be executed when it can be processed. You can disable this feature by setting the enableOfflineQueue option to false:

const redis = new Redis({ enableOfflineQueue: false });

TLS Profiles

Warning TLS profiles described in this section are going to be deprecated in the next major version. Please provide TLS options explicitly.

To make it easier to configure we provide a few pre-configured TLS profiles that can be specified by setting the tls option to the profile’s name or specifying a tls.profile option in case you need to customize some values of the profile.

Profiles:

  • RedisCloudFixed: Contains the CA for Redis.com Cloud fixed subscriptions
  • RedisCloudFlexible: Contains the CA for Redis.com Cloud flexible subscriptions
const redis = new Redis({
  host: "localhost",
  tls: "RedisCloudFixed",
});

const redisWithClientCertificate = new Redis({
  host: "localhost",
  tls: {
    profile: "RedisCloudFixed",
    key: "123",
  },
});

Read-Write Splitting

A typical redis cluster contains three or more masters and several slaves for each master. It’s possible to scale out redis cluster by sending read queries to slaves and write queries to masters by setting the scaleReads option.

scaleReads is “master” by default, which means ioredis will never send any queries to slaves. There are other three available options:

  1. “all”: Send write queries to masters and read queries to masters or slaves randomly.
  2. “slave”: Send write queries to masters and read queries to slaves.
  3. a custom function(nodes, command): node: Will choose the custom function to select to which node to send read queries (write queries keep being sent to master). The first node in nodes is always the master serving the relevant slots. If the function returns an array of nodes, a random node of that list will be selected.

For example:

const cluster = new Redis.Cluster(
  [
    /* nodes */
  ],
  {
    scaleReads: "slave",
  }
);
cluster.set("foo", "bar"); // This query will be sent to one of the masters.
cluster.get("foo", (err, res) => {
  // This query will be sent to one of the slaves.
});

NB In the code snippet above, the res may not be equal to “bar” because of the lag of replication between the master and slaves.

Running Commands to Multiple Nodes

Every command will be sent to exactly one node. For commands containing keys, (e.g. GET, SET and HGETALL), ioredis sends them to the node that serving the keys, and for other commands not containing keys, (e.g. INFO, KEYS and FLUSHDB), ioredis sends them to a random node.

Sometimes you may want to send a command to multiple nodes (masters or slaves) of the cluster, you can get the nodes via Cluster#nodes() method.

Cluster#nodes() accepts a parameter role, which can be “master”, “slave” and “all” (default), and returns an array of Redis instance. For example:

// Send `FLUSHDB` command to all slaves:
const slaves = cluster.nodes("slave");
Promise.all(slaves.map((node) => node.flushdb()));

// Get keys of all the masters:
const masters = cluster.nodes("master");
Promise.all(
  masters
    .map((node) => node.keys())
    .then((keys) => {
      // keys: [['key1', 'key2'], ['key3', 'key4']]
    })
);

NAT Mapping

Sometimes the cluster is hosted within a internal network that can only be accessed via a NAT (Network Address Translation) instance. See Accessing ElastiCache from outside AWS as an example.

You can specify nat mapping rules via natMap option:

const cluster = new Redis.Cluster(
  [
    {
      host: "203.0.113.73",
      port: 30001,
    },
  ],
  {
    natMap: {
      "10.0.1.230:30001": { host: "203.0.113.73", port: 30001 },
      "10.0.1.231:30001": { host: "203.0.113.73", port: 30002 },
      "10.0.1.232:30001": { host: "203.0.113.73", port: 30003 },
    },
  }
);

This option is also useful when the cluster is running inside a Docker container.

Pub/Sub

Pub/Sub in cluster mode works exactly as the same as in standalone mode. Internally, when a node of the cluster receives a message, it will broadcast the message to the other nodes. ioredis makes sure that each message will only be received once by strictly subscribing one node at the same time.

const nodes = [
  /* nodes */
];
const pub = new Redis.Cluster(nodes);
const sub = new Redis.Cluster(nodes);
sub.on("message", (channel, message) => {
  console.log(channel, message);
});

sub.subscribe("news", () => {
  pub.publish("news", "highlights");
});

Events

Event Description
connect emits when a connection is established to the Redis server.
ready emits when CLUSTER INFO reporting the cluster is able to receive commands (if enableReadyCheck is true) or immediately after connect event (if enableReadyCheck is false).
error emits when an error occurs while connecting with a property of lastNodeError representing the last node error received. This event is emitted silently (only emitting if there’s at least one listener).
close emits when an established Redis server connection has closed.
reconnecting emits after close when a reconnection will be made. The argument of the event is the time (in ms) before reconnecting.
end emits after close when no more reconnections will be made.
+node emits when a new node is connected.
-node emits when a node is disconnected.
node error emits when an error occurs when connecting to a node. The second argument indicates the address of the node.

Password

Setting the password option to access password-protected clusters:

const Redis = require("ioredis");
const cluster = new Redis.Cluster(nodes, {
  redisOptions: {
    password: "your-cluster-password",
  },
});

If some of nodes in the cluster using a different password, you should specify them in the first parameter:

const Redis = require("ioredis");
const cluster = new Redis.Cluster(
  [
    // Use password "password-for-30001" for 30001
    { port: 30001, password: "password-for-30001" },
    // Don't use password when accessing 30002
    { port: 30002, password: null },
    // Other nodes will use "fallback-password"
  ],
  {
    redisOptions: {
      password: "fallback-password",
    },
  }
);

Example of Automatic Pipeline Enqueuing

This sample code uses ioredis with automatic pipeline enabled.

const Redis = require("./built");
const http = require("http");

const db = new Redis({ enableAutoPipelining: true });

const server = http.createServer((request, response) => {
  const key = new URL(request.url, "https://localhost:3000/").searchParams.get(
    "key"
  );

  db.get(key, (err, value) => {
    response.writeHead(200, { "Content-Type": "text/plain" });
    response.end(value);
  });
});

server.listen(3000);

When Node receives requests, it schedules them to be processed in one or more iterations of the events loop.

All commands issued by requests processing during one iteration of the loop will be wrapped in a pipeline automatically created by ioredis.

In the example above, the pipeline will have the following contents:

GET key1
GET key2
GET key3
...
GET keyN

When all events in the current loop have been processed, the pipeline is executed and thus all commands are sent to the server at the same time.

While waiting for pipeline response from Redis, Node will still be able to process requests. All commands issued by request handler will be enqueued in a new automatically created pipeline. This pipeline will not be sent to the server yet.

As soon as a previous automatic pipeline has received all responses from the server, the new pipeline is immediately sent without waiting for the events loop iteration to finish.

This approach increases the utilization of the network link, reduces the TCP overhead and idle times and therefore improves throughput.

Benchmarks

Here’s some of the results of our tests for a single node.

Each iteration of the test runs 1000 random commands on the server.

Samples Result Tolerance
default 1000 174.62 op/sec ± 0.45 %
enableAutoPipelining=true 1500 233.33 op/sec ± 0.88 %

And here’s the same test for a cluster of 3 masters and 3 replicas:

Samples Result Tolerance
default 1000 164.05 op/sec ± 0.42 %
enableAutoPipelining=true 3000 235.31 op/sec ± 0.94 %

Error Handling

All the errors returned by the Redis server are instances of ReplyError, which can be accessed via Redis:

const Redis = require("ioredis");
const redis = new Redis();
// This command causes a reply error since the SET command requires two arguments.
redis.set("foo", (err) => {
  err instanceof Redis.ReplyError;
});

This is the error stack of the ReplyError:

ReplyError: ERR wrong number of arguments for 'set' command
    at ReplyParser._parseResult (/app/node_modules/ioredis/lib/parsers/javascript.js:60:14)
    at ReplyParser.execute (/app/node_modules/ioredis/lib/parsers/javascript.js:178:20)
    at Socket.<anonymous> (/app/node_modules/ioredis/lib/redis/event_handler.js:99:22)
    at Socket.emit (events.js:97:17)
    at readableAddChunk (_stream_readable.js:143:16)
    at Socket.Readable.push (_stream_readable.js:106:10)
    at TCP.onread (net.js:509:20)

By default, the error stack doesn’t make any sense because the whole stack happens in the ioredis module itself, not in your code. So it’s not easy to find out where the error happens in your code. ioredis provides an option showFriendlyErrorStack to solve the problem. When you enable showFriendlyErrorStack, ioredis will optimize the error stack for you:

const Redis = require("ioredis");
const redis = new Redis({ showFriendlyErrorStack: true });
redis.set("foo");

And the output will be:

ReplyError: ERR wrong number of arguments for 'set' command
    at Object.<anonymous> (/app/index.js:3:7)
    at Module._compile (module.js:446:26)
    at Object.Module._extensions..js (module.js:464:10)
    at Module.load (module.js:341:32)
    at Function.Module._load (module.js:296:12)
    at Function.Module.runMain (module.js:487:10)
    at startup (node.js:111:16)
    at node.js:799:3

This time the stack tells you that the error happens on the third line in your code. Pretty sweet! However, it would decrease the performance significantly to optimize the error stack. So by default, this option is disabled and can only be used for debugging purposes. You shouldn’t use this feature in a production environment.

Running tests

Start a Redis server on 127.0.0.1:6379, and then:

npm test

FLUSH ALL will be invoked after each test, so make sure there’s no valuable data in it before running tests.

If your testing environment does not let you spin up a Redis server ioredis-mock is a drop-in replacement you can use in your tests. It aims to behave identically to ioredis connected to a Redis server so that your integration tests is easier to write and of better quality.

Debug

You can set the DEBUG env to ioredis:* to print debug info:

$ DEBUG=ioredis:* node app.js

Join in!

I’m happy to receive bug reports, fixes, documentation enhancements, and any other improvements.

And since I’m not a native English speaker, if you find any grammar mistakes in the documentation, please also let me know. :)

Contributors

This project exists thanks to all the people who contribute:

Articles

  • coming soon...