systemd multi-instance Redis

Instead of working with one redis server instance I would like to use multiple instances of redis because that way I can limit memory and do additional tuning for the instances. systemd can help on this. I will create an instance for amavisd-new in this guide.

Installation

apt-get install redis

Make sure that redis is disabled and redis is not running. Just to make sure issue

systemctl stop redis
systemctl disable redis

systemd multi instance template

A unit template is created by adding a file called redis@.service in /usr/local/lib/systemd/system. This unit template allows to start redis@instance-name.service (e.g. redis@amavis). So we can start instances without writing systemd files for them.

Unit

The Unit-Section allows us to set some informative data like description and documentation as well as some things like “Conflicts” and “AssertPathExists”. The former makes sure that the multi-instance stuff won’t be started if redis-server is running (You might not need that line) and the AssertPathExists simply won’t start this service if /etc/redis/redis-%i.conf does not exist. %i is the instance name – so in this example systemd will check if /etc/redis/redis-amavis.conf exists.

[Unit]
Description=Multi-Instance Redis - a fast key value store
Documentation=https://redis.io/documentation man:redis-server(1)
Conflicts=redis-server.service
AssertPathExists=/etc/redis/redis-%i.conf

Service

This service definition is taken from the original redis-server.service file shipped with debian bullseye. I only modified ExecStart to contain redis-%i.conf and PIDFile to use redis-%i.pid. It is possible (additional work though) to modify the ReadWritePaths and the specific redis-instance configuration later to have different log and db paths instead of just different names in the same directory like I do here.

[Service]
# Service taken from default debian systemd redis-server.service
# and modified for %i usage
Type=notify
ExecStart=/usr/bin/redis-server /etc/redis/redis-%i.conf --supervised systemd --daemonize no
PIDFile=/run/redis/redis-%i.pid
TimeoutStopSec=0
Restart=always
User=redis
Group=redis
RuntimeDirectory=redis
RuntimeDirectoryMode=2755

UMask=007
PrivateTmp=yes
LimitNOFILE=65535
PrivateDevices=yes
ProtectHome=yes
ReadOnlyDirectories=/
ReadWritePaths=-/var/lib/redis
ReadWritePaths=-/var/log/redis
ReadWritePaths=-/var/run/redis

NoNewPrivileges=true
CapabilityBoundingSet=CAP_SETGID CAP_SETUID CAP_SYS_RESOURCE
MemoryDenyWriteExecute=true
ProtectKernelModules=true
ProtectKernelTunables=true
ProtectControlGroups=true
RestrictRealtime=true
RestrictNamespaces=true
RestrictAddressFamilies=AF_INET AF_INET6 AF_UNIX

# redis-server can write to its own config file when in cluster mode so we
# permit writing there by default. If you are not using this feature, it is
# recommended that you replace the following lines with "ProtectSystem=full".
ProtectSystem=true
ReadWriteDirectories=-/etc/redis

Install

Finally the install-definition which is also just taken from the original redis-server.service. I believe Alias does not make much sense here, I kept it though.

[Install]
WantedBy=multi-user.target
Alias=redis.service

Redis instance configuration

Basically we will include the original default configuration and just override a few of that settings. You might comment some settings like dir, dbfilename, logfile and pidfile in redis.conf to make sure nobody starts redis-server (which uses redis.conf directly).

By the way. I got this idea of running multiple instances when I read through the rspamd documentation and when I found a step-by-step tutorial about redis-replication / bayesian statistics and fuzzy storage replication with multi-instance Redis backend there. I liked that idea. Unlike that article this guide is for Debian Bullseye (instead of FreeBSD) and this guide also covers systemd.

/etc/redis/redis-amavis.conf

include /etc/redis/redis.conf

port 6377

pidfile /run/redis/redis-amavis.pid
logfile /var/log/redis/redis-amavis.log
dbfilename amavis.rdb
dir /var/lib/redis

maxmemory 300M

Enabling the service

systemctl enable redis@amavis
Created symlink /etc/systemd/system/multi-user.target.wants/redis@amavis.service → /usr/local/lib/systemd/system/redis@.service.

Starting the service

systemctl start redis@amavis
systemctl status redis@amavis
● redis@amavis.service - Multi-Instance Redis - a fast key value store
     Loaded: loaded (/usr/local/lib/systemd/system/redis@.service; enabled; vendor preset: enabled)
     Active: active (running) since Sat 2021-08-21 23:13:21 CEST; 3s ago
       Docs: https://redis.io/documentation
             man:redis-server(1)
   Main PID: 2740 (redis-server)
     Status: "Ready to accept connections"
      Tasks: 5 (limit: 2337)
     Memory: 7.2M
        CPU: 39ms
     CGroup: /system.slice/system-redis.slice/redis@amavis.service
             └─2740 /usr/bin/redis-server 127.0.0.1:6377

Aug 21 23:13:21 mail systemd[1]: Starting Multi-Instance Redis - a fast key value store...
Aug 21 23:13:21 mail systemd[1]: Started Multi-Instance Redis - a fast key value store.

More?

To add more instances all you have to do is to add another redis-INSTANCENAME.conf configuration to /etc/redis (which looks like the redis-amavis.conf above), change the name “amavis” and the port and enable it using systemctl enable redis@INSTANCENAME.

Another example, instancename: spamassassin

Don’t forget to change the port in the configuration file.

root@mail:/etc/redis# cp redis-amavis.conf redis-spamassassin.conf
# replace "amavis" with "spamassassin" using your
# favorite text editor. sed would work, too :)
root@mail:/etc/redis# vim redis-spamassassin.conf 
root@mail:/etc/redis# systemctl enable redis@spamassassin
Created symlink /etc/systemd/system/multi-user.target.wants/redis@spamassassin.service → /usr/local/lib/systemd/system/redis@.service.
root@mail:/etc/redis# systemctl start redis@spamassassin
root@mail:/etc/redis# systemctl status redis@spamassassin
● redis@spamassassin.service - Multi-Instance Redis - a fast key value store
     Loaded: loaded (/usr/local/lib/systemd/system/redis@.service; enabled; vendor preset: enabled)
     Active: active (running) since Sat 2021-08-21 23:44:16 CEST; 17s ago
       Docs: https://redis.io/documentation
             man:redis-server(1)
   Main PID: 2904 (redis-server)
     Status: "Ready to accept connections"
      Tasks: 5 (limit: 2337)
     Memory: 7.2M
        CPU: 62ms
     CGroup: /system.slice/system-redis.slice/redis@spamassassin.service
             └─2904 /usr/bin/redis-server 127.0.0.1:6378

Don’t forget logrotate!

root@mail:/etc/logrotate.d# cat redis-server 
/var/log/redis/redis-server*.log {
        weekly
        missingok
        rotate 12
        compress
        notifempty
}

You may want to add a few definitions for every instance to this file, or you rather create files per instance for logrotate.

One thought on “systemd multi-instance Redis”

David Allouch says:

May 8, 2023 at 15:28

One thing. You need to daemon-reload systemctl before it works.
Beside that… amazing tutorial… thanks.