systemd multi-instance Redis

Instead of working with one redis server instance I would like to use multiple instances of redis because that way I can limit memory and do additional tuning for the instances. systemd can help on this. I will create an instance for amavisd-new in this guide.

Installation

apt-get install redis

Make sure that redis is disabled and redis is not running. Just to make sure issue

systemctl stop redis
systemctl disable redis

Update 15.04.24

I’m leaving this post here for reference / learning purposes. However, I noticed that there is already (To be honest, either I did not check this before or I somehow must have missed it when I wrote this post) a systemd script which does exactly, what I am doing here. So check for:

/usr/lib/systemd/system/redis-server@.service

The file also contains comments on how to set it up. So while my post here still works – you may prefer to go with your distribution supported / provided file instead. Just check the comments in above file if it exists for a explanation on how to set it up.

systemd multi instance template

A unit template is created by adding a file called redis@.service in /usr/local/lib/systemd/system. This unit template allows to start redis@instance-name.service (e.g. redis@amavis). So we can start instances without writing systemd files for them.

Unit

The Unit-Section allows us to set some informative data like description and documentation as well as some things like “Conflicts” and “AssertPathExists”. The former makes sure that the multi-instance stuff won’t be started if redis-server is running (You might not need that line) and the AssertPathExists simply won’t start this service if /etc/redis/redis-%i.conf does not exist. %i is the instance name – so in this example systemd will check if /etc/redis/redis-amavis.conf exists.

[Unit]
Description=Multi-Instance Redis - a fast key value store
Documentation=https://redis.io/documentation man:redis-server(1)
Conflicts=redis-server.service
AssertPathExists=/etc/redis/redis-%i.conf

Service

This service definition is taken from the original redis-server.service file shipped with debian bullseye. I only modified ExecStart to contain redis-%i.conf and PIDFile to use redis-%i.pid. It is possible (additional work though) to modify the ReadWritePaths and the specific redis-instance configuration later to have different log and db paths instead of just different names in the same directory like I do here.

[Service]
# Service taken from default debian systemd redis-server.service
# and modified for %i usage
Type=notify
ExecStart=/usr/bin/redis-server /etc/redis/redis-%i.conf --supervised systemd --daemonize no
PIDFile=/run/redis/redis-%i.pid
TimeoutStopSec=0
Restart=always
User=redis
Group=redis
RuntimeDirectory=redis
RuntimeDirectoryMode=2755

UMask=007
PrivateTmp=yes
LimitNOFILE=65535
PrivateDevices=yes
ProtectHome=yes
ReadOnlyDirectories=/
ReadWritePaths=-/var/lib/redis
ReadWritePaths=-/var/log/redis
ReadWritePaths=-/var/run/redis

NoNewPrivileges=true
CapabilityBoundingSet=CAP_SETGID CAP_SETUID CAP_SYS_RESOURCE
MemoryDenyWriteExecute=true
ProtectKernelModules=true
ProtectKernelTunables=true
ProtectControlGroups=true
RestrictRealtime=true
RestrictNamespaces=true
RestrictAddressFamilies=AF_INET AF_INET6 AF_UNIX

# redis-server can write to its own config file when in cluster mode so we
# permit writing there by default. If you are not using this feature, it is
# recommended that you replace the following lines with "ProtectSystem=full".
ProtectSystem=true
ReadWriteDirectories=-/etc/redis

Install

Finally the install-definition which is also just taken from the original redis-server.service. I believe Alias does not make much sense here, I kept it though.

[Install]
WantedBy=multi-user.target
Alias=redis.service

Redis instance configuration

Basically we will include the original default configuration and just override a few of that settings. You might comment some settings like dir, dbfilename, logfile and pidfile in redis.conf to make sure nobody starts redis-server (which uses redis.conf directly).

By the way. I got this idea of running multiple instances when I read through the rspamd documentation and when I found a step-by-step tutorial about redis-replication / bayesian statistics and fuzzy storage replication with multi-instance Redis backend there. I liked that idea. Unlike that article this guide is for Debian Bullseye (instead of FreeBSD) and this guide also covers systemd.

/etc/redis/redis-amavis.conf

include /etc/redis/redis.conf

port 6377

pidfile /run/redis/redis-amavis.pid
logfile /var/log/redis/redis-amavis.log
dbfilename amavis.rdb
dir /var/lib/redis

maxmemory 300M

Enabling the service

systemctl enable redis@amavis
Created symlink /etc/systemd/system/multi-user.target.wants/redis@amavis.service → /usr/local/lib/systemd/system/redis@.service.

Starting the service

systemctl start redis@amavis
systemctl status redis@amavis
● redis@amavis.service - Multi-Instance Redis - a fast key value store
     Loaded: loaded (/usr/local/lib/systemd/system/redis@.service; enabled; vendor preset: enabled)
     Active: active (running) since Sat 2021-08-21 23:13:21 CEST; 3s ago
       Docs: https://redis.io/documentation
             man:redis-server(1)
   Main PID: 2740 (redis-server)
     Status: "Ready to accept connections"
      Tasks: 5 (limit: 2337)
     Memory: 7.2M
        CPU: 39ms
     CGroup: /system.slice/system-redis.slice/redis@amavis.service
             └─2740 /usr/bin/redis-server 127.0.0.1:6377

Aug 21 23:13:21 mail systemd[1]: Starting Multi-Instance Redis - a fast key value store...
Aug 21 23:13:21 mail systemd[1]: Started Multi-Instance Redis - a fast key value store.

More?

To add more instances all you have to do is to add another redis-INSTANCENAME.conf configuration to /etc/redis (which looks like the redis-amavis.conf above), change the name “amavis” and the port and enable it using systemctl enable redis@INSTANCENAME.

Another example, instancename: spamassassin

Don’t forget to change the port in the configuration file.

root@mail:/etc/redis# cp redis-amavis.conf redis-spamassassin.conf
# replace "amavis" with "spamassassin" using your
# favorite text editor. sed would work, too :)
root@mail:/etc/redis# vim redis-spamassassin.conf 
root@mail:/etc/redis# systemctl enable redis@spamassassin
Created symlink /etc/systemd/system/multi-user.target.wants/redis@spamassassin.service → /usr/local/lib/systemd/system/redis@.service.
root@mail:/etc/redis# systemctl start redis@spamassassin
root@mail:/etc/redis# systemctl status redis@spamassassin
● redis@spamassassin.service - Multi-Instance Redis - a fast key value store
     Loaded: loaded (/usr/local/lib/systemd/system/redis@.service; enabled; vendor preset: enabled)
     Active: active (running) since Sat 2021-08-21 23:44:16 CEST; 17s ago
       Docs: https://redis.io/documentation
             man:redis-server(1)
   Main PID: 2904 (redis-server)
     Status: "Ready to accept connections"
      Tasks: 5 (limit: 2337)
     Memory: 7.2M
        CPU: 62ms
     CGroup: /system.slice/system-redis.slice/redis@spamassassin.service
             └─2904 /usr/bin/redis-server 127.0.0.1:6378

Don’t forget logrotate!

root@mail:/etc/logrotate.d# cat redis-server 
/var/log/redis/redis-server*.log {
        weekly
        missingok
        rotate 12
        compress
        notifempty
}

You may want to add a few definitions for every instance to this file, or you rather create files per instance for logrotate.

One thought on “systemd multi-instance Redis”

David Allouch says:

May 8, 2023 at 15:28

One thing. You need to daemon-reload systemctl before it works.
Beside that… amazing tutorial… thanks.