Nextcloud with S3 using rclone

I had trouble with Nextcloud’s own implementation (external storage s3) as it would corrupt some of my photos on auto upload using the Android Nextcloud Client. This did not happen with s3fs, goofys, rclone and also did not happen without s3. Disabling chunking, tuning timeouts, nothing really helped. After testing with goofys, s3fs and rclone I decided to use rclone. Here’s how I did…

Rclone can be used to mount cloud storage as a network device. Whether it is a good idea to use object storage as block storage is another topic. I would – whenever possible – prefer a native s3 implementation over mounting s3 as block storage. This being said…

Install rclone

That’s as simple as:

apt-get install rclone

Also create a rclone configuration folder

mkdir /etc/rclone

Configure and mount

You can mount S3 using rclone for example like this:

/usr/bin/rclone mount minio:bucketname mountpoint options

You can save the credentials in /etc/rclone/rclone.conf and let the environment variable RCLONE_CONFIG point to it. For minio this could look like this:

[minio]
type = s3
region = somewhere-over-the-rainbow
endpoint = http://127.0.0.1:9000
provider = Minio 
env_auth = false
access_key_id = ...
secret_access_key = ...
acl = bucket-owner-full-control

Where to mount it?

There are multiple options on how to mount your cloud storage with rclone into Nextcloud. I saw some who use the data directory directly e.g. if your nextcloud installation is in /var/www/nextcloud and your data directory is /var/www/nextcloud/data then they would mount it there. However:

root@nc:/var/www/nextcloud/data# ls
appdata_xxx  files_external  index.html  jean  updater.log  updater-xxx

The appdata folder contains 2.4 GB of data in my case, the updater folder contains additional 2.3 GB of data. And I really don’t need these two in S3 (maybe you do). Then maybe mounting it to /var/www/nextcloud/data/jean would be the right thing to do, but let’s see:

root@nc:/var/www/nextcloud/data/jean# ls
files  files_trashbin

I also do not think that putting the trashbin into s3 is a good choice. Furthermore also temporary uploads might end in that directory depending on your configuration. Hence I use /var/www/nextcloud/data/jean/files as mountpoint. How about multiple users? Similar to my multi-redis systemd service file I use a multi-instance rclone systemd service file. I just have to enable it whenever I add a new bucket and a new user.

Multi-instance systemd service file

First I define a Unit-Section with the usual information like description, documentation, and so on. I use AssertPathExists to make sure the specific configuration exists beforehand. I use %i for this. It will later be used for the username. E.g. systemctl enable rclone@jean would enable the rclone mount for the user jean which will use /etc/rclone/rclone-jean.conf to mount everything to /var/www/nextcloud/data/jean/files.

I also tell it that I need it after the network is online and I need it before nginx. Because I am using Nginx with my Nextcloud. You may want to replace nginx.service with apache2.service or httpd.service or something like this. I also use RequiresMountsFor so that everything which might need to be mounted beforehand, is mounted.

[Unit]
Description=rclone - s3 mount for nextcloud %i data
Documentation=https://rclone.org/ man:rclone(1)
AssertPathExists=/etc/rclone/rclone-%i.conf
RequiresMountsFor=/var/www/nextcloud/data
Before=nginx.service
After=network-online.target
Wants=network-online.target

The next step is to define the Service itself. I decided that all buckets which I use for nextcloud start with “nextcloud” and end with the user. So in my case this would be nextcloud-jean:

[Service]
Type=notify
Environment=RCLONE_CONFIG=/etc/rclone/rclone-%i.conf
ExecStart=/usr/bin/rclone \
    mount minio:nextcloud-%i /var/www/nextcloud/data/%i/files \
    --allow-other \
    --vfs-cache-mode writes \
    --log-level INFO \
    --log-file /var/log/rclone/rclone-%i.log \
    --umask 002
ExecStop=/bin/fusermount -uz /var/www/nextcloud/data/%i/files
Restart=on-failure
User=www-data
Group=www-data

In my case everything will use www-data:www-data if you have another user for nextcloud you will need to change this accordingly. You also need to create /var/log/rclone beforehand and chown it to www-data:www-data.

To finalize the above the [Unit] entry is needed:

[Install]
WantedBy=default.target

Reload Systemd-Scripts, Enable it, Start it and check the status:

systemctl daemon-reload
systemctl enable rclone@jean
systemctl start rclone@jean

Try a reboot, just to make sure that it will mount the directory correctly before the webserver gets up. A systemctl status rclone@jean a few hours later should look similar to this:

● rclone@jean.service - rclone - s3 mount for nextcloud jean data
     Loaded: loaded (/etc/systemd/system/rclone@.service; enabled; preset: enabled)
     Active: active (running) since Sun 2024-04-07 04:30:55 CEST; 13h ago
       Docs: https://rclone.org/
             man:rclone(1)
   Main PID: 4105 (rclone)
     Status: "[17:53] vfs cache: objects 21 (was 21) in use 0, to upload 0, uploading 0, total size 895.986Mi (was 895.986Mi)"
      Tasks: 12 (limit: 4685)
     Memory: 124.7M
        CPU: 1min 39.028s
     CGroup: /system.slice/system-rclone.slice/rclone@jean.service
             └─4105 /usr/bin/rclone mount minio:nextcloud-jean /var/www/nextcloud/data/jean/files --allow-other --buffer-size 128M --log-level DEBUG --log-file /var/log/rclone/rclone-jean.log --umask 002 --vfs>

Apr 07 04:30:55 nc systemd[1]: Starting rclone@jean.service - rclone - s3 mount for nextcloud jean data...

The mount options

I’m still experimenting somewhat with the settings, but the following seems to work fine so far.

   --allow-other \
   --log-level DEBUG \
   --log-file /var/log/rclone/rclone-%i.log \
   --umask 002 \
   --use-server-modtime \
   --transfers 16 \
   --cache-dir /var/www/cache/rclone/%i \
   --vfs-cache-mode full \
   --vfs-fast-fingerprint \
   --vfs-read-chunk-size 8M \
   --vfs-read-chunk-size-limit off \
   --vfs-read-ahead 256M \
   --vfs-cache-max-age 168h \
   --vfs-cache-max-size 15G

You can read a lot about this in the rclone mount documentation.

And that’s it basically.