Docker Update Automation: Advanced Bash Pipelining. paperless-ngx

This article documents a reliable update script for the Paperless-NGX stack, which minimizes the risk of container failures during automated maintenance. The focus here is not just on simple automation, but on ensuring the integrity of the process—especially handling logs and exit codes within complex Bash pipelines.

Part I: Defining the Problem (The Log and Exit Code Dilemma)

The initial simple script worked, but it suffered from two critical flaws that make it unsuitable for production cron jobs:

1. Inaccurate Timestamp: The start and end time logged was identical, as the $DATE variable was only defined once at the script’s initiation.
2. Broken Exit Codes (The Fatal Flaw): Commands inside a pipe (|) often run in a subshell. If docker compose down fails, the pipe’s overall exit code ($?) often reflects the status of the final command (e.g., while read), hiding the initial failure. This means the script might proceed with docker compose pull even if the service failed to stop.

Part II: Solution – Hardening the Bash Pipeline

To create a production-ready script, I implement advanced Bash features to guarantee reliable command execution and accurate logging.

1. The wlog Function (Adding Timestamps and Centralizing Output)

The wlog function is introduced to wrap commands, timestamp the output of every line, and consolidate stdout and stderr (2>&1), enabling central logging.

wlog () {
  local cmd="$@"
  # Redirects command output through the pipeline
  $cmd 2>&1 | while read -r l; do d=`date`; echo "$d: $l"; done
} 

2. Resolving Exit Codes and Pipeline Integrity

The failure of the initial script to correctly capture the exit code is solved by enabling two shell options, which are available since Bash 4.2:

# Required for reliable pipelines (available Bash 4.2+)
shopt -s lastpipe
shopt -so pipefail

• shopt -s lastpipe: Forces the last segment of the pipe (while read) to run in the current shell, allowing $? to be reliably checked.
• shopt -so pipefail: Ensures the exit code of the pipeline is that of the first command that failed (this is critical for safe automation).

Part III: The Final Automation Script

The final script applies these techniques, ensuring that docker compose pull only executes if docker compose down was successful (&& operator).

#!/bin/bash

set -e
shopt -s lastpipe
shopt -so pipefail

PDIR=/opt/paperless/paperless-ngx
LOG=/opt/paperless/docker-compose-cron.log

wlog () {
  local cmd="$@"
  $cmd 2>&1 | while read -r l; do d=`date`; echo "$d: $l"; done
}

wlog echo "Starting Docker Compose Update" >> $LOG
cd $PDIR

# 1. Stop and pull only if successful
wlog /usr/bin/docker compose down >> $LOG && wlog /usr/bin/docker compose pull >> $LOG

# 2. Start all containers
wlog /usr/bin/docker compose up --wait -d >> $LOG

wlog echo "Finished Docker Compose Update" >> $LOG

Verification (Log Output)

The log output now provides precise timestamps for every step of the Docker Compose operation, fulfilling the Observability requirement.

Wed Feb 21 21:29:45 CET 2024: Container paperless-webserver-1  Stopping
Wed Feb 21 21:29:53 CET 2024: Container paperless-webserver-1  Stopped
...
Wed Feb 21 21:30:03 CET 2024: Network paperless_default  Removed
...

Part IV: Conclusion and Alternatives

This solution provides reliable automation using pure Bash. However, be aware that solutions like Docker Watchtower may offer a simpler, container-native approach if complex exit code logic is not required.

Sources / See Also

1. GNU Bash Reference Manual. Shell Options for Pipeline Management (shopt -s lastpipe, pipefail). https://www.gnu.org/software/bash/manual/bash.html
2. Docker Documentation. Docker Compose Upgrade and Maintenance. https://docs.docker.com/compose/compose-file/08-upgrade/
3. Docker Documentation. Reference for Docker Compose CLI commands (down, pull, up). https://docs.docker.com/compose/reference/overview/
4. Linux Manpage: date. Usage of the date command for precise timestamping.
5. Linux Manpage: cron. Syntax and execution environment for automated job scheduling.

---