Nginx Dynamic Modules: Automating Recompilation with APT Hooks

If you’ve ever dealt with Nginx and its dynamic modules, you know the drill. An Nginx package update hits, and suddenly your custom modules – like ModSecurity or GeoIP2 – are no longer compatible. The whole process is a headache: you have to stop Nginx, recompile your modules against the new version, copy the files, and restart the service.

I was looking for a way to automate this. The goal was simple: ensure that dynamic modules are always compatible with a new Nginx version. And if the recompilation fails for any reason, the entire Nginx update must be aborted.

The Challenge: Choosing the Right Hook

My initial thought was to use a simple dpkg hook (pre-install.d/). It seemed logical: the hook runs right before the package is unpacked. However, for some reason it did not work at all; I was unable to find a reason for this but the script was not run at all.

The solution was to switch to an APT Hook, specifically DPkg::Pre-Invoke. This hook triggers reliably before the dpkg transaction even starts. If the hook’s script exits with a non-zero code, the entire apt command fails. This is exactly the kind of fail-safe mechanism I needed to guarantee the Nginx update would be blocked if anything went wrong.

# /etc/apt/apt.conf.d/10nginx-pre-invoke
DPkg::Pre-Invoke { "/usr/local/bin/nginx-pre-invoke-hook.sh"; };

The Once And Only Once Principle (OAOO)

As I was building the script, I knew I wanted to avoid hardcoding and make it easy to manage new modules in the future. I’m a big fan of the “Once And Only Once” (OAOO) principle.

My solution: I used a Bash array called MODULE_REPOS. Now, to add a new module, I just add its Git URL to the array. The script handles the rest automatically—cloning, pulling, compiling, and copying. No more repetitive code blocks for each module.

MODULE_REPOS=(
    "https://github.com/owasp-modsecurity/ModSecurity-nginx.git"
    "https://github.com/leev/ngx_http_geoip2_module.git"
)

I also built some helper functions, like run_command_quietly and manage_repo, to streamline the code. These functions handle common tasks like error checking, debugging, and Git operations, which makes the main script clean and easy to read.

run_command_quietly() {
    local cmd=("$@")
    if [ "$DEBUG" = "false" ]; then
        if ! "${cmd[@]}" &>/dev/null; then
            return 1
        fi
    else
        if ! "${cmd[@]}"; then
            return 1
        fi
    fi
}

manage_repo() {
    local repo_url="$1"
    local local_repo_name="$2"
    local repo_path="$BASE_MODULE_DIR/$local_repo_name" 

    if [ ! -d "$repo_path" ]; then
        echo "$local_repo_name repository not found. Cloning into $repo_path..." >&2 
        if ! run_command_quietly git clone "$repo_url" "$repo_path"; then
            echo "ERROR: Git clone failed for $local_repo_name. Aborting update." >&2
            exit 1
        fi
    else
        echo "Updating $local_repo_name repository..." >&2
        if ! (cd "$repo_path" && run_command_quietly git pull); then
            echo "ERROR: Git pull failed for $local_repo_name. Aborting update." >&2
            exit 1
        fi
    fi
    echo "$repo_path" # ONLY the path is sent to STDOUT for capture
}

Critical Lessons and Fixes

Building a production-ready script isn’t just about writing code; it’s about hardening it against failure. I learned a few important lessons along the way:

  • The Nginx -t test: I noticed this one without trying first but I think it’s worth to mention: My first version included an nginx -t test. The pre-invoke environment only sees the old Nginx binary, not the new one that’s about to be installed. The version mismatch would cause the test to fail, incorrectly blocking a perfectly valid update. I had to remove it.
  • Redirecting status messages: When using command substitution, any output to STDOUT gets captured. My manage_repo function was echoing status messages, which was corrupting the module path variable. The fix? Redirecting all status messages to STDERR (>&2).
  • Safe cleanup: You can’t be too careful with rm -rf. I hardened the cleanup command by prefixing it with /tmp/. I also made sure the script changes out of the temporary directory before attempting to delete it. This prevents (hopefully) accidental deletion of critical system files.

Putting It All Together: The Code and Configuration

Here’s the final solution. The script itself is a single, self-contained file that lives in /usr/local/bin/.

1. The Script (nginx-pre-invoke-hook.sh)

The script’s logic is straightforward:

  • Check if the Nginx package is scheduled for an upgrade or reinstall.
  • Determine the new Nginx version number from the APT package cache.
  • Download and unpack the Nginx source code.
  • Loop through the MODULE_REPOS array to clone or update each module’s Git repository.
  • Dynamically build the ./configure command with the correct module paths.
  • Run ./configure and make modules.
  • Copy the newly compiled .so files to /etc/nginx/modules/.
  • If any of the critical steps fail, the script exits with code 1, stopping the entire apt process.
#!/bin/bash
# Nginx Dynamic Module Pre-Invoke Hook
# Runs BEFORE any package operation begins.
# Always recompiles and copies modules when Nginx is updated, ensuring compatibility.
# Aborts the entire 'apt' operation if module compilation fails.

set -e

# --- CONFIGURATION ---

DEBUG="false"
BASE_MODULE_DIR="/usr/src/dynamic-nginx-modules"
NGINX_MODULE_PATH="/etc/nginx/modules/"
NGINX_PACKAGE_NAME="nginx"

# Array of module repository URLs.
MODULE_REPOS=(
    "https://github.com/owasp-modsecurity/ModSecurity-nginx.git"
    "https://github.com/leev/ngx_http_geoip2_module.git"
)

# --- HELPER FUNCTIONS ---

# Runs a command and suppresses output if DEBUG=false.
run_command_quietly() {
    local cmd=("$@")
    if [ "$DEBUG" = "false" ]; then
        if ! "${cmd[@]}" &>/dev/null; then
            return 1
        fi
    else
        if ! "${cmd[@]}"; then
            return 1
        fi
    fi
}

# Clones or pulls a Git repository.
manage_repo() {
    local repo_url="$1"
    local local_repo_name="$2"
    local repo_path="$BASE_MODULE_DIR/$local_repo_name"

    if [ ! -d "$repo_path" ]; then
        echo "$local_repo_name repository not found. Cloning into $repo_path..." >&2
        if ! run_command_quietly git clone "$repo_url" "$repo_path"; then
            echo "ERROR: Git clone failed for $local_repo_name. Aborting update." >&2
            exit 1
        fi
    else
        echo "Updating $local_repo_name repository..." >&2
        if ! (cd "$repo_path" && run_command_quietly git pull); then
            echo "ERROR: Git pull failed for $local_repo_name. Aborting update." >&2
            exit 1
        fi
    fi
    echo "$repo_path" # ONLY the path is sent to STDOUT for capture
}

# --- MAIN EXECUTION START ---

echo "--- Starting Nginx Pre-Invoke Check (Package: $NGINX_PACKAGE_NAME) ---"

# 1. CHECK FOR UPGRADE/REINSTALL
NGINX_IS_UPGRADABLE=$(apt list --upgradable 2>/dev/null | grep -q "^$NGINX_PACKAGE_NAME/"; echo $?)
NGINX_IS_INSTALLED=$(dpkg -s $NGINX_PACKAGE_NAME 2>/dev/null | grep -q "Status: install ok installed"; echo $?)

if [ "$NGINX_IS_UPGRADABLE" -ne 0 ] && [ "$NGINX_IS_INSTALLED" -ne 0 ]; then
    echo "Nginx is neither scheduled for upgrade nor currently installed. Exiting hook."
    exit 0
fi

echo "Nginx package action (Upgrade or Reinstall) detected. Starting dynamic module recompilation."

# Determine the target Nginx version
NGINX_FULL_VERSION=$(apt-cache policy $NGINX_PACKAGE_NAME | awk '/Candidate/{print $2}')

if [ -z "$NGINX_FULL_VERSION" ]; then
    echo "ERROR: Could not determine the Nginx target version." >&2
    exit 1
fi

NGINX_CORE_VERSION=$(echo "$NGINX_FULL_VERSION" | sed -E 's/([0-9]+\.[0-9]+\.[0-9]+).*/\1/')

if [ "$DEBUG" = "true" ]; then
    echo "Target Nginx Version (Core): $NGINX_CORE_VERSION"
fi

# 2. PREPARING NGINX SOURCE
echo "Preparing Nginx source code (Downloading/Unpacking)..."

TEMP_FOLDER_NAME="nginx_src_$NGINX_CORE_VERSION"
TEMP_PATH="$TEMP_FOLDER_NAME"

mkdir -p "/tmp/$TEMP_PATH"
cd "/tmp/$TEMP_PATH"

NGINX_TARBALL="nginx-$NGINX_CORE_VERSION.tar.gz"
NGINX_DOWNLOAD_URL="https://nginx.org/download/$NGINX_TARBALL"

if [ ! -f "$NGINX_TARBALL" ]; then
    if ! run_command_quietly wget "$NGINX_DOWNLOAD_URL"; then
        echo "ERROR: Could not download Nginx source code $NGINX_CORE_VERSION from nginx.org. Aborting update." >&2
        exit 1
    fi
fi

tar -xzf "$NGINX_TARBALL"
cd "nginx-$NGINX_CORE_VERSION"

# Clean up any leftover *.so files from previous runs.
rm -f objs/*.so

# 3. MANAGE REPOS AND BUILD ARGS
echo "Managing dynamic module repositories..."

DYNAMIC_MODULES_ARGS=()
mkdir -p "$BASE_MODULE_DIR"

# Loop through the repositories, manage the source, and build the configure arguments
for i in "${!MODULE_REPOS[@]}"; do
    REPO_URL="${MODULE_REPOS[$i]}"
    LOCAL_NAME="module$((i + 1))"
    
    # Clone/Pull the repository and get the path
    MODULE_PATH=$(manage_repo "$REPO_URL" "$LOCAL_NAME")
    
    # Collect arguments for ./configure
    DYNAMIC_MODULES_ARGS+=( "--add-dynamic-module=$MODULE_PATH" )
done

# 4. MODULE COMPILATION
echo "Starting compilation of the dynamic modules..."

if ! run_command_quietly ./configure "${DYNAMIC_MODULES_ARGS[@]}" --with-compat; then
    echo "ERROR: ./configure for modules failed. Aborting update. Check log if DEBUG=true." >&2
    exit 1
fi

if ! run_command_quietly make modules; then
    echo "ERROR: make modules failed. Aborting update. Check log if DEBUG=true." >&2
    exit 1
fi

# 5. UNCONDITIONAL COPY
echo "Copying compiled modules to $NGINX_MODULE_PATH..."

mkdir -p "$NGINX_MODULE_PATH"

# Copy all compiled .so files at once.
if ! cp objs/*.so "$NGINX_MODULE_PATH/"; then
    echo "ERROR: Could not copy one or more modules to $NGINX_MODULE_PATH. Aborting update." >&2
    exit 1
fi

# 6. CLEANUP
echo "SUCCESS: Modules compiled and copied for the new Nginx version. Nginx restart is required."

cd ..
rm -rf "/tmp/$TEMP_PATH"

exit 0

2. The APT Hook Configuration

To integrate the script with APT, I just need a simple configuration file. I put it in /etc/apt/apt.conf.d/.

# /etc/apt/apt.conf.d/10nginx-pre-invoke
DPkg::Pre-Invoke { "/usr/local/bin/nginx-pre-invoke-hook.sh"; };

Now, whenever you run apt upgrade or apt install nginx, this hook will automatically trigger. It’s a set-it-and-forget-it solution.

Test and Final Thoughts

While simple, this approach embodies the core principles of Infrastructure as Code (IaC). It automates a critical and often brittle manual process, which is a significant step toward making my setup more robust in the long run.

To test it, I simply ran:

root@lucy:~# apt-get install --reinstall nginx
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
0 upgraded, 0 newly installed, 1 reinstalled, 0 to remove and 0 not upgraded.
Need to get 0 B/1162 kB of archives.
After this operation, 0 B of additional disk space will be used.


--- Starting Nginx Pre-Invoke Check (Package: nginx) ---
Nginx package action (Upgrade or Reinstall) detected. Starting dynamic module recompilation.
Preparing Nginx source code (Downloading/Unpacking)...
Managing dynamic module repositories...
Updating module1 repository...
Updating module2 repository...
Starting compilation of the dynamic modules...
Copying compiled modules to /etc/nginx/modules/...
SUCCESS: Modules compiled and copied for the new Nginx version. Nginx restart is required.
(Reading database ... 132725 files and directories currently installed.)
Preparing to unpack .../nginx_1.29.2-1~noble_amd64.deb ...
Unpacking nginx (1.29.2-1~noble) over (1.29.2-1~noble) ...
Setting up nginx (1.29.2-1~noble) ...


Processing triggers for man-db (2.12.0-4build2) ...
Scanning processes...                                                                                                                  
Scanning candidates...                                                                                                                 
Scanning linux images...                                                                                                               

Running kernel seems to be up-to-date.

Restarting services...
 systemctl restart nginx.service

If everything works, the script will output SUCCESS: Modules compiled and copied for the new Nginx version. Nginx restart is required. before the normal package installation continues. If something goes wrong, the apt command will fail, and you’ll get an error message, just as intended.

I’m using the official nginx package by the way. You may need change nginx to nginx-full or nginx-core or something if you’re using my script.

root@lucy:~# cat /etc/apt/sources.list.d/nginx.list 
deb [signed-by=/usr/share/keyrings/nginx-archive-keyring.gpg] http://nginx.org/packages/mainline/ubuntu noble nginx

This little experiment took me from a brittle, manual process to a more reliable, automated system. It shows that even with something as seemingly simple as a package update, a small investment in automation and scripting can save you a lot of headaches down the road.

References

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.