Automating IPS: Real-Time Suricata Rule Generation via Fail2ban Hook

In my last posts, I established a central syslog hub feeding Fail2ban and demonstrated Suricata as an intrusion prevention system (IPS). This final piece connects the two: feeding Suricata with the ban results from Fail2ban by creating a dynamic, external rule file.

This process is highly automated, but requires robust Bash scripting and careful handling of security context.

1. Fail2ban Action and Scripting Logic

The core idea is to replace Fail2ban’s default firewall action with a custom script that modifies a public rule file.

Custom Action Definition

The actionban and actionunban directives in /etc/fail2ban/action.d/firewall.conf point to simple Bash wrappers.

[Definition]

actionstart = 
actionstop =

actioncheck =

actionban = /var/www/f2b/bin/ban.sh <ip>
actionunban = /var/www/f2b/bin/unban.sh <ip>

Security-Hardened ban.sh Script

The ban script must: (1) validate the input, (2) generate a unique Signature ID (SID), (3) append the rule, and (4) atomically update the ruleset’s MD5 checksum for Suricata-Update to fetch the change.

#!/bin/bash
#
# ban.sh: Adds a banned IP to the Fail2ban Suricata ruleset.

IP="$1"
RULESFILE="/var/www/f2b/htdocs/fail2ban.rules"
MSG="BRUTE-FORCE detected by fail2ban"

# INPUT VALIDATION: Ensure the input is a valid IPv4/IPv6 address.
if ! [[ $IP =~ ^([0-9]{1,3}\.){3}[0-9]{1,3}$ || $IP =~ ^([0-9a-fA-F]{1,4}:){7}[0-9a-fA-F]{1,4}$ ]]; then
    echo "ERROR: Invalid IP address received: $IP" >&2
    exit 1
fi

# 1. Generate Unique SID (Timestamp + Counter)
TSTAMP=$(date +%s)
CNT=$(wc -l $RULESFILE | cut -d' ' -f1);
# Generate a SID in a high range to avoid conflicts with commercial rules (e.g., 90000000+)
SID=$(($CNT + $TSTAMP + 90000000))

if ! grep -q "$IP" $RULESFILE; then
  # Rule: Drop all traffic from the banned IP to any network home port.
  # Using 'drop ip' is robust; adjust ports if required (e.g., $SSH_PORTS).
  RULE="drop ip $IP any -> \$HOME_NET any (msg:\"$MSG - $IP\"; sid:$SID; rev:1;)"

  echo $RULE >> $RULESFILE
  
  # Set correct permissions (Critical step for web delivery)
  chown www-data:www-data $RULESFILE

  # 2. Atomically update the MD5 checksum file
  SUM=$(md5sum $RULESFILE | cut -d' ' -f1);
  echo $SUM > $RULESFILE.md5
fi

The Unban Script (unban.sh)

The unban script removes the line and performs the critical MD5 update.

#!/bin/bash
#
# unban.sh: Removes a banned IP from the Suricata ruleset.

IP="$1"
RULESFILE="/var/www/f2b/htdocs/fail2ban.rules"

if grep -q "$IP" $RULESFILE; then
  # Use sed -i to remove the line containing the IP address
  sed -i '/'$IP'/d' $RULESFILE
  
  # Atomically update the MD5 checksum
  SUM=$(md5sum $RULESFILE | cut -d' ' -f1);
  echo $SUM > $RULESFILE.md5
fi

2. Integration and Verification

The final step is to make the ruleset publicly available (via HTTPS/SSL) and configure Suricata to fetch it.

Suricata-Update Configuration

The rule file (fail2ban.rules) must be made available via a web server (e.g., NGINX) with a specific URL (e.g., https://f2b.example.com/fail2ban.rules). I add this URL as a new source to Suricata-Update.

root@fw2:~# suricata-update add-source
URL: https://f2b.example.com/fail2ban.rules

# Running the update process
19/11/2023 -- 20:17:38 - <Info> -- Fetching https://f2b.example.com/fail2ban.rules.
 100% - 18344/18344                   
19/11/2023 -- 20:17:38 - <Info> -- Done.

Verification and Observability

Verification confirms that the new rules are loaded and actively dropping traffic. The log analysis command must be adapted to track these specific fail2ban drops.

# Awk command to filter and count dropped packets (Excerpt showing drop sources)
# awk '/Drop/{...}' fast.log | sort | uniq -c | sort -hr

   6505 IP dropped due to fail2ban detection
    638 ET DROP Dshield Block Listed Source 
    ...

This ensures a comprehensive, self-healing Incident Response Chain.

Sources / See Also

1. Suricata Documentation. High-performance AF_PACKET IPS mode configuration and usage. https://docs.suricata.io/en/latest/install/af-packet.html
2. Suricata Documentation. Working with Suricata-Update (Ruleset Management). https://suricata-update.readthedocs.io/en/latest/update.html
3. Suricata Documentation. EVE JSON Output for Structured Logging. https://docs.suricata.io/en/latest/output/eve/eve-json-format.html
4. Google Development. Google Perftools (TCMalloc) Documentation. https://github.com/google/gperftools
5. Emerging Threats (Proofpoint). Information on the Emerging Threats Open Ruleset. https://www.proofpoint.com/us/security-awareness/blog/emerging-threats
6. Elastic Stack (ELK) Documentation for Log Analysis. https://www.elastic.co/what-is/elk-stack
7. Linux Manpage: ethtool (Network Offload Configuration). https://man7.org/linux/man-pages/man8/ethtool.8.html

---