Category: Incident Response

This article documents a two-part process: successfully upgrading Suricata to version 7 on Debian Bookworm and solving a critical stability issue required to run the AF-Packet IPS mode with high-performance VirtIO NICs in a virtual machine. Without this specific configuration, the IPS failed to function.

Part I: Suricata 7 Upgrade and Policy Changes

A much newer Suricata version can be installed by utilizing Debian’s bookworm-backports repository, which is essential for access to the latest security features and performance enhancements.

The Backports Installation

1. Ensure the backports repository is configured in your /etc/apt/sources.list:
   
   deb https://ftp.debian.org/debian/ bookworm-backports contrib main non-free non-free-firmware
   
2. Install Suricata using the specific target:
   
   apt-get install -t bookworm-backports suricata

Post-Upgrade Security Alert (Critical)

After upgrading to Suricata 7, you may experience immediate traffic blocking. This is not a bug, but a deliberate change in the application’s default security posture.

• Reason: Suricata 7 introduced new policy rules that are often set to drop by default.
• Action: You must review your new suricata.yaml configuration. The recommended approach is to install the new configuration files, compare them with your old setup, and set unwanted policies to ignore.

Reference: This new behavior is explicitly documented in the official Suricata 7 Changelog. Consult the Suricata FAQ for troubleshooting details on blocking issues.

Part II: The VirtIO and AF-Packet Critical Failure Fix

When using Suricata in IPS mode with the high-performance AF-Packet acquisition method, using VirtIO NICs is preferred. However, without a specific Libvirt configuration, the IPS fails entirely to process bridged traffic.

The Problematic Default VirtIO Config

If the VirtIO NIC is defined simply with <model type='virtio'/> in the Libvirt XML, AF-Packet fails to initialize or correctly process traffic.

The Solution: Disabling Guest Checksum Offload

The fix requires overriding the default driver settings by introducing the <driver> block and explicitly setting checksum (csum) offloading to off for the guest system.

This solution was found while troubleshooting similar packet loss issues in a thread related to XDP drivers in RHEL environments, suggesting a common kernel/driver interaction problem with aggressive offloading features.

The minimal required working Libvirt XML configuration looks like this:

    <interface type='bridge'>
      <mac address='..:..:..:..:..:..'/>
      <source bridge='ovs-guests'/>
      <virtualport type='openvswitch'>
      </virtualport>
      <model type='virtio'/>      
      <driver name='vhost'>
        <host csum='off' gso='off' tso4='off' tso6='off' ecn='off' ufo='off' mrg_rxbuf='off'/>
        <guest csum='off' tso4='off' tso6='off' ecn='off' ufo='off'/>
      </driver>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </interface>

Crucial Insight: The key fix is the parameter csum='off' within the <guest/> tag. If checksum offloading is left enabled (csum='on'), the system fails to bridge traffic completely.

Part III: The Deep Dive: Why Checksum Offload Causes Complete Failure

Here is the rationale for why Checksum Offload (CSUM) leads to complete non-functionality:

1. The CSUM Optimization Paradigm (CSUM=’on’)

When you set csum='on', you are performing a performance optimization aimed at saving CPU cycles:

• The Host/Hypervisor receives packets and passes them to the VirtIO Driver (Vhost).
• The Vhost Driver passes the packets into the VirtIO Ring in the Guest System, but marks them with a special flag (e.g., in the skb—Socket Buffer—metadata) signaling to the Guest Kernel: “Attention, the L3/L4 checksum is invalid/missing and must be corrected or calculated before further processing up the stack.”
• This is a performance trick: the CPU-intensive checksum calculation is delegated to the Guest Kernel, but only when it is truly necessary.

2. The Collision Point: AF-Packet Bypass

Suricata using AF-Packet now bypasses precisely this process:

• AF-Packet is a very low-level packet capture method. It operates directly above the driver (or in the kernel) and fetches the raw L2 frames directly from the VirtIO Ring.
• Suricata receives the packet at a point before the standard kernel stack has performed the checksum finalization.
• Suricata’s Deep Packet Inspection (DPI) engine relies on the integrity of the Layer 3/Layer 4 headers (e.g., to check the TCP segment length, track the TCP state machine, or evaluate the validity of IP headers).
• The Non-Functionality: Since Suricata receives a packet with the “Checksum missing/invalid” flag, it interprets this not as an optimization instruction, but as a critical error in the packet itself (Corrupted Packet).

3. The Resolution (CSUM=’off’)

By explicitly setting <guest csum='off'>, we force the Host/Vhost Driver to deliver the packets to the Guest as if they were ‘normal’ Ethernet frames that already contain all checksums. Suricata therefore only sees complete, consistent packets and can apply the DPI logic without error.

Sources / See Also

1. Suricata Documentation. Suricata 7 Changelog (Note new policy behavior). https://suricata.io/changelog/
2. Suricata Documentation. FAQ: Traffic gets blocked after upgrading to Suricata 7. https://suricata-update.readthedocs.io/en/latest/faq.html#my-traffic-gets-blocked-after-upgrading-to-suricata-7
3. Suricata Documentation. Working with AF-Packet. https://docs.suricata.io/en/latest/install/af-packet.html
4. Libvirt Documentation. VirtIO Device Configuration (Driver Offload Parameters). https://libvirt.org/formatdomain.html#elementsNICS
5. Debian Wiki. Instructions for using Debian Backports. https://wiki.debian.org/Backports
6. Suricata Community Forums. Troubleshooting references for XDP/Packet Loss (Context for driver tuning). https://forum.suricata.io/
7. Linux Networking. Understanding the Checksum Offload Mechanism. https://www.kernel.org/doc/Documentation/networking/checksum-offloads.txt