High CPU usage while IDLing

[vgdh]

What installation are you running?

Production (netalertx) 📦

Is there an existing issue for this?

• I have searched the existing open and closed issues and I checked the docs https://docs.netalertx.com/

The issue occurs in the following browsers. Select at least 2.

• Firefox
• Chrome
• Edge
• Safari (unsupported) - PRs welcome
• N/A - This is an issue with the backend

Current Behavior

High CPU usage

Expected Behavior

near-zero usage.
Currently, for example, my OPNsense VM uses the same amount of CPU (under mid network load)

Steps To Reproduce

No response

Relevant app.conf settings

docker-compose.yml

services:
  netalertx:
    container_name: netalertx
    hostname: Net-Alert-X-IR
    image: ghcr.io/jokob-sk/netalertx:latest
    restart: unless-stopped

    network_mode: "host"

    read_only: true                                 # Make the container filesystem read-only
    cap_drop:                                       # Drop all capabilities for enhanced security
      - ALL
    cap_add:                                        # Add only the necessary capabilities
      - NET_ADMIN                                   # Required for ARP scanning
      - NET_RAW                                     # Required for raw socket operations
      - NET_BIND_SERVICE                            # Required to bind to privileged ports (nbtscan)
      - CHOWN                                       # Required for root-entrypoint to chown /data + /tmp before dropping privileges
      - SETUID                                      # Required for root-entrypoint to switch to non-root user
      - SETGID                                      # Required for root-entrypoint to switch to non-root group

    volumes:
      - ./data:/data
      - /etc/localtime:/etc/localtime:ro

    environment:
      - PORT=20211
      - LISTEN_ADDR=0.0.0.0                   # Listen for connections on all interfaces
      - GRAPHQL_PORT=20212                   # GraphQL API port (passed into APP_CONF_OVERRIDE at runtime)

    tmpfs:
      # All writable runtime state resides under /tmp; comment out to persist logs between restarts
      - "/tmp:uid=20211,gid=20211,mode=1700,rw,noexec,nosuid,nodev,async,noatime,nodiratime"


    # Resource limits to prevent resource exhaustion
    mem_limit: 2048m            # Maximum memory usage
    mem_reservation: 1024m      # Soft memory limit
    cpu_shares: 512             # Relative CPU weight for CPU contention scenarios
    pids_limit: 512             # Limit the number of processes/threads to prevent fork bombs
    logging:
      options:
        max-size: "10m"         # Rotate log files after they reach 10MB
        max-file: "3"           # Keep a maximum of 3 log files

Debug or Trace enabled

• I have read and followed the steps in the wiki link above and provided the required debug logs and the log section covers the time when the issue occurs.

Relevant app.log section

Docker Logs

.

[jokob-sk]

Hi @vgdh ,

Thanks for the report.

1. What ids your DB size?
2. What is your WAL file size?
3. What is the value of the PRAGMA_JOURNAL_SIZE_LIMIT setting?
4. How often is you DBCLNP plugin run?
5. Are there any execution errors in your log file?

You can find how certain configuration options influence system resources here: https://docs.netalertx.com/PERFORMANCE/?h=perfo

[vgdh]

I removed all from the data folder. So basically it's a clean install (after the red line on the screenshot).
For some strange reason `CPU pressure stall" has increased. But CPU usage the same
All screenshots are after I have made a clean install.

root@Net-Alert-X:~# docker compose logs
netalertx  | 
netalertx  |  _   _      _    ___  _           _  __   __
netalertx  | | \ | |    | |  / _ \| |         | | \ \ / /
netalertx  | |  \| | ___| |_/ /_\ \ | ___ _ __| |_ \ V /
netalertx  | | .   |/ _ \ __|  _  | |/ _ \  __| __|/   \
netalertx  | | |\  |  __/ |_| | | | |  __/ |  | |_/ /^\ \
netalertx  | \_| \_/\___|\__\_| |_/_|\___|_|   \__\/   \/
netalertx  |    Network intruder and presence detector.
netalertx  |    https://netalertx.com
netalertx  | 
netalertx  | 
netalertx  | Startup pre-checks
netalertx  | --> data migration.sh 
netalertx  | --> capabilities audit.sh 
netalertx  | --> mounts.py 
netalertx  |  Path                     | R | W | Mount | RAMDisk | Performance | DataLoss 
netalertx  | --------------------------+---+---+-------+---------+-------------+----------
netalertx  |  /data                    | ✅| ✅|   ✅  |    ➖   |      ➖     |    ✅     
netalertx  |  /data/db                 | ✅| ✅|   ✅  |    ➖   |      ➖     |    ✅     
netalertx  |  /data/config             | ✅| ✅|   ✅  |    ➖   |      ➖     |    ✅     
netalertx  |  /tmp/run/tmp             | ✅| ✅|   ✅  |    ✅   |      ✅     |    ✅     
netalertx  |  /tmp/api                 | ✅| ✅|   ✅  |    ✅   |      ✅     |    ✅     
netalertx  |  /tmp/log                 | ✅| ✅|   ✅  |    ✅   |      ✅     |    ✅     
netalertx  |  /tmp/run                 | ✅| ✅|   ✅  |    ✅   |      ✅     |    ✅     
netalertx  |  /tmp/nginx/active-config | ✅| ✅|   ✅  |    ✅   |      ✅     |    ✅     
netalertx  | --> first run config.sh 
netalertx  | ══════════════════════════════════════════════════════════════════════════════
netalertx  | 🆕  First run detected. Default configuration written to /data/config/app.conf.
netalertx  | 
netalertx  |     Review your settings in the UI or edit the file directly before trusting
netalertx  |     this instance in production.
netalertx  | ══════════════════════════════════════════════════════════════════════════════
netalertx  | --> first run db.sh 
netalertx  | ══════════════════════════════════════════════════════════════════════════════
netalertx  | 🆕  First run detected — building initial database at: /data/db/app.db
netalertx  | 
netalertx  |     Do not interrupt this step. When complete, consider backing up the fresh
netalertx  |     DB before onboarding sensitive or critical networks.
netalertx  | ══════════════════════════════════════════════════════════════════════════════
netalertx  | --> mandatory folders.sh 
netalertx  |     * Creating NetAlertX log directory.
netalertx  |     * Creating NetAlertX API cache.
netalertx  |     * Creating System services runtime directory.
netalertx  |     * Creating nginx active configuration directory.
netalertx  |     * Creating Plugins log.
netalertx  |     * Creating System services run log.
netalertx  |     * Creating System services run tmp.
netalertx  |     * Creating DB locked log.
netalertx  |     * Creating Execution queue log.
netalertx  | --> apply conf override.sh 
netalertx  | --> override individual settings.sh 
netalertx  | --> host optimization.sh 
netalertx  | ══════════════════════════════════════════════════════════════════════════════
netalertx  | ⚠️  WARNING: ARP flux sysctls are not set.
netalertx  | 
netalertx  |     Expected values:
netalertx  |       net.ipv4.conf.all.arp_ignore=1
netalertx  |       net.ipv4.conf.all.arp_announce=2
netalertx  | 
netalertx  |     Detection accuracy may be reduced until configured.
netalertx  | 
netalertx  |     See: https://docs.netalertx.com/docker-troubleshooting/arp-flux-sysctls/
netalertx  | ══════════════════════════════════════════════════════════════════════════════
netalertx  | --> writable config.sh 
netalertx  | --> nginx config.sh 
netalertx  | --> expected user id match.sh 
netalertx  | --> host mode network.sh 
netalertx  | ══════════════════════════════════════════════════════════════════════════════
netalertx  | ⚠️  ATTENTION: NetAlertX is not running with --network=host.
netalertx  | 
netalertx  |     Bridge networking blocks passive discovery (ARP, NBNS, mDNS) and active
netalertx  |     scanning accuracy. Most plugins expect raw access to the LAN through host
netalertx  |     networking and CAP_NET_RAW capabilities.
netalertx  | 
netalertx  |     Restart the container with:
netalertx  |         docker run --network=host --cap-add=NET_RAW --cap-add=NET_ADMIN --cap-add=NET_BIND_SERVICE
netalertx  |     or set "network_mode: host" in docker-compose.yml.
netalertx  | 
netalertx  |     https://docs.netalertx.com/docker-troubleshooting/network-mode
netalertx  | ══════════════════════════════════════════════════════════════════════════════
netalertx  | --> excessive capabilities.sh 
netalertx  | --> appliance integrity.sh 
netalertx  | --> ports available.sh 
netalertx  | APP_CONF_OVERRIDE detected (set from GRAPHQL_PORT)
netalertx  | Starting /usr/sbin/php-fpm83 -y "/services/config/php/php-fpm.conf" -F (tee stderr to app.php_errors.log)
netalertx  | Starting supercronic --quiet "/services/config/cron/crontab" >>"/tmp/log/cron.log" 2>&1 &
netalertx  | Starting python3  -m server > /tmp/log/stdout.log 2> >(tee /tmp/log/stderr.log >&2)
netalertx  | Starting /usr/sbin/nginx -p "/tmp/run/" -c "/tmp/nginx/active-config/nginx.conf" -g "error_log stderr; error_log /tmp/log/nginx-error.log; daemon off;" &
netalertx  | /opt/venv/lib/python3.12/site-packages/requests/__init__.py:113: RequestsDependencyWarning: urllib3 (2.6.3) or chardet (7.0.1)/charset_normalizer (3.4.5) doesn't match a supported version!
netalertx  |   warnings.warn(
netalertx  | Successfully updated IEEE OUI database (113327 entries)

[jokob-sk]

I'd go then with this is probably expected.

On my Synology a clean installation idles around 3% of CPU, not sure what system you are running this on.

Since you didn't mention any settings I assume you are running the defaults. You can try decreasing the scan frequency to try to decrease the CPU usage. I might look into trying to decrease CPU usage, but I'm a bit swamped with other things, so won't get to it for while. Feel free to suggest optimizations yourself.

[vgdh]

@jokob-sk thank you for your work.

[EyeOfIllu]

Hi,

I have the exact same problem and the same CPU Usage graph. I don't have such a high CPU Pressure Stall, but that's probably because my LXC container has four CPU cores assigned. It's not a scanning frequency issue, because with the scanning set to */15, the exact same problem occurs. Even disabling all additional modules except Arp-Scan doesn't help. Higher usage can be observed approximately every 5-7 seconds. In htop, I see that the usage is being caused by "python3 -m server." This problem has been occurring for over two years and was present even before the code was refactored.