Crowdsec CTI Integration python script
I’ve been working on an integration between Crowdsec CTI and Suricata, both running on OPNsense. The purpose of the integration is to have Suricata in IPS mode with a set of preferred rules configured to alert only. All triggered alerts are logged in the fast.log format. A CrowdSec CTI script made in python parses the fast.log file and queries all entries in the log. If an IP is marked as Malicious or Suspicious by the CrowdSec CTI, it is added to a new log called filtered-fast.log and subsequently blocked based on the Suricata collection. The script implements a TTL cache mechanism with a 1-hour duration to avoid excessive queries and conserve allocated API requests when an IP triggers multiple alerts. IPs marked as Benign, Safe, Known, or Unknown are skipped and not added to filtered-fast.log. The public IP of the gateway (in my case, the WAN/PPPoE interface) is also explicitly excluded from queries. In this version of the script I am also sending Benign IP’s to the filterd-fast.log and filtering out Nmap alerts. You can define in the script if there is anything that you want to exclude from the query search. The script also maintains its own logs for informational and debugging purposes, stored at /usr/local/bin/crowdsec/crowdsec_cti.log and /usr/local/bin/crowdsec/crowdsec_cti_debug.log. The script handles API errors (e.g., 429 rate limits with exponential back off, 404 for unknown IPs)and caches “Unknown” responses for failed queries. Before having the CTI integration functional, have a look on how to Integrate Crowdsec with Suricata and push the logs to fast.log In order for this integration to work you also need the Crowdsec for Suricata Collection installed
...