LF_MODSEC not blocking as expected

[kirkre]

I'm trying to get LF_MODSEC to block IPs that trigger mod_security rules, but so far it's not working as I expect. Here are my settings:

MODSEC_LOG = /var/log/httpd/error_log
All vhosts are set to put errors in this log

LF_TRIGGER = 0
LF_TRIGGER_PERM = 0
LF_MODSEC = 5
LF_MODSEC_PERM = 86400

Here are some example rule triggers from /var/log/httpd/error_log. Each of these examples is repeated more than 5 times in the logs.

[Sun Nov 08 07:15:34 2015] [error] [client 31.11.143.54] ModSecurity: [file "/etc/httpd/modsecurity.d/99_asl_zzzz_threat_intelligence.conf"] [line "70"] [id "355504"] [rev "1"] [msg "Atomicorp.com WAF Rules: Threat Intelligence Match for Known attacker Source on Atomicorp Threat Intelligence RBL (TI-4). See this URL for details http://www.atomicrbl.com/lookup"] [severity "ERROR"] Access denied with code 403 (phase 2). RBL lookup of 54.143.11.31.threat4.atomicrbl.com. succeeded at REMOTE_ADDR. [hostname "www.ourdomain.org"] [uri "/path/to/script.js"] [unique_id "Vj9K9qLy1rMAAEUlScIAAAAr"]

[Sun Nov 08 13:33:37 2015] [error] [client 150.162.127.14] ModSecurity: [file "/etc/httpd/modsecurity.d/99_asl_zzzz_threat_intelligence.conf"] [line "52"] [id "350054"] [rev "1"] [msg "Atomicorp.com WAF Rules: Threat Intelligence Match for known Attacker source on Atomicorp Threat Intelligence RBL. See this URL for details http://www.atomicrbl.com/lookup (Previous TI-4 Match)"] [severity "ERROR"] Access denied with code 403 (phase 2). Operator EQ matched 1 at IP:threat4. [hostname "www.ourdomain.org"] [uri "/index.php"] [unique_id "Vj@jkaLy1rMAAEk3gFAAAAAT"]

[Sun Nov 08 10:12:53 2015] [error] [client 46.118.155.216] ModSecurity: [file "/etc/httpd/modsecurity.d/99_asl_zzzz_threat_intelligence.conf"] [line "56"] [id "350055"] [rev "1"] [msg "Atomicorp.com WAF Rules: Threat Intelligence Match for known multi event Attacker source on Atomicorp Threat Intelligence RBL. See this URL for details http://www.atomicrbl.com/lookup (Previous TI-5 Match)"] [severity "ALERT"] Access denied with code 403 (phase 2). Operator EQ matched 1 at IP:threat5. [hostname "www.ourdomain.org"] [uri "/"] [unique_id "Vj90haLy1rMAAELheDoAAAAs"]


[Sun Nov 08 14:58:57 2015] [error] [client 52.23.156.32] ModSecurity: [file "/etc/httpd/modsecurity.d/99_asl_zzzz_threat_intelligence.conf"] [line "73"] [id "355506"] [rev "1"] [msg "Atomicorp.com WAF Rules: Threat Intelligence Match for Known multi event attacker Source on Atomicorp Threat Intelligence RBL. See this URL for details http://www.atomicrbl.com/lookup"] [severity "ALERT"] Access denied with code 403 (phase 2). RBL lookup of 32.156.23.52.threat5.atomicrbl.com. succeeded at REMOTE_ADDR. [hostname "www.ourdomain.org"] [uri "/robots.txt"] [unique_id "Vj@3kaLy1rMAAAnKrs8AAAAY"]

Should the '#mod_security v2 (apache)' entry in regex.pm not match these log entries?

Any help getting this working would be appreciated.

Thanks,

Kirk

[marcele]

You can see that the CSF regex does match these lines. Did you restart LFD after making your changes:

The CSF regex for modsecurity works:
https://regex101.com/r/lN8sD8/1

[kirkre]

Thanks for the reply Marcele. I did restart CSF and LFD from the Webmin UI. I have restarted again from the shell with csf -ra, but it has not helped.

Your regex101 link is going to help me either way, because what I would most like is custom regex to generate a firewall block based on the trigger below, and a working regex101 example of the stock regex helps greatly. Since I am somewhat challenged writing regex, I would like to see the stock LF_MODSEC regex working before I try and write a custom rule, and that is the reason I am asking about this first. Any idea what could be wrong?

Here is what I would most like to trigger a firewall block, which I think will require custom regex:

[Sun Nov 08 21:50:19 2015] [warn] ModSecurity: Access denied with code 400. Too many threads [255] of 100 allowed in WRITE state from 177.141.142.53 - Possible DoS Consumption Attack [Rejected]

ModSecurity is unable to stop these attacks on it's own. 300 of these requests and Apache is put into a several hour long wait state, even though the server has enough resources to handle all normal traffic with ease.

Thanks,

Kirk

[marcele]

Here is a custom rule . This will permanently block when it is detected 1 time for an IP.

The regex:
https://regex101.com/r/lN8sD8/2

The rule

Code: Select all

if (($config{LF_MODSEC}) and ($lgfile eq $config{MODSEC_LOG}) and ($line =~ /^\[\S+\s+\S+\s+\S+\s+\S+\s+\S+\] \[(\w*:)?warn\] (\[pid \d+(:tid \d+)?\] )?ModSecurity: Access denied with code \d+. Too many threads \[\d+\] of \d+ allowed in WRITE state from (\S+) - Possible DoS Consumption Attack \[\S+\]/)) (
    return ("Possible DoS Consumption Attack",$1,"mod_security_dos",1,"80,443","1");
}

[kirkre]

Thanks Macele! I'm sure this should work, but just like LF_MODSEC I can't get it to work on my systems. I tested your regex on a test server, using an ab script on another server that easily triggered the alert, but no block in CSF :/

[marcele]

Make sure that your IP address isn't in csf.ignore / csf.allow or its not going to block you when you test it.

[kirkre]

Thanks but I did check that, and also grepped the iptables rules for the ip with 'csf -g', no match. Another thing, if a block is processed it sends me an email alert even if the IP is in csf.allow, with a notation that the block might not take effect because the IP is whitelisted. Even if by some bug the IP were was still whitelisted from being previously in csf.allow, I would expect to get the block notification, which didn't show up.

Is there any possibility that CSF has some other check that is negating the regex match? Like maybe for instance no modsecurity match is processed if it says 'warn'? The other alert that does not work has 'severity "ALERT"', and I wonder if there could be something in CSF cross checking this and taking no action because of the reported severity level.

[marcele]

Like maybe for instance no modsecurity match is processed if it says 'warn'?

Nope CSF does not have anything like that. If the regex matches it matches. I suggest starting with a fresh csf.conf and see if that fixes it for you. I'm thinking that you have some other setting that is doing this.

[kirkre]

I tested with a default csf.conf, and I also discovered something I did not know. Apparently you need to set LF_SELECT to 1 in order to enable custom regex, correct? But I still could not get this to work. I used a default csf.conf, and only changed two settings, TESTING=0 and LF_SELECT=1. After setting LF_SELECT to 1, restaring LFD gave me a syntax error for regex.custom.pm. I resolved this warning by replacing one left parenthesis in your custom regex with a left curly bracket like so:

Code: Select all

if (($config{LF_MODSEC}) and ($lgfile eq $config{MODSEC_LOG}) and ($line =~ /^\[\S+\s+\S+\s+\S+\s+\S+\s+\S+\] \[(\w*:)?warn\] (\[pid \d+(:tid \d+)?\] )?ModSecurity: Access denied with code \d+. Too many threads \[\d+\] of \d+ allowed in WRITE state from (\S+) - Possible DoS Consumption Attack \[\S+\]/)) {
    return ("Possible DoS Consumption Attack",$1,"mod_security_dos",1,"80,443","1");
}

Now I can restart lfd with no warning, but even with the default csf.conf and this change I can't get the custom regex rule to trigger. Was this change the right way to resolve the syntax error? Not sure.

Thanks!

[kirkre]

I just found something very disconcerting that could explain why the custom regex is not working for us. Permanent blocks are not working in our CSF install at all. Temporary blocks do work. When a block reaches permenant status it is added to csf.tempip, but nothing is added to csf.deny, and csf -g shows that when the block status goes from temporary to permanant and the ip is removed from csf.tempban, the block is simply removed completely :-| I will be very surprised if it was NOT me that broke this, but what could I have done?

Thanks for the help,

Kirk