Custom REGEX rules for CSF.

[Sergio]

@sergio:
I don't know, something like this? REGEX to block IPs that triggered via COMODO-WAF modsecurity in DirectAdmin Custombuild 2.0

I will use the name of your post "REGEX for DirectAdmin modsecurity denials not blocked by CSF/LFD"

[ethical]

Hi Sergio

very helpful thread. Do you know how I could adjust the script to work with ASSP and exim? I have a cpanel server with the ASSP spam filtering proxy sitting in front of exim.

thanks!
John

[vhortex]

Hi,

Can we adjust the rules/regex to scan for a longer time. I am not sure if there is any way to do it via customs.regex

Scenario is
Attacker will randomly attack once or twice an hour to avoid being blacklisted.

If there is anyway that this can be achieve, please give an idea. I don't want to tamper the main CSF scripts.

[ethanpil]

Here are some rules I recently developed. I will try and keep them updated here: https://gist.github.com/ethanpil/97b62d ... 8b3837843d

NginX Security to block bad behaving web visitors
These rules have helped me block vulnerability scanners, and bots/hackers scanning for varios versions of PHP tools, etc.

Code: Select all

# NginX security rules trigger (Default: 4 errors bans for 24 hours)
# Catch ip that attempts to access a URL that is forbidden by NginX rules
if (($globlogs{CUSTOM1_LOG}{$lgfile}) and ($line =~ /.*access forbidden by rule, client: (\S+).*/)) {
    return ("NGINX Security rule triggered from",$1,"nginx_security","4","80,443","86400");
}

Code: Select all

# NginX 404 errors (Default: 4 errors bans for 24 hours)
# Catch ip that accesses non-existant files and directories
if (($globlogs{CUSTOM1_LOG}{$lgfile}) and ($line =~ /.*No such file or directory\), client: (\S+),.*/)) {
    return ("NGINX Security rule triggered from",$1,"nginx_404s","4","80,443","86400");
}

Code: Select all

#Trying to download htaccess or htpasswd  (Default: 1 error bans for 24 hours)
if (($globlogs{CUSTOM1_LOG}{$lgfile}) and ($line =~ /.*\.(htpasswd|htaccess).*client: (\S+),.*GET)/) {
    return ("Trying to download .ht files",$2,"nginx_htfiles","1","80,443","86400");
}

WordPress fail2Ban
The below five rules below work well with the WordPress fail2ban plugins https://wordpress.org/plugins/wp-fail2ban-redux/ plugin. Instead of a custom log file, these plugins write into the syslog which is already defined as SYSLOG_LOG

These rules are the equivalent of the wordpress Hard ruleset in the fail2ban plugins. You still need to install and activate the fail2ban plugin, but it will work with LFD (without fail2ban installed) with the below custom regex.

Code: Select all

# Wordpress fail2ban plugin (Default: 5 errors bans for 24 hours)
if (($globlogs{SYSLOG_LOG}{$lgfile}) and ($line =~ /.*Authentication attempt for unknown user .* from (.*)\n/)) {
  return ("Wordpress unknown user from",$1,"fail2ban_unknownuser","2","80,443","86400");
}

# Wordpress fail2ban plugin (Default: 2 errors bans for 24 hours)
if (($globlogs{SYSLOG_LOG}{$lgfile}) and ($line =~ /.*Blocked user enumeration attempt from (.*)\n/)) {
  return ("WordPress user enumeration attempt from",$1,"fail2ban_userenum","2","80,443","86400");
}

# Wordpress fail2ban plugin (Default: 2 errors bans for 24 hours)
if (($globlogs{SYSLOG_LOG}{$lgfile}) and ($line =~ /.*Pingback error .* generated from (.*)\n/)) {
  return ("WordPress pingback error",$1,"fail2ban_pingback","2","80,443","86400");
}

# Wordpress fail2ban plugin (Default: 2 errors bans for 24 hours)
if (($globlogs{SYSLOG_LOG}{$lgfile}) and ($line =~ /.*Spammed comment from (.*)\n/)) {
  return ("WordPress spam comments from",$1,"fail2ban_spam","2","80,443","86400");
}
# Wordpress fail2ban plugin (Default: 2 errors bans for 24 hours)
if (($globlogs{SYSLOG_LOG}{$lgfile}) and ($line =~ /.*XML-RPC multicall authentication failure (.*)\n/)) {
  return ("WordPress XML-RPC multicall fail from",$1,"fail2ban_xmlrpc","5","80,443","86400");
}

[Paarsch]

Hello!

I am trying to implement a new custom Regex rule, To specifically target crawlerbots. i formulated the following:

Code: Select all

# MJ12-Bot / Baidu / Ahrefs
if (($globlogs{CUSTOM2_LOG}{$lgfile}) and ($line =~ /(\S+).*] "\w*(?:GET|POST|HEAD).* (MJ12bot|Baiduspider|AhrefsBot|UptimeRobot).*" /)) {
         return ("LFD - MJ12-Baidu-Ahrefs-bot Overuse",$1,"BOTOVERUSE","15","80,443,21,25,22,23","48800");
}

When i parse it through my tester it does pickup the correct rules out of the following example:

Code: Select all

69.162.111.222 - - [07/Feb/2017:15:54:14 +0200] "HEAD / HTTP/1.1" 200 296 "http://dummydomain.nl/" "Mozilla/5.0+(compatible; UptimeRobot/2.0; http://www.uptimerobot.com/)"
185.119.111.222 - - [07/Feb/2017:15:55:37 +0200] "GET /wp-login.php HTTP/1.1" 200 2860 "-" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36"
185.119.111.222 - - [07/Feb/2017:15:55:37 +0200] "POST /wp-login.php HTTP/1.1" 200 3610 "http://www.dummydomain.nl/wp-login.php" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36"
149.210.111.222 - - [07/Feb/2017:15:59:13 +0200] "POST /wp-cron.php?doing_wp_cron=1496930353.4784278869628906250000 HTTP/1.1" 200 181 "http://www.dummydomain.nl/wp-cron.php?doing_wp_cron=1496930353.4784278869628906250000" "WordPress/4.7.5; http://www.dummydomain.nl"
69.162.111.222 - - [07/Feb/2017:15:59:12 +0200] "HEAD / HTTP/1.1" 301 229 "http://dummydomain.nl/" "Mozilla/5.0+(compatible; UptimeRobot/2.0; http://www.uptimerobot.com/)"

However it simply won't add the targetted IP-adresses to the blacklist. What am i missing? Any suggestions are very welcome!

[ethanpil]

I tweaked the regex little, but you should be getting the IP in $1 even with your original code.... not sure why you aren't. Are you sure you are reading the correct log file with

Code: Select all

CUSTOM2_LOG

?

Code: Select all

# MJ12-Bot / Baidu / Ahrefs
if (($globlogs{CUSTOM2_LOG}{$lgfile}) and ($line =~ /(\S+).*\"*(?:GET|POST|HEAD).* (MJ12bot|Baiduspider|AhrefsBot|UptimeRobot).*/)) {
         return ("LFD - MJ12-Baidu-Ahrefs-bot Overuse",$1,"BOTOVERUSE","15","80,443,21,25,22,23","48800");
}

Take a look here: https://regex101.com/r/JagOJb/1

You can also probably add the exact bot name in the log line

Code: Select all

return ("LFD - $2 bot Overuse",$1,"BOTOVERUSE","15","80,443,21,25,22,23","48800");

[Paarsch]

Groovy, that did the trick! Thank you for your help, also good suggestion on the bot name parameter!

[dswimr615]

Redacted

[Sergio]

Hi, all.

I have been using CSF to block massive spammers on my servers, the same REGEX rules are used simultaneously with SPAMASSASSIN and the results have been very satisfactory. Here is an example of a rule that blocks spam that the subject comes with names of TV or Movie related people:

ANTI SPAM REGEX RULE:
DOS: CloudLINUX
Use this rule on CSF: regex.custom.pm
Execution time: -1MS / Interaction: 104 steps
LOG: /var/log/exim_mainlog

Code: Select all

	if (($lgfile eq $config{SMTPAUTH_LOG}) and ($line =~ /^\S+\s\S+\s\S+\s<=\s\S+\sH=(?>\S+\s)+?\[(\S+)\](?>\S+\s)+?T="(?>\S*\s*)*?(angelina jolie|beyonce|brad pitt|eva longoria|dr\.? seuss|jessica alba| kanye |mark cuban|megyn kelly|melissa mccarthy|shark tank|sharon stone|taylor swift|vanna white|warren buffett|zuckerb.rg)/i)) {
		return ("SPAM BLKList $2",$1,"SPAM_BLK_LISTMovies","1","1");
	}

Example that will trigger the rule:

2017-08-18 14:02:33 email-ID-obfuscated <= Sara-Shaffer@debris.azurespecials.bid H=debris.azurespecials.bid [37.28.158.14]:55744 P=esmtp S=7554 T="Cosmo: Sharon Stone's Gorgeous Skin Secret." for email@domain.com

SPAMASSASSIN RULE:

Code: Select all

header   SECMAS_BLKMovies  Subject =~ /angelina jolie|beyonce|brad pitt|eva longoria|dr\.? seuss|jessica alba| kanye |mark cuban|megyn kelly|melissa mccarthy|shark tank|sharon stone|taylor swift|vanna white|warren buffett|zuckerb.rg/i
describe SECMAS_BLKMovies  SPAM_BLKMovies
score    SECMAS_BLKMovies  22

To use this rule in SPAMASSASSIN, you should create a file called "MyRules.cf" inside directory /etc/mail/spamassassin and copy this and any other home made rules in there.

DISCLAIMER:
Any use of my rules are at your own risk, Don't use them if you don't know what the rules are intended for.

[awalilko]

I made a bit of regex to monitor failed plesk logins (which also seems to work for failed webmail logins on plesk)

Working OS: RHEL7
Action: Block IP address after 5 failed Plesk login attempts over 3600s
CUSTOM1_LOG = "/var/log/plesk/panel.log"

Code: Select all

	if (($globlogs{CUSTOM1_LOG}{$lgfile}) and ($line =~ /^.*Failed\slogin\sattempt\swith\slogin\s\S+\sfrom\sIP\s(\d+\.\d+\.\d+\.\d+)/)) {
                return ("Failed plesk login from",$1,"plesk","5","8443","1");
        }

Sample blocked lines:

[2018-01-01 20:11:18] ERR [panel] [Action Log] Failed login attempt with login 'username' from IP 123.45.67.89