regex.custom.pm stopped working

[ocahui]

I started using regex.custom.pm several years ago, with great success.

However, for reasons unknown, it stopped working over a year ago. The regex.custom.pm didn't change. There have been changes to csf.conf, including automatic update changes. However, I have a copy of the conf file saved some months before the regex script stopped working, and the only difference I can see in the latest conf is comment lines added by automatic update.

I would like to get it running again because of a substantial increase in exim traffic aimed at (failed) exploits. Would like to block these bad boys from filling up my mail logs.

My server is running CentOS 7 on KVM. I am using iptables+ipset. I do some blocking by country code, and the file where iptables does its logging is quite large every day, so that part is certainly working.

Lfd warns of changed files on every update, so that is working as expected.

With both csf and lfd appearing to run OK and with no change to the setup of either csf or lfd, I don't know what else to look for. Any ideas on what to look for that would block the operation of that feature would be greatly appreciated. My latest thought was perhaps permissions or ownership of the file to be tested, but doesn't lfd run as root?

[BallyBasic79]

Suggestions:

• Ensure the CUSTOMX_LOGs defined that the bottom of csf.conf match those specified in your custom rules.
• Verify that the log format or log message your rules are searching for have not changed.
• GREP your logs with the rules to test them and verify that they work.

I would verify these before anything else.

[ocahui]

Could you post a (very brief) example of a grep command to test my regex expression against one of my files? Or a pointer to somewhere that does?

I have only used grep for trivial searching whole folders for brief phrases, etc. My regex uses PCRE syntax with various shortcuts, and tests whole line /^....$/

[ocahui]

Never mind. I see I need to use -P. option There does appear to be some issue with the main pattern that was working previously.

[ocahui]

Suggestions:

• Verify that the log format or log message your rules are searching for have not changed.

You nailed it! The lines I am searching begin with date/time. The logging application apparently added ms to the time.
From:
2018-02-18 04:01:33
To:
2019-09-30 04:13:56.188

Just added "\.\d{3}" int the pattern and it now works again.

What I really need to do is simplify the pattern to ignore non-essentials (.*) and focus on:
a) the parts I control (such as a message in the line)
b) the necessary parts that are architected not to change (i.e., format and position of the IP to block)

[BallyBasic79]

Nice job puzzling that out.

For reference, the grep search is:

Code: Select all

grep -P 'PATTERN' file

It's best to match only the essential identifying phrase and the IP, then ignore the rest. Although my rules have '.*' at the ends, It appears unnecessary to match the whole line with the pattern because any match identifies the log line as a match..

HTH

[ocahui]

I tested the changed regex file by catenating a line to the watched log. Sure enough, an entry was logged in lfd.log of the form:
(myftpmatch) expected info [LF_CUSTOMTRIGGER]

As a result, I am pretty sure it is now working.

However, I have some more question related to this topic.

Question 1: When this was working last year the /etc/csf/csf.deny file would acquire a corresponding entry for each blocked IP. I had assumed that csf used csf.deny to keep track of what was to be blocked and for how long. However, this does not appear to occur with the current version of csf/lfd. My question boils down to: Where does csf now keep track of the blocking information, such that it will survive restart/reboot etc.?

Question 2: The instructions in regex.custom.pm imply that the "ports to block" returned on a successful match must be a comma delimited list of literal port numbers. Is that true or can I return (for instance) $2 to indicate the 2nd back reference from the match? (which would be a single port number)

Question 3: Is there a way to request "block all ports for this IP"?

[BallyBasic79]

Question 1: When this was working last year the /etc/csf/csf.deny file would acquire a corresponding entry for each blocked IP. I had assumed that csf used csf.deny to keep track of what was to be blocked and for how long. However, this does not appear to occur with the current version of csf/lfd. My question boils down to: Where does csf now keep track of the blocking information, such that it will survive restart/reboot etc.?

There are two categories of blocks: temporary and permanent (and permanent permanent).
Your config will determine which are used and how. Be sure to read:

Code: Select all

###############################################################################
# SECTION:Login Failure Blocking and Alerts
###############################################################################
# LF_TRIGGER = "0" => triggers are application specific
# LF_TRIGGER_PERM = "1" => the IP is blocked permanently
# LF_TRIGGER_PERM = "3600" => the IP is blocked temporarily for 1 hour

###############################################################################
# SECTION:Temp to Perm/Netblock Settings
###############################################################################
# Temporary to Permanent IP blocking. The following enables this feature to
# permanently block IP addresses that have been temporarily blocked more than
# LF_PERMBLOCK_COUNT times in the last LF_PERMBLOCK_INTERVAL seconds. Set
# LF_PERMBLOCK  to "1" to enable this feature
#
# Care needs to be taken when setting LF_PERMBLOCK_INTERVAL as it needs to be
# at least LF_PERMBLOCK_COUNT multiplied by the longest temporary time setting
# (TTL) for blocked IPs, to be effective
#
# Set LF_PERMBLOCK to "0" to disable this feature
LF_PERMBLOCK = "1"
LF_PERMBLOCK_INTERVAL = "86400"
LF_PERMBLOCK_COUNT = "4"
LF_PERMBLOCK_ALERT = "0"

You need to pay attention to LF_TRIGGER, LF_TRIGGER_PERM and LF_SELECT settings.

These dictate if:

• a block is temporary or permanent,
• if and when temporary blocks get converted to permanent,
• and if blocks are just application/port-specific or global (all ports.)

And I highly recommend:

Code: Select all

###############################################################################
# SECTION:Temp to Perm/Netblock Settings
###############################################################################
# Permanently block IPs by network class. The following enables this feature
# to permanently block classes of IP address where individual IP addresses
# within the same class LF_NETBLOCK_CLASS have already been blocked more than
# LF_NETBLOCK_COUNT times in the last LF_NETBLOCK_INTERVAL seconds. Set
# LF_NETBLOCK  to "1" to enable this feature
#
# This can be an affective way of blocking DDOS attacks launched from within
# the same network class
#
# Valid settings for LF_NETBLOCK_CLASS are "A", "B" and "C", care and
# consideration is required when blocking network classes A or B
#
# Set LF_NETBLOCK to "0" to disable this feature
LF_NETBLOCK = "1"
LF_NETBLOCK_INTERVAL = "86400"
LF_NETBLOCK_COUNT = "2"
LF_NETBLOCK_CLASS = "C"
LF_NETBLOCK_ALERT = "1"

Temp blocks are stored in /var/lib/csf. Don't mess with these files.
Permanent blocks are stored in csf.deny to the limit of DENY_IP_LIMIT. Once the limit is reached, the oldest records will be rotated out – unless you add the text "do not delete" to the comments of an entry.

Question 2: The instructions in regex.custom.pm imply that the "ports to block" returned on a successful match must be a comma delimited list of literal port numbers. Is that true or can I return (for instance) $2 to indicate the 2nd back reference from the match? (which would be a single port number)

It is true and you can try to return a specific port in back reference. It likely will work. For me, any machine violating one of my rules is up to no good and worth blocking altogether.

Question 3: Is there a way to request "block all ports for this IP"?

Yes. Do not specify a port, as in "".

[ocahui]

There are two categories of blocks: temporary and permanent (and permanent permanent).
Your config will determine which are used and how. Be sure to read:
......
Temp blocks are stored in /var/lib/csf. Don't mess with these files.
Permanent blocks are stored in deny.txt to the limit of DENY_IP_LIMIT. Once the limit is reached, the oldest records will be rotated out – unless you add the text "do not delete" to the comments of an entry.

I found the files with the temp IP blocks. I now see how that works.

Thanks for reminding me of the LF_PERMBLOCK, LF_NETBLOCK details. After reviewing that, I saw I needed to make some changes to the PERMBLOCK settings and the temporary intervals in my regex.custom.pm so they work together sanely. (I had the temporary block interval in the pm file set too long, and the LF_PERMBLOCK_INTERVAL set too short)

On the port question: That does not seem to work on my system as documented. No matter what I put into that argument, all ports are blocked in both directions. The port setting is basically ignored. But as you remarked "For me, any machine violating one of my rules is up to no good and worth blocking altogether." I agree, and therefore blocking all ports makes sense.

I have LF_NETBLOCK set to 0, because I believe this too often "throws out the baby with the bath". My email server's IP got blocked by some large actors (such as AT&T) simply because it happened to reside in a block that had a server that got compromised. (However, I do use CC_DENY, a form of netblocking.)

I also don't block any IPs permanently, because an IP used by a spammer today, could be allocated to a perfectly legitimate prospective customer tomorrow. Again, personal experience provides insight. I got assigned an IP on one occasion that had previously been assigned to a zombie server.

[BallyBasic79]

Temp blocks are stored in /var/lib/csf. Don't mess with these files.
Permanent blocks are stored in csfdeny to the limit of DENY_IP_LIMIT. Once the limit is reached, the oldest records will be rotated out – unless you add the text "do not delete" to the comments of an entry.

On the port question: That does not seem to work on my system as documented. No matter what I put into that argument, all ports are blocked in both directions.

Note that I made a correction. Permanent blocks are stored in csf.deny, not deny.txt.

What is the syntax of your port declaration?

This just came in while I was typing this reply:

Subject: lfd on server: Network class C 160.20.111.0/24 (NO/Norway/-/-/-/[AS203377 Mehmet Uzunca]) has been blocked

Time: Mon Oct 7 11:26:07 2019 -0700
Block: 160.20.111.0/24 (NO/Norway/-/-/-/[AS203377 Mehmet Uzunca])
Hits: 3

IP addresses that triggered the block
Mon Oct 7 10:10:05 2019 (junkmailer) Junkmail sender 160.20.111.64 (NO/Norway/-/-/-/[AS203377 Mehmet Uzunca]): 1 in the last 300 secs
Mon Oct 7 10:42:42 2019 (junkmailer) Junkmail sender 160.20.111.65 (NO/Norway/-/-/-/[AS203377 Mehmet Uzunca]): 1 in the last 300 secs
Mon Oct 7 11:26:05 2019 (junkmailer) Junkmail sender 160.20.111.66 (NO/Norway/-/-/-/[AS203377 Mehmet Uzunca]): 1 in the last 300 secs

Notice the IP addresses on the individual blocks. I'd bet money on the IP of the next junkmail that was/will be sent to the server. Double or nothing it is an ugly baby!

Through the combination of strategic countries, ASN, and netblocks, I've reduced spam, email account attempts, and log size by 96%.