help with logfile parsing

[DCBSupafly]

I'm looking to collect data about attackers, and I've been able to get started by parsing lfd.log into a database, but lfd.log only shows brutes; it doesn't show all failed logins and port knocks. I'd like to be able to get all failures. I have found the list of logs at the bottom of the csf.conf file, but writing a parser for each log seems like something I might not have to do, since CSF is parsing those logs already, right?

Is there some log file where csf/lfd aggregate failed logins?

[ForumAdmin]

The simplest way would be to set the DEBUG settings at the bottom of csf.conf to "1" then restart lfd. You will then get more detailed output about detections before blocks are triggered.

The alternative would be to use the BLOCK_REPORT option to have information passed to an external script.

[DCBSupafly]

Thanks a lot,

DEBUG mode will make the job a lot more straightforward! I will mess with BLOCK_REPORT and see what I can get out of that; it would be simpler for many reasons to log these to MySQL in realtime, rather than having to go back through logs after the fact.

Unfortunately the usernames of login attempts aren't passed via DEBUG mode, but this is a good starting point. Thanks again!

[DCBSupafly]

I turned on DEBUG and collected some logs, but DEBUG mode unfortunately still doesn't aggregate any failed logins. It logs startup and shutdown process, successful logins, and blocks, but not the individual failures that caused the blocks. (This actually makes the log harder to parse because there are more lines in it than just the blocks.)

Block_Report will definitely be the best way for me to collect info about blocks, because it makes it so all I have to do is dump the info passed via args into a db and I'm done.

The issue is that the data I want is the attackers that don't get blocked! Far more interesting than the dumb brutes are the ones changing users/ports and hitting just infrequently enough to not get noticed. For example, while combing through logs I found one the other day trying a different email username twice every 20 minutes. That one will never get blocked but I want to take all that info out and pull it apart to detect patterns and trends.

I get that for lfd to store that kind of data would be silly redundant overhead for an application that specifically needs to be lightweight, but that's the data I want

For now I'll focus on Block_Report and look for another solution to the login failure data collection, but if anyone has any tips, I'd greatly appreciate it!