Debian 12 - LFD doesn't read logs

[Vark]

Hi,
I see that Debian 12 isn't yet listed on the working distros, but not sure where to report this from my testing. Following my test server being updated to Deb 12 (Bookworm), CSF appears to work, but LFD fails to read logs. I get the failed SYSLOG email warnings - although it is writing the check code to SYSLOG, it's not able to read it. Looking at the debug log under debug level 2 every log file it scans comes back with 0 lines, even though the log files are being added to.

Jun 28 08:23:37 vroster lfd[642990]: debug: Tick: 5 [5]
Jun 28 08:23:37 vroster lfd[642990]: debug: Parsing /var/log/apache2/error.log (0 lines)
Jun 28 08:23:37 vroster lfd[642990]: debug: Parsing /var/log/messages (0 lines)
Jun 28 08:23:37 vroster lfd[642990]: debug: Parsing /var/log/customlog (0 lines)
Jun 28 08:23:37 vroster lfd[642990]: debug: Parsing /var/log/auth.log (0 lines)
Jun 28 08:23:37 vroster lfd[642990]: debug: Parsing /var/log/mail.log (0 lines)
Jun 28 08:23:37 vroster lfd[642990]: debug: Parsing /var/log/secure (0 lines)

(Note - syslog isn't in the above because I'd turned the syslog check back off to prevent me being overloaded with warning emails, as I'm aware it's not working)

A snippet of strace gives this

newfstatat(AT_FDCWD, "/var/log/syslog", {st_mode=S_IFREG|0640, st_size=8543894, ...}, 0) = 0
newfstatat(AT_FDCWD, "/etc/localtime", {st_mode=S_IFREG|0644, st_size=419, ...}, 0) = 0
getuid() = 0
openat(AT_FDCWD, "/var/log/lfd.log", O_WRONLY|O_CREAT|O_APPEND|O_CLOEXEC, 0666) = 9
lseek(9, 0, SEEK_END) = 61531240
ioctl(9, TCGETS, 0x7ffd86991c10) = -1 ENOTTY (Inappropriate ioctl for device)
lseek(9, 0, SEEK_CUR) = 61531240
newfstatat(9, "", {st_mode=S_IFREG|0600, st_size=61531240, ...}, AT_EMPTY_PATH) = 0
flock(9, LOCK_EX) = 0
getpid() = 641384
write(9, "Jun 28 07:37:34 vroster lfd[6413"..., 78) = 78
close(9) = 0

perl -v

This is perl 5, version 36, subversion 0 (v5.36.0) built for x86_64-linux-gnu-thread-multi
(with 52 registered patches, see perl -V for more detail)


Any ideas how I can troubleshoot this further?

Thanks

[marcele]

Most OS are switching to using journald instead of rsyslog. If you look at the Debian 12 announcement you will see this:

https://www.debian.org/News/2023/20230610

Quote:
There are some changes to system logging; the rsyslog package is no longer needed on most systems, and is not installed by default. Users may change to journalctl or use the new "high precision timestamps" that rsyslog now uses.

First I would double check that rsyslog is installed, enabled and running. I have a feature request to add journald support here:

https://forum.configserver.com/viewtopic.php?t=12656

[Vark]

Thanks for response. Yes, I'd seen that - this was an upgrade from Bullseye, so rsyslogd is still there and running.

ps ax | grep rsyslogd
823 ? Ssl 11:50 /usr/sbin/rsyslogd -n

Wondering if any access rights to read the logs changed, given it can still write to them.

[marcele]

My guess is that it might be related to this then:

Users may change to journalctl or use the new "high precision timestamps" that rsyslog now uses.

If the timestamp format has changed then the regex that CSF uses would have to account for that. Unfortunately that also means that you would likely have to wait for chirpy to fix it

[Vark]

Thanks - I tried reverting to non-high precision format as a test just now, still no luck.

[marcele]

Regex patterns are stored in /usr/local/csf/lib/ConfigServer/RegexMain.pm. It's pretty easy to do some failed logins then test the regex using the matching pattern in that file. Then you can use a website like https://regex101.com/ to see if the pattern works.

Can you post some sample failed logins from your Debian 12 logs?

[Vark]

The Regex pattern correctly identifies the code against an exemplar from the log, the issue appears to be it's simply not getting anything from the log when it tries to read.

I added some extra output to the getlogfile function with LFD itself to get an idea of what it might be grabbing, and I get blank lines, even though the filesize has increased (ignore the Vark prefix - that was just an easy way to GREP the log file):

(Original open of log file)
Jul 1 17:40:47 vroster lfd[936300]: Vark: For /var/log/syslog we now have inode 34866459 and size 17629198

(some checks later ...)
Jul 1 17:54:24 vroster lfd[937171]: Vark: For /var/log/syslog we now have inode 34866459 (was 34866459) and size 17650010 (was 17647656)
Jul 1 17:54:24 vroster lfd[937171]: Vark: FD for /var/log/syslog is GLOB(0x556083f04328)
Jul 1 17:54:24 vroster lfd[937171]: Vark: We read , error return was Inappropriate ioctl for device

So the readline call just gets nothing, and I'm not sure that the error returned is appropriate. I'm getting close to what I can vaguely remember in PERL right now - any ideas?

[Vark]

I found a solution! It's to do with changes in Perl 5.38

readline no longer clears errors, so once one read fails, so does every further read

--> https://perldoc.perl.org/perldelta#read ... -eof-flags

You can overcome this by amending the lfd script and adding a clearerr() at the right spot

Code: Select all

sub getlogfile {
        my $logfile = shift;
        my $lfn = shift;
        my $totlines = shift;
    my $ino;
        my $size;
    my $line;
        my $count;



    if (!defined($lffd[$lfn]))  {
                if (&openlogfile($logfile,$lfn)) {return undef}
    }

    $lffd[$lfn]->clearerr(); <-- THIS IS THE ADDED LINE

Once that was in place, LFD started seeing lines in the logs again.

There may be other places this will catch things out, but in my case this was triggered because one file in the list of logfiles to read doesn't actually exist - which gave an error - which never cleared.

Hope that helps the maintainers.

[armator]

Can you indicate the path of the file to be modified?