CSF/LFD Very Slow with Alma, Repeated Error Messages

[jman]

AlmaLinux: v8.9.0 Standard KVM
cPanel: 120.0.8
CSF: 14.20
Perl: 5 v26

Hi, this may not be anything to do with csf/lfd per se; just checking.

Just elevated from CentOS, and am getting lfd notification emails every 10 minutes:
Error: Failed to detect code [ej7sUoB0R3icb6lAwEgjXo17sLK8u] in SYSLOG_LOG [/var/log/messages]

And cron messages being sent every hour:
logger: socket /dev/log: Connection refused

Manually sending messages via logger while shelled in (logger "my_test_message") also fails with the same socket error.

Additionally, both CSF and LFD are extremely slow to come up in WHM's GUI (> 1 minute), and trying to restart them via ssh and systemctl just hangs.

FWIW, /var/log/messages is accumulating entries, and Mailscanner appears to be working fine.

I did update csf's config entry for RESTRICT_SYSLOG from 1 to 3, set the restrict group to "systemd-journal" and restarted csf, but that had no offect.

Logger failing from the command line says this *probably* isn't a csf/lfd issue per se. Am on a dedicated VPS with plenty of memory/cpu/storage, and did not have these issues prior to elevating, so it may be something to do with Alma. Have reached out to cPanel but not yet heard back.

Has anyone else run into this?

[jman]

Oh, the bracketed code in the lfd emails is different every time, and as expected, grepping for the code in /var/log/messages produces no result.

[jman]

Oops, just noticed I was *not* supposed to use something already in /etc/group, so changed it to "csf_syslog". Will have to wait to see if that does anything.

[jman]

'k, per https://forum.configserver.com/viewtopi ... und#p34009

Checked paths in csf.conf and some of the old centOS paths were /sbin while Alma uses /usr/sbin. Adjusted as needed and restarted, but still hanging on restart via CLI, and very slow to come up in the GUI. Waiting to see if this affects the logging and cron emails...

[jman]

Reinstalled csf, no change.

Is anyone else seeing this behavior?

cPanel doesn't want to touch it as it's not their script.

Please advise.

[warmwhisky]

Hi,

I have just created a new server with Rocky Linux and WHM and am seeing exactly the same message

Code: Select all

Error: Failed to detect code [xMxgbsroyRhTJoNncKyKya] in SYSLOG_LOG [/var/log/messages]

SYSLOG may not be running correctly on server.server.com

Did you ever solve it?

[Texas Trophy]

I have this issue as well and I am told it's server related but I am not so sure. It only occurs when the server is rebooted. It's not a fix but what I have to do so it doesn't constantly trigger the SYSLOG error is delete all files in /var/log/journal/[alphanumeric_string]/ and then reboot the server again.

[jestep]

Same here with an upgrade from CentOS7. I have not found a solution, reverting is not an option on this server. I've been scouring threads and also did a diff comparison from a working server running alma which was also an upgrade and there's no difference in csf.conf between these 2 servers for me. I don't get any errors using journalctl --verify as well, and have went ahead and rotated and restarted the rsyslog and systemd-journald and no change. Following just in case someone figures out what this might be caused by.

[jestep]

Separately, if you're using cc_allow_filter, it can dramatically slow down the csf/lfd restarts, I have some servers that take 30 minutes to restart csf when using these filters. I would disable that while testing if it is being used, if not disregard.

[jestep]

Posting for anyone who might be frustrated by this, by this I mean the errors, and this may not be the fix for everyone but this was the cause of my issues which IMO were not fixable through csf and/or syslog and similar for good reason.

This is so seemingly obvious right now, but after the alma elevation, I failed to realize the real issue here was the kernel and not the csf configuration. In my case, the server was still using a centos7 kernel because leapp failed installing the new kernel and properly updating the grub bootloader, even though the process completed successfully, and the server rebooted fine, which was the cause of the issues even though the control panel and server were 99% functional.

The easiest way IMO if you want to see if this is your issue is in whm, run Security Advisor under security center, and see if there is a mismatch in your kernel. Or cli check the running kernel (uname -sr) and see if it's not almalinux. That was the later obvious issue to correct the running kernel with the updated alma kernel, once fixed, the errors went away completely.

Issue number 2 which I'm pointing out that I ran into in case anyone hits it, when I wasn't paying close attention to the output, was initially generating grub config and didn't notice that mine didn't generate any actual usable configuration, which then of course I bricked this server when I rebooted it. Detailed here that someone else had the same issue with that I stumbled on: https://serverfault.com/questions/11035 ... ly-grub-at

Anyway, just posting all this in case it might help someone else, I stupidly spend an insane amount of time looking in the wrong direction on this, hopefully can save someone else from the same mistakes.