Global ignore list seems to cause high CPU usage

[Domineaux]

Hello Chirpy,
I have recently been seeing frequent high CPU usage reported for LFD processes that appears to occur during the retrieval of the global ignore list we use which contains around 100 specific IP addresses and 15 class C blocks like 123.123.123.0/24 .
Is there any reason that you know of for this to be contributing excessively to high CPU usage?

ps. the master global ignore list does have comments for each IP or range in the following format:

# IP ranges for monitoring systems
123.123.122.1 #Monitor one
123.123.123.0/24 #Monitoring network one

[Doobla]

The problem is that LFD uses upwards of 100% cpu for up to 3 seconds when checking the (locally cached) global ignore list. Since this is done continually in LFD then LFD becomes a cpu hog. The loop which is responsible for the "continual" part begins with

Code: Select all

while (1)  {
        $0 = "lfd - processing";

The actual code block that is generating all of the cpu usage is

Code: Select all

        if (-e "/etc/csf/csf.gignore") {
                open (IN, "</etc/csf/csf.gignore") or &cleanup(__LINE__,$!);
                flock (IN, LOCK_SH);
                my @ignore = <IN>;
                close (IN);
                chomp @ignore;
                foreach my $line (@ignore) {
                if ($line =~ /^(\#|\n|\r|\s)/ or $line eq "") {next}
                        my ($first,undef) = split(/\s/,$line);
                        my ($ip,$iscidr) = split(/\//,$first);
                        if (&checkip($first)) {
                                if ($iscidr) {push @cidrs,$first} else {$ignoreips{$ip} = 1}
                        }
                        elsif ($ip ne "127.0.0.1") {&logfile("Invalid CIDR in csf.gignore: [$first]")}
                }
                foreach my $entry (@cidrs) {
                        if (&checkip($entry) == 6) {
                                eval {$cidr6->add($entry)};
                        } else {
                                eval {$cidr->add($entry)};
                        }
                        if ($@) {&logfile("Invalid CIDR in csf.gignore: $entry")}
                }
        }

Oddly, after restarting LFD and allowing it to complete its initial sanity/security checks against itself, the cpu usage is low. It only increases gradually over time.

I've tried changing the content of the /etc/csf/csf.gignore file even to the point of emptying it completely and the high cpu usage still occurs. The only way to address this is to set GLOBAL_IGNORE = "" in /etc/csf/csf.conf and then remove the /etc/csf/csf.gignore file completely. Once that is done and csf+lfd are restarted, then cpu usage stays sane.

We're seeing this on a variety of server types from the single core lowly xen vps servers to high end 16 core dells. LFD, when acting problematic, will simply use a full core for 2-3 seconds at a time and then since LF_PARSE defaults to 5 seconds then you have an almost continual cpu hog.

[ForumAdmin]

If you modify that code to read:

Code: Select all

	if (-e "/etc/csf/csf.gignore") {
		open (IN, "</etc/csf/csf.gignore") or &cleanup(__LINE__,$!);
		flock (IN, LOCK_SH);
		my @ignore = <IN>;
		close (IN);
		chomp @ignore;
		foreach my $line (@ignore) {
		if ($line =~ /^(\#|\n|\r|\s)/ or $line eq "") {next}
			my ($first,undef) = split(/\s/,$line);
			my ($ip,$iscidr) = split(/\//,$first);
			if (&checkip($first)) {
				if ($iscidr) {push @cidrs,$first} else {$ignoreips{$ip} = 1}
			}
			elsif ($ip ne "127.0.0.1") {&logfile("Invalid CIDR in csf.gignore: [$first]")}
		}
		foreach my $entry (@cidrs) {
			if (&checkip($entry) == 6) {
				eval {$cidr6->add($entry)};
			} else {
				eval {$cidr->add($entry)};
			}
			if ($@) {&logfile("Invalid CIDR in csf.gignore: $entry")}
		}
		unlink "/etc/csf/csf.gignore";
	}

Then restart lfd it should resolve the issue. We'll look into it in more depth if anything else is needed and then release a new version.

[Domineaux]

Thanks that should help as a quick and dirty improvement.
Hope you can get a more refined way of handling into a future release.

[Doobla]

The problem I can see with that solution is that if lfd is restarted then it will lose any knowledge of our global ignore list. Or am I misreading the code?

Thanks for the quick reply. If I can get verification on the above then I'll work on implementing this tonight.

[ForumAdmin]

No, that file should be deleted. It is created when the latest GLOBAL_IGNORE file is downloaded which happens when lfd restarts and every LF_GLOBAL seconds. This triggers lfd to reload the list of ignored entries that were just downloaded. We should have a new release out in a few minutes that resolves this issue and actually improves the load time.

[Doobla]

Perfect. Appreciate the quick turnaround.

[ForumAdmin]

v5.59 has been released to resolve this issue. Thank you for reporting the issue and in particular Doobla for the detailed response.

[Doobla]

Awesome. Thank you.

[Domineaux]

Thanks for the quick resolution Chirpy.