CSF Global Deny Lists - Not Properly Removing IP (IPSet Issue)

This forum is only for reproducible bugs with csf and lfd (i.e. not iptables problems, lack of understanding how to use a feature, etc). Posts must be accompanied with full technical details of the problem and how it can be recreated. Any posts not adhering to this, or not considered bugs, will be moved to the General Discussion (csf) forum.
Post Reply
JoshDargie
Junior Member
Posts: 4
Joined: 01 Jan 2016, 19:16

CSF Global Deny Lists - Not Properly Removing IP (IPSet Issue)

Post by JoshDargie »

Hoping someone can help point me in the right direction...

I've developed a solution to generate a global list for both allowing IP's & denying IP's. Populating the csf.gdeny works perfectly, every seven (7) minutes the server calls for the file reads it and blacklists the IP's.

Unfortunately, the server does not seem to be releasing IP's that have been removed from the list. csf.gdeny no longer has the IP blocked however doing a "csf -g" commands shows its blacklisted.

Server Configuration:
CentOS: v6.7 x86_64bit
cPanel: v11.52.2 (build 1)
IPTables: v1.4.7
IPSet: v6.11, protocol version: 6
CSF: v8.08 (cPanel)

How I've determined this issue ..

Initial Server Load:

Code: Select all

root@server [/var/lib/csf]# cat csf.gdeny | wc -l
1199

Code: Select all

root@server [/var/lib/csf]# grep 174.1.144.2 csf.gdeny
root@server [/var/lib/csf]#

Code: Select all

root@server [/var/lib/csf]# csf -g 174.1.144.2

Chain            num   pkts bytes target     prot opt in     out     source               destination         
No matches found for 174.1.144.2 in iptables

IPSET: No matches found for 174.1.144.2

Code: Select all

root@server [/var/lib/csf]# ipset -test chain_GDENY 174.1.144.2
174.1.144.2 is NOT in set chain_GDENY.
As you can see the IP is not found in any of the reports.... So now we enable the blacklisting of the IP on the global deny list file.

Global Deny IP Address:

Code: Select all

root@server [/var/lib/csf]# cat csf.gdeny | wc -l
1200

Code: Select all

root@server [/var/lib/csf]# grep 174.1.144.2 csf.gdeny
174.1.144.2
root@server [/var/lib/csf]#

Code: Select all

root@server [/var/lib/csf]# csf -g 174.1.144.2

Chain            num   pkts bytes target     prot opt in     out     source               destination         
No matches found for 174.1.144.2 in iptables


IPSET: Set:chain_GDENY Match:174.1.144.2 Setting:GLOBAL_DENY

Code: Select all

root@server [/var/lib/csf]# ipset -test chain_GDENY 174.1.144.2
174.1.144.2 is in set chain_GDENY.
So the IP has properly been blocked and is set correctly in IPSet, and read correctly from CSF... Now to remove the IP Address from the Global Deny List...

Removal of IP Address:

Code: Select all

root@server [/var/lib/csf]# cat csf.gdeny | wc -l
1199

Code: Select all

root@server [/var/lib/csf]# grep 174.1.144.2 csf.gdeny
root@server [/var/lib/csf]#

Code: Select all

root@server [/var/lib/csf]# csf -g 174.1.144.2

Chain            num   pkts bytes target     prot opt in     out     source               destination         
No matches found for 174.1.144.2 in iptables


IPSET: Set:chain_GDENY Match:174.1.144.2 Setting:GLOBAL_DENY

Code: Select all

root@server [/var/lib/csf]# ipset -test chain_GDENY 174.1.144.2
174.1.144.2 is in set chain_GDENY.
So it seems, well CSF properly fetchs the Global Deny file & writes it to its temporary csf.gdeny file it doesn't properly flush & configure IPSet. Only way to fully release the IP in question (and to be clear it could be any IP) is to restart CSF ("csf -r").

Looking for solution to fix this problem.
Last edited by JoshDargie on 08 Jan 2016, 00:27, edited 1 time in total.
Top
Sergio
Junior Member
Posts: 1712
Joined: 12 Dec 2006, 14:56

Re: CSF Global Deny Lists - Not Properly Removing IP

Post by Sergio »

What command are you using to release the IP from the iptable?
Top
JoshDargie
Junior Member
Posts: 4
Joined: 01 Jan 2016, 19:16

Re: CSF Global Deny Lists - Not Properly Removing IP

Post by JoshDargie »

I'm not running anything, its just ConfigServer Firewall doing the process.

CSF fetches the global deny list and parses it. It's then suppose to push the updates to IPSet (which it does for adding IP's not releasing them).

Seems that the flush command isn't executing properly during the chain swap.

CSF's Global Deny Code for LFD

Code: Select all

               if ($config{GLOBAL_DENY}) {
                        my ($status, $text) = &urlget($config{GLOBAL_DENY});
                        if ($status) {
                                &logfile("Unable to retrieve global deny list - $text");
                        } else {
                                if (&csflock) {&lockfail("GLOBAL_DENY")}
                                &logfile("Global Deny - retrieved and blocking IP address ranges");
                                my $drop = $config{DROP};
                                if ($config{DROP_IP_LOGGING}) {$drop = "BLOCKDROP"}

                                if ($config{SAFECHAINUPDATE}) {
                                       	&iptablescmd(__LINE__,"$config{IPTABLES} -N NEWGDENYIN");
                                       	&iptablescmd(__LINE__,"$config{IPTABLES} -N NEWGDENYOUT");
                                       	if ($config{LF_IPSET}) {
                                               	my $pktin = $config{DROP};
                                               	my $pktout = $config{DROP};
                                               	if ($config{DROP_IP_LOGGING}) {$pktin = "LOGDROPIN"}
                                               	if ($config{DROP_OUT_LOGGING}) {$pktout = "LOGDROPOUT"}
                                               &iptablescmd(__LINE__,"$config{IPTABLES} -A NEWGDENYIN -m set --match-set chain_GDENY src -j $pktin");
                                              unless ($config{LF_BLOCKINONLY}) {&iptablescmd(__LINE__,"$config{IPTABLES} -A NEWGDENYOUT -m set --match-set chain_GDENY dst -j $pktout")}
                                       	}
                                        if ($config{IPV6}) {
                                               	&iptablescmd(__LINE__,"$config{IP6TABLES} -N NEWGDENYIN");
                                               	&iptablescmd(__LINE__,"$config{IP6TABLES} -N NEWGDENYOUT");
                                       	}
                               	} else {
                                        &iptablescmd(__LINE__,"$config{IPTABLES} -F GDENYIN");
                                        &iptablescmd(__LINE__,"$config{IPTABLES} -F GDENYOUT");
                                        if ($config{LF_IPSET}) {
                                                my $pktin = $config{DROP};
                                               	my $pktout = $config{DROP};
                                               	if ($config{DROP_IP_LOGGING}) {$pktin = "LOGDROPIN"}
                                               	if ($config{DROP_OUT_LOGGING}) {$pktout = "LOGDROPOUT"}
                                               	&iptablescmd(__LINE__,"$config{IPTABLES} -A GDENYIN -m set --match-set chain_GDENY src -j $pktin");
                                               	unless ($config{LF_BLOCKINONLY}) {&iptablescmd(__LINE__,"$config{IPTABLES} -A GDENYOUT -m set --match-set chain_GDENY dst -j $pktout")}
}
                                        if ($config{IPV6}) {
                                                &iptablescmd(__LINE__,"$config{IP6TABLES} -F GDENYIN");
                                                &iptablescmd(__LINE__,"$config{IP6TABLES} -F GDENYOUT");
                                        }
                                }
                                sysopen (GDENY, "/var/lib/csf/csf.gdeny", O_WRONLY | O_CREAT) or &childcleanup(__LINE__,"*Error* Cannot open out file: $!");
                                flock (GDENY, LOCK_EX);
                                seek (GDENY, 0, 0);
                                truncate (GDENY, 0);
                                if ($config{FASTSTART}) {$faststart = 1}
                                foreach my $line (split (/\n/,$text)) {
                                        if ($line =~ /^\#/) {next}
                                        my ($ip,$comment) = split (/\s/,$line,2);
                                        print GDENY "$ip\n";
                                        if ($config{SAFECHAINUPDATE}) {
                                                &linefilter($ip, "deny","NEWGDENY");
                                        } else {
                                                &linefilter($ip, "deny","GDENY");
                                        }
                                }
                                if ($config{FASTSTART}) {&faststart("GLOBAL_DENY")}
                                close (GDENY);
                               if ($config{SAFECHAINUPDATE}) {
                                        &iptablescmd(__LINE__,"$config{IPTABLES} -A LOCALINPUT $ethdevin -j NEWGDENYIN");
                                        &iptablescmd(__LINE__,"$config{IPTABLES} -A LOCALOUTPUT $ethdevout -j NEWGDENYOUT");
                                        &iptablescmd(__LINE__,"$config{IPTABLES} -D LOCALINPUT $ethdevin -j GDENYIN");
                                        &iptablescmd(__LINE__,"$config{IPTABLES} -D LOCALOUTPUT $ethdevout -j GDENYOUT");
                                        &iptablescmd(__LINE__,"$config{IPTABLES} -F GDENYIN");
                                        &iptablescmd(__LINE__,"$config{IPTABLES} -F GDENYOUT");
                                        &iptablescmd(__LINE__,"$config{IPTABLES} -X GDENYIN");
                                        &iptablescmd(__LINE__,"$config{IPTABLES} -X GDENYOUT");
                                        &iptablescmd(__LINE__,"$config{IPTABLES} -E NEWGDENYIN GDENYIN");
                                        &iptablescmd(__LINE__,"$config{IPTABLES} -E NEWGDENYOUT GDENYOUT");
                                        if ($config{IPV6}) {
                                                &iptablescmd(__LINE__,"$config{IP6TABLES} -A LOCALINPUT $eth6devin -j NEWGDENYIN");
                                                &iptablescmd(__LINE__,"$config{IP6TABLES} -A LOCALOUTPUT $eth6devout -j NEWGDENYOUT");
                                                &iptablescmd(__LINE__,"$config{IP6TABLES} -D LOCALINPUT $eth6devin -j GDENYIN");
                                                &iptablescmd(__LINE__,"$config{IP6TABLES} -D LOCALOUTPUT $eth6devout -j GDENYOUT");
                                                &iptablescmd(__LINE__,"$config{IP6TABLES} -F GDENYIN");
                                                &iptablescmd(__LINE__,"$config{IP6TABLES} -F GDENYOUT");
                                                &iptablescmd(__LINE__,"$config{IP6TABLES} -X GDENYIN");
                                                &iptablescmd(__LINE__,"$config{IP6TABLES} -X GDENYOUT");
                                                &iptablescmd(__LINE__,"$config{IP6TABLES} -E NEWGDENYIN GDENYIN");
                                                &iptablescmd(__LINE__,"$config{IP6TABLES} -E NEWGDENYOUT GDENYOUT");
                                        }
                                }
                        }
                }
Top
JoshDargie
Junior Member
Posts: 4
Joined: 01 Jan 2016, 19:16

Re: CSF Global Deny Lists - Not Properly Removing IP

Post by JoshDargie »

Extra Update:

I can confirm that this works perfectly fine when running normally with IPTables. This is only an issue with the IPSet configuration.

Unfortunately I'm not the most proficient with perl but based on the lfd.pl code it would seem that the chain's are being matched (merged) before writting the new ranges into it. Instead a new chain should be created which is replaced by the old chain.

Could be way off, but that logic would seem to make sense and resolve the issue I'm seeing.
Top
JoshDargie
Junior Member
Posts: 4
Joined: 01 Jan 2016, 19:16

Re: CSF Global Deny Lists - Not Properly Removing IP (IPSet Issue)

Post by JoshDargie »

Additional Update:

I turned on CSF debug log and noticed the following errors.

CSF Debug to /var/log/lfd.log:

Code: Select all

Jan  7 22:16:57 server lfd[15339]: Global Deny - retrieved and blocking IP address ranges
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:04 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:04 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:04 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:04 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:04 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:04 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:04 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:04 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:04 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:04 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:04 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:04 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:04 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:04 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:04 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:04 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:04 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:04 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:04 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:04 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:04 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:04 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:10 server lfd[15339]: csf is currently restarting - command [/sbin/iptables -A LOCALINPUT ! -i lo -j NEWGDENYIN] skipped on line 5429
Does this mean CSF's lfd.pl file is not properly calling its IPSet Chain name causing it to not properly remove the IP's which have been removed from the Global Deny .txt list?
Top
ForumAdmin
Moderator
Posts: 1524
Joined: 01 Oct 2008, 09:24

Re: CSF Global Deny Lists - Not Properly Removing IP (IPSet Issue)

Post by ForumAdmin »

We've found the issue with GLOBAL_ALLOW and GLOBAL_DENY when using LF_IPSET and will have a fix for this in the next release of csf.
Top
Post Reply

Return to “Report Bugs (csf)”