ConfigServer Community Forum

Peer support forums for ConfigServer Scripts
https://mail.forum.configserver.com/

CSF Global Deny Lists - Not Properly Removing IP (IPSet Issue)

https://mail.forum.configserver.com/viewtopic.php?t=9129

Page 1 of 1

CSF Global Deny Lists - Not Properly Removing IP (IPSet Issue)

Posted: 06 Jan 2016, 23:59
by JoshDargie
Hoping someone can help point me in the right direction...

I've developed a solution to generate a global list for both allowing IP's & denying IP's. Populating the csf.gdeny works perfectly, every seven (7) minutes the server calls for the file reads it and blacklists the IP's.

Unfortunately, the server does not seem to be releasing IP's that have been removed from the list. csf.gdeny no longer has the IP blocked however doing a "csf -g" commands shows its blacklisted.

Server Configuration:
CentOS: v6.7 x86_64bit
cPanel: v11.52.2 (build 1)
IPTables: v1.4.7
IPSet: v6.11, protocol version: 6
CSF: v8.08 (cPanel)

How I've determined this issue ..

Initial Server Load:

Code: Select all

root@server [/var/lib/csf]# cat csf.gdeny | wc -l
1199

Code: Select all

root@server [/var/lib/csf]# grep 174.1.144.2 csf.gdeny
root@server [/var/lib/csf]#

Code: Select all

root@server [/var/lib/csf]# csf -g 174.1.144.2

Chain            num   pkts bytes target     prot opt in     out     source               destination         
No matches found for 174.1.144.2 in iptables

IPSET: No matches found for 174.1.144.2

Code: Select all

root@server [/var/lib/csf]# ipset -test chain_GDENY 174.1.144.2
174.1.144.2 is NOT in set chain_GDENY.
As you can see the IP is not found in any of the reports.... So now we enable the blacklisting of the IP on the global deny list file.

Global Deny IP Address:

Code: Select all

root@server [/var/lib/csf]# cat csf.gdeny | wc -l
1200

Code: Select all

root@server [/var/lib/csf]# grep 174.1.144.2 csf.gdeny
174.1.144.2
root@server [/var/lib/csf]#

Code: Select all

root@server [/var/lib/csf]# csf -g 174.1.144.2

Chain            num   pkts bytes target     prot opt in     out     source               destination         
No matches found for 174.1.144.2 in iptables


IPSET: Set:chain_GDENY Match:174.1.144.2 Setting:GLOBAL_DENY

Code: Select all

root@server [/var/lib/csf]# ipset -test chain_GDENY 174.1.144.2
174.1.144.2 is in set chain_GDENY.
So the IP has properly been blocked and is set correctly in IPSet, and read correctly from CSF... Now to remove the IP Address from the Global Deny List...

Removal of IP Address:

Code: Select all

root@server [/var/lib/csf]# cat csf.gdeny | wc -l
1199

Code: Select all

root@server [/var/lib/csf]# grep 174.1.144.2 csf.gdeny
root@server [/var/lib/csf]#

Code: Select all

root@server [/var/lib/csf]# csf -g 174.1.144.2

Chain            num   pkts bytes target     prot opt in     out     source               destination         
No matches found for 174.1.144.2 in iptables


IPSET: Set:chain_GDENY Match:174.1.144.2 Setting:GLOBAL_DENY

Code: Select all

root@server [/var/lib/csf]# ipset -test chain_GDENY 174.1.144.2
174.1.144.2 is in set chain_GDENY.
So it seems, well CSF properly fetchs the Global Deny file & writes it to its temporary csf.gdeny file it doesn't properly flush & configure IPSet. Only way to fully release the IP in question (and to be clear it could be any IP) is to restart CSF ("csf -r").

Looking for solution to fix this problem.

Re: CSF Global Deny Lists - Not Properly Removing IP

Posted: 07 Jan 2016, 04:25
by Sergio
What command are you using to release the IP from the iptable?

Re: CSF Global Deny Lists - Not Properly Removing IP

Posted: 07 Jan 2016, 05:25
by JoshDargie
I'm not running anything, its just ConfigServer Firewall doing the process.

CSF fetches the global deny list and parses it. It's then suppose to push the updates to IPSet (which it does for adding IP's not releasing them).

Seems that the flush command isn't executing properly during the chain swap.

CSF's Global Deny Code for LFD

Code: Select all

               if ($config{GLOBAL_DENY}) {
                        my ($status, $text) = &urlget($config{GLOBAL_DENY});
                        if ($status) {
                                &logfile("Unable to retrieve global deny list - $text");
                        } else {
                                if (&csflock) {&lockfail("GLOBAL_DENY")}
                                &logfile("Global Deny - retrieved and blocking IP address ranges");
                                my $drop = $config{DROP};
                                if ($config{DROP_IP_LOGGING}) {$drop = "BLOCKDROP"}

                                if ($config{SAFECHAINUPDATE}) {
                                       	&iptablescmd(__LINE__,"$config{IPTABLES} -N NEWGDENYIN");
                                       	&iptablescmd(__LINE__,"$config{IPTABLES} -N NEWGDENYOUT");
                                       	if ($config{LF_IPSET}) {
                                               	my $pktin = $config{DROP};
                                               	my $pktout = $config{DROP};
                                               	if ($config{DROP_IP_LOGGING}) {$pktin = "LOGDROPIN"}
                                               	if ($config{DROP_OUT_LOGGING}) {$pktout = "LOGDROPOUT"}
                                               &iptablescmd(__LINE__,"$config{IPTABLES} -A NEWGDENYIN -m set --match-set chain_GDENY src -j $pktin");
                                              unless ($config{LF_BLOCKINONLY}) {&iptablescmd(__LINE__,"$config{IPTABLES} -A NEWGDENYOUT -m set --match-set chain_GDENY dst -j $pktout")}
                                       	}
                                        if ($config{IPV6}) {
                                               	&iptablescmd(__LINE__,"$config{IP6TABLES} -N NEWGDENYIN");
                                               	&iptablescmd(__LINE__,"$config{IP6TABLES} -N NEWGDENYOUT");
                                       	}
                               	} else {
                                        &iptablescmd(__LINE__,"$config{IPTABLES} -F GDENYIN");
                                        &iptablescmd(__LINE__,"$config{IPTABLES} -F GDENYOUT");
                                        if ($config{LF_IPSET}) {
                                                my $pktin = $config{DROP};
                                               	my $pktout = $config{DROP};
                                               	if ($config{DROP_IP_LOGGING}) {$pktin = "LOGDROPIN"}
                                               	if ($config{DROP_OUT_LOGGING}) {$pktout = "LOGDROPOUT"}
                                               	&iptablescmd(__LINE__,"$config{IPTABLES} -A GDENYIN -m set --match-set chain_GDENY src -j $pktin");
                                               	unless ($config{LF_BLOCKINONLY}) {&iptablescmd(__LINE__,"$config{IPTABLES} -A GDENYOUT -m set --match-set chain_GDENY dst -j $pktout")}
}
                                        if ($config{IPV6}) {
                                                &iptablescmd(__LINE__,"$config{IP6TABLES} -F GDENYIN");
                                                &iptablescmd(__LINE__,"$config{IP6TABLES} -F GDENYOUT");
                                        }
                                }
                                sysopen (GDENY, "/var/lib/csf/csf.gdeny", O_WRONLY | O_CREAT) or &childcleanup(__LINE__,"*Error* Cannot open out file: $!");
                                flock (GDENY, LOCK_EX);
                                seek (GDENY, 0, 0);
                                truncate (GDENY, 0);
                                if ($config{FASTSTART}) {$faststart = 1}
                                foreach my $line (split (/\n/,$text)) {
                                        if ($line =~ /^\#/) {next}
                                        my ($ip,$comment) = split (/\s/,$line,2);
                                        print GDENY "$ip\n";
                                        if ($config{SAFECHAINUPDATE}) {
                                                &linefilter($ip, "deny","NEWGDENY");
                                        } else {
                                                &linefilter($ip, "deny","GDENY");
                                        }
                                }
                                if ($config{FASTSTART}) {&faststart("GLOBAL_DENY")}
                                close (GDENY);
                               if ($config{SAFECHAINUPDATE}) {
                                        &iptablescmd(__LINE__,"$config{IPTABLES} -A LOCALINPUT $ethdevin -j NEWGDENYIN");
                                        &iptablescmd(__LINE__,"$config{IPTABLES} -A LOCALOUTPUT $ethdevout -j NEWGDENYOUT");
                                        &iptablescmd(__LINE__,"$config{IPTABLES} -D LOCALINPUT $ethdevin -j GDENYIN");
                                        &iptablescmd(__LINE__,"$config{IPTABLES} -D LOCALOUTPUT $ethdevout -j GDENYOUT");
                                        &iptablescmd(__LINE__,"$config{IPTABLES} -F GDENYIN");
                                        &iptablescmd(__LINE__,"$config{IPTABLES} -F GDENYOUT");
                                        &iptablescmd(__LINE__,"$config{IPTABLES} -X GDENYIN");
                                        &iptablescmd(__LINE__,"$config{IPTABLES} -X GDENYOUT");
                                        &iptablescmd(__LINE__,"$config{IPTABLES} -E NEWGDENYIN GDENYIN");
                                        &iptablescmd(__LINE__,"$config{IPTABLES} -E NEWGDENYOUT GDENYOUT");
                                        if ($config{IPV6}) {
                                                &iptablescmd(__LINE__,"$config{IP6TABLES} -A LOCALINPUT $eth6devin -j NEWGDENYIN");
                                                &iptablescmd(__LINE__,"$config{IP6TABLES} -A LOCALOUTPUT $eth6devout -j NEWGDENYOUT");
                                                &iptablescmd(__LINE__,"$config{IP6TABLES} -D LOCALINPUT $eth6devin -j GDENYIN");
                                                &iptablescmd(__LINE__,"$config{IP6TABLES} -D LOCALOUTPUT $eth6devout -j GDENYOUT");
                                                &iptablescmd(__LINE__,"$config{IP6TABLES} -F GDENYIN");
                                                &iptablescmd(__LINE__,"$config{IP6TABLES} -F GDENYOUT");
                                                &iptablescmd(__LINE__,"$config{IP6TABLES} -X GDENYIN");
                                                &iptablescmd(__LINE__,"$config{IP6TABLES} -X GDENYOUT");
                                                &iptablescmd(__LINE__,"$config{IP6TABLES} -E NEWGDENYIN GDENYIN");
                                                &iptablescmd(__LINE__,"$config{IP6TABLES} -E NEWGDENYOUT GDENYOUT");
                                        }
                                }
                        }
                }

Re: CSF Global Deny Lists - Not Properly Removing IP

Posted: 07 Jan 2016, 23:41
by JoshDargie
Extra Update:

I can confirm that this works perfectly fine when running normally with IPTables. This is only an issue with the IPSet configuration.

Unfortunately I'm not the most proficient with perl but based on the lfd.pl code it would seem that the chain's are being matched (merged) before writting the new ranges into it. Instead a new chain should be created which is replaced by the old chain.

Could be way off, but that logic would seem to make sense and resolve the issue I'm seeing.

Re: CSF Global Deny Lists - Not Properly Removing IP (IPSet Issue)

Posted: 08 Jan 2016, 03:19
by JoshDargie
Additional Update:

I turned on CSF debug log and noticed the following errors.

CSF Debug to /var/log/lfd.log:

Code: Select all

Jan  7 22:16:57 server lfd[15339]: Global Deny - retrieved and blocking IP address ranges
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:03 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:04 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:04 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:04 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:04 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:04 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:04 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:04 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:04 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:04 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:04 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:04 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:04 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:04 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:04 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:04 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:04 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:04 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:04 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:04 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:04 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:04 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:04 server lfd[15339]: *Error* IPSET: [ipset v6.11: The set with the given name does not exist]
Jan  7 22:17:10 server lfd[15339]: csf is currently restarting - command [/sbin/iptables -A LOCALINPUT ! -i lo -j NEWGDENYIN] skipped on line 5429
Does this mean CSF's lfd.pl file is not properly calling its IPSet Chain name causing it to not properly remove the IP's which have been removed from the Global Deny .txt list?

Re: CSF Global Deny Lists - Not Properly Removing IP (IPSet Issue)

Posted: 17 Jan 2016, 11:53
by ForumAdmin
We've found the issue with GLOBAL_ALLOW and GLOBAL_DENY when using LF_IPSET and will have a fix for this in the next release of csf.
All times are UTC
Page 1 of 1

Powered by phpBB® Forum Software © phpBB Limited