Cleanup csf.deny if netblock is added, delete ip's

[Karel]

If csf.deny reaches it maximum limit [DENY_IP_LIMIT =] then the oldest entries are removed unless "do not delete" is specified.

To avoid quick removal of rules due to DENY_IP_LIMIT please clean up csf.deny after a Netblock is added. When a Netblock is added there are multiple single ip adresses redundant in csf.deny and can be removed from csf.deny. No need to double block them.
This way important blocks are longer active in csf.deny.

See example below from csf.deny.
188.165.15.0/24 # lfd: (NETBLOCK) 188.165.15.0/24
Older entries in this range are individual ip adresses

Code: Select all

188.165.15.61 # lfd: (PERMBLOCK) 188.165.15.61 (FR/France/boson073.ahrefs.com) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 13:16:38 2014
188.165.15.211 # lfd: (PERMBLOCK) 188.165.15.211 (FR/France/boson046.ahrefs.com) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 13:40:31 2014
188.165.15.222 # lfd: (PERMBLOCK) 188.165.15.222 (FR/France/boson081.ahrefs.com) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 13:43:03 2014
188.165.15.50 # lfd: (PERMBLOCK) 188.165.15.50 (FR/France/boson048.ahrefs.com) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 13:43:18 2014
188.165.15.22 # lfd: (PERMBLOCK) 188.165.15.22 (FR/France/boson015.ahrefs.com) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 13:50:46 2014
188.165.15.87 # lfd: (PERMBLOCK) 188.165.15.87 (FR/France/boson023.ahrefs.com) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 13:52:41 2014
188.165.15.210 # lfd: (PERMBLOCK) 188.165.15.210 (FR/France/boson078.ahrefs.com) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 13:56:19 2014
188.165.15.200 # lfd: (PERMBLOCK) 188.165.15.200 (FR/France/boson007.ahrefs.com) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 13:57:34 2014
188.165.15.227 # lfd: (PERMBLOCK) 188.165.15.227 (FR/France/boson082.ahrefs.com) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 14:04:20 2014
46.105.168.170 # lfd: (PERMBLOCK) 46.105.168.170 (FR/France/-) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 14:05:36 2014
94.228.34.207 # lfd: (PERMBLOCK) 94.228.34.207 (GB/United Kingdom/-) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 14:08:02 2014
188.165.15.121 # lfd: (PERMBLOCK) 188.165.15.121 (FR/France/boson069.ahrefs.com) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 14:10:13 2014
78.129.218.35 # lfd: (PERMBLOCK) 78.129.218.35 (GB/United Kingdom/-) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 14:14:58 2014
193.201.224.70 # lfd: (PERMBLOCK) 193.201.224.70 (UA/Ukraine/-) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 14:21:24 2014
5.135.214.250 # lfd: (PERMBLOCK) 5.135.214.250 (FR/France/-) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 14:37:29 2014
69.12.88.57 # lfd: (PERMBLOCK) 69.12.88.57 (US/United States/69.12.88.57.static.quadranet.com) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 14:39:00 2014
188.165.15.13 # lfd: (PERMBLOCK) 188.165.15.13 (FR/France/boson060.ahrefs.com) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 14:41:52 2014
188.165.15.176 # lfd: (PERMBLOCK) 188.165.15.176 (FR/France/boson011.ahrefs.com) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 14:43:33 2014
188.165.15.152 # lfd: (PERMBLOCK) 188.165.15.152 (FR/France/boson030.ahrefs.com) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 14:48:44 2014
5.157.41.77 # lfd: (PERMBLOCK) 5.157.41.77 (LU/Luxembourg/-) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 14:53:31 2014
95.42.29.42 # lfd: (PERMBLOCK) 95.42.29.42 (BG/Bulgaria/95-42-29-42.btc-net.bg) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 14:58:02 2014
188.165.15.214 # lfd: (PERMBLOCK) 188.165.15.214 (FR/France/boson080.ahrefs.com) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 14:58:22 2014
188.165.15.10 # lfd: (PERMBLOCK) 188.165.15.10 (FR/France/boson013.ahrefs.com) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 14:59:08 2014
188.165.15.44 # lfd: (PERMBLOCK) 188.165.15.44 (FR/France/boson083.ahrefs.com) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 15:09:17 2014
192.255.69.174 # lfd: (PERMBLOCK) 192.255.69.174 (US/United States/-) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 15:15:55 2014
46.105.168.162 # lfd: (PERMBLOCK) 46.105.168.162 (FR/France/-) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 15:18:33 2014
188.165.15.41 # lfd: (PERMBLOCK) 188.165.15.41 (FR/France/boson084.ahrefs.com) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 15:22:50 2014
78.129.131.15 # lfd: (PERMBLOCK) 78.129.131.15 (GB/United Kingdom/plush.champday.com) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 15:26:40 2014
188.165.15.95 # lfd: (PERMBLOCK) 188.165.15.95 (FR/France/boson064.ahrefs.com) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 15:36:00 2014
188.165.15.129 # lfd: (PERMBLOCK) 188.165.15.129 (FR/France/boson010.ahrefs.com) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 15:36:25 2014
188.165.15.81 # lfd: (PERMBLOCK) 188.165.15.81 (FR/France/boson091.ahrefs.com) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 15:38:15 2014
111.13.2.132 # lfd: (PERMBLOCK) 111.13.2.132 (CN/China/-) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 15:45:55 2014
188.165.15.183 # lfd: (PERMBLOCK) 188.165.15.183 (FR/France/boson088.ahrefs.com) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 15:50:12 2014
188.165.15.32 # lfd: (PERMBLOCK) 188.165.15.32 (FR/France/boson020.ahrefs.com) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 15:58:56 2014
188.165.15.59 # lfd: (PERMBLOCK) 188.165.15.59 (FR/France/boson090.ahrefs.com) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 16:00:23 2014
188.165.15.126 # lfd: (PERMBLOCK) 188.165.15.126 (FR/France/boson009.ahrefs.com) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 16:25:14 2014
188.165.15.36 # lfd: (PERMBLOCK) 188.165.15.36 (FR/France/boson097.ahrefs.com) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 16:27:07 2014
188.165.15.181 # lfd: (PERMBLOCK) 188.165.15.181 (FR/France/boson035.ahrefs.com) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 16:28:53 2014
188.165.15.205 # lfd: (PERMBLOCK) 188.165.15.205 (FR/France/boson042.ahrefs.com) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 17:09:49 2014
188.165.15.238 # lfd: (PERMBLOCK) 188.165.15.238 (FR/France/boson005.ahrefs.com) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 17:16:07 2014
188.165.15.5 # lfd: (PERMBLOCK) 188.165.15.5 (FR/France/boson071.ahrefs.com) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 17:17:39 2014
69.64.37.118 # lfd: (PERMBLOCK) 69.64.37.118 (US/United States/static-ip-69-64-37-118.inaddr.ip-pool.com) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 17:35:59 2014
188.165.15.23 # lfd: (PERMBLOCK) 188.165.15.23 (FR/France/boson000.ahrefs.com) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 17:38:41 2014
192.240.198.164 # lfd: (PERMBLOCK) 192.240.198.164 (US/United States/-) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 17:44:43 2014
113.105.93.80 # lfd: (PERMBLOCK) 113.105.93.80 (CN/China/-) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 18:00:16 2014
107.190.163.130 # lfd: (PERMBLOCK) 107.190.163.130 (IE/Ireland/-) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 18:09:12 2014
188.165.15.207 # lfd: (PERMBLOCK) 188.165.15.207 (FR/France/boson003.ahrefs.com) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 18:12:38 2014
69.12.88.56 # lfd: (PERMBLOCK) 69.12.88.56 (US/United States/69.12.88.56.static.quadranet.com) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 18:13:03 2014
188.165.15.162 # lfd: (PERMBLOCK) 188.165.15.162 (FR/France/boson072.ahrefs.com) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 18:14:18 2014
188.165.15.14 # lfd: (PERMBLOCK) 188.165.15.14 (FR/France/boson014.ahrefs.com) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 18:14:19 2014
188.165.15.198 # lfd: (PERMBLOCK) 188.165.15.198 (FR/France/boson039.ahrefs.com) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 18:22:39 2014
188.165.138.2 # lfd: (PERMBLOCK) 188.165.138.2 (FI/Finland/188-165-138-2.kimsufi.com) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 18:23:24 2014
192.230.61.68 # lfd: (PERMBLOCK) 192.230.61.68 (US/United States/-) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 18:30:08 2014
188.165.15.49 # lfd: (PERMBLOCK) 188.165.15.49 (FR/France/boson017.ahrefs.com) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 18:40:08 2014
188.165.15.236 # lfd: (PERMBLOCK) 188.165.15.236 (FR/France/boson056.ahrefs.com) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 18:47:16 2014
192.240.207.238 # lfd: (PERMBLOCK) 192.240.207.238 (US/United States/-) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 18:54:24 2014
192.255.69.32 # lfd: (PERMBLOCK) 192.255.69.32 (US/United States/-) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 19:03:43 2014
188.165.15.203 # lfd: (PERMBLOCK) 188.165.15.203 (FR/France/boson041.ahrefs.com) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 19:10:14 2014
192.255.69.132 # lfd: (PERMBLOCK) 192.255.69.132 (US/United States/-) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 19:13:45 2014
23.232.255.154 # lfd: (PERMBLOCK) 23.232.255.154 (YE/Yemen/-) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 19:19:01 2014
192.230.61.48 # lfd: (PERMBLOCK) 192.230.61.48 (US/United States/-) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 19:34:01 2014
188.165.15.241 # lfd: (PERMBLOCK) 188.165.15.241 (FR/France/boson094.ahrefs.com) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 19:38:28 2014
192.240.207.171 # lfd: (PERMBLOCK) 192.240.207.171 (US/United States/-) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 19:43:32 2014
192.240.198.177 # lfd: (PERMBLOCK) 192.240.198.177 (US/United States/-) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 20:05:05 2014
192.255.69.77 # lfd: (PERMBLOCK) 192.255.69.77 (US/United States/-) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 20:08:15 2014
192.240.207.37 # lfd: (PERMBLOCK) 192.240.207.37 (US/United States/-) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 20:09:55 2014
192.255.79.159 # lfd: (PERMBLOCK) 192.255.79.159 (US/United States/-) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 20:11:25 2014
192.230.61.50 # lfd: (PERMBLOCK) 192.230.61.50 (US/United States/-) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 20:27:57 2014
23.232.255.16 # lfd: (PERMBLOCK) 23.232.255.16 (YE/Yemen/-) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 20:35:14 2014
192.240.198.205 # lfd: (PERMBLOCK) 192.240.198.205 (US/United States/-) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 20:36:59 2014
188.165.138.172 # lfd: (PERMBLOCK) 188.165.138.172 (FI/Finland/-) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 20:37:19 2014
192.255.79.172 # lfd: (PERMBLOCK) 192.255.79.172 (US/United States/-) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 20:37:44 2014
188.165.15.206 # lfd: (PERMBLOCK) 188.165.15.206 (FR/France/boson043.ahrefs.com) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 20:39:24 2014
192.255.79.106 # lfd: (PERMBLOCK) 192.255.79.106 (US/United States/-) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 20:40:44 2014
192.230.61.9 # lfd: (PERMBLOCK) 192.230.61.9 (US/United States/-) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 20:42:10 2014
188.165.15.191 # lfd: (PERMBLOCK) 188.165.15.191 (FR/France/boson037.ahrefs.com) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 20:42:30 2014
192.255.79.87 # lfd: (PERMBLOCK) 192.255.79.87 (US/United States/-) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 20:45:50 2014
188.165.15.29 # lfd: (PERMBLOCK) 188.165.15.29 (FR/France/boson032.ahrefs.com) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 20:53:12 2014
192.255.69.50 # lfd: (PERMBLOCK) 192.255.69.50 (US/United States/-) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 20:55:33 2014
192.240.198.242 # lfd: (PERMBLOCK) 192.240.198.242 (US/United States/-) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 21:05:48 2014
192.255.69.158 # lfd: (PERMBLOCK) 192.255.69.158 (US/United States/-) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 21:14:52 2014
192.255.79.90 # lfd: (PERMBLOCK) 192.255.79.90 (US/United States/-) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 21:16:52 2014
23.232.255.49 # lfd: (PERMBLOCK) 23.232.255.49 (YE/Yemen/-) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 21:31:06 2014
192.240.207.183 # lfd: (PERMBLOCK) 192.240.207.183 (US/United States/-) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 21:34:18 2014
192.240.198.114 # lfd: (PERMBLOCK) 192.240.198.114 (US/United States/-) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 21:35:08 2014
188.165.15.223 # lfd: (PERMBLOCK) 188.165.15.223 (FR/France/boson050.ahrefs.com) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 21:42:17 2014
188.165.15.37 # lfd: (PERMBLOCK) 188.165.15.37 (FR/France/boson086.ahrefs.com) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 22:00:16 2014
61.164.110.184 # lfd: (PERMBLOCK) 61.164.110.184 (CN/China/-) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 22:05:17 2014
23.232.255.2 # lfd: (PERMBLOCK) 23.232.255.2 (YE/Yemen/-) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 22:05:52 2014
188.165.15.202 # lfd: (PERMBLOCK) 188.165.15.202 (FR/France/boson040.ahrefs.com) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 22:22:11 2014
188.165.15.224 # lfd: (PERMBLOCK) 188.165.15.224 (FR/France/boson051.ahrefs.com) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 22:27:22 2014
188.165.15.64 # lfd: (PERMBLOCK) 188.165.15.64 (FR/France/boson047.ahrefs.com) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 22:28:33 2014
173.208.39.251 # lfd: (PERMBLOCK) 173.208.39.251 (US/United States/173-208-39-251.ipvnow.com) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 23:01:44 2014
218.204.17.70 # lfd: (PERMBLOCK) 218.204.17.70 (CN/China/-) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 23:16:40 2014
188.165.15.89 # lfd: (PERMBLOCK) 188.165.15.89 (FR/France/boson095.ahrefs.com) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 23:27:29 2014
113.107.57.76 # lfd: (PERMBLOCK) 113.107.57.76 (CN/China/-) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 23:34:37 2014
188.165.15.26 # lfd: (PERMBLOCK) 188.165.15.26 (FR/France/boson016.ahrefs.com) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 23:46:44 2014
188.165.15.55 # lfd: (PERMBLOCK) 188.165.15.55 (FR/France/boson021.ahrefs.com) has had more than 4 temp blocks in the last 86400 secs - Sun Nov 16 23:54:26 2014
222.88.236.236 # lfd: (PERMBLOCK) 222.88.236.236 (CN/China/-) has had more than 4 temp blocks in the last 86400 secs - Mon Nov 17 00:18:11 2014
181.49.15.162 # lfd: (PERMBLOCK) 181.49.15.162 (CO/Colombia/-) has had more than 4 temp blocks in the last 86400 secs - Mon Nov 17 00:18:21 2014
188.165.15.0/24 # lfd: (NETBLOCK) 188.165.15.0/24 has had more than 4 blocks in the last 86400 secs - Mon Nov 17 00:20:06 2014

[rootStar]

Seems like I also suggested sort of the same idea >
(Unfortunately I'm not allowed to post the link here).
So please go check for the entry: Advancing the Blacklist (csf.deny)
Was posted on the 27th of Dec 2014 in the CSF Suggestions Forum

I totally agree with Karel that CSF need to at least have the relevant ip(s) automatically cleared when adding a netblock/range.

Furthermore, if considering the pretty much unlimited amount of ip ranges out there. To only have 900 entries total in csf.deny is just not sufficient. ( Maybe it's just me the scum hackers hate most?)

Anyway +1 to what Karel is saying as this would be a major improvement.
Thanks

[marcele]

I like this idea. All that would be needed is to check if the input was a CIDR then loop though all entries and check if the IP fell within that CIDR range. If so then remove it.

It looks like Perl has some nice CIDR functions. I'm not sure if CSF would need to use them or if Chirpy would just right his own. Just search for Net-CIDR or Net-CIDR-Lite on CPAN.

[kpmedia]

I'd also like it if "# do not delete" could automatically be appended to the netblocks. Some may not want this, but many admins will want the option. If the traffic is junk once, it usually will be later as well.

Even with 2k entries allowed, the firewall is recycling every 2-3 days!

..