I've optimized and fixed a bug for the user-agent trigger. Also note that this list is the result of analyzing user-agents from production servers. The good thing is that its easy to adjust the regex for your own needs.Mick wrote:Regarding your 'custom login failure triggers to handle bad scrapers', if I wanted to add further user agents, is there anything I should be aware of?
There is quite an extensive list here:
https://github.com/JayBizzle/Crawler-De ... ct.php#L73
and although I don't want to add them all, there are some that are hitting my sites I would like to add.
Thanks for any help.
Firehol iplists
Re: Firehol iplists
Re: Firehol iplists
Perfect, thank you for the update.