client denied by server configuration

[john01]

Config Server installed Firewall & Security, and uses the two options to run this plugin.

After several hours the plugin have put in, I get error messages in my log DirectAdmin,

Site Summary / Statistics / Logs> Error Log Web Full Error Log

Code: Select all

Sat Feb 18 17: 44: 25.414434 2017] [authz_core: error] [pid 4082] [client 172.68.10.101:21762] AH01630: client denied by server configuration: /home/admin/domains/nieuws-feiten.nl/public_html/ robots.txt
[Sat Feb 18 17: 44: 25.414567 2017] [authz_core: error] [pid 4082] [client 172.68.10.101:21762] AH01630: client denied by server configuration: /home/admin/domains/nieuws-feiten.nl/public_html /403.shtml
[Sat Feb 18 17: 44: 28.858357 2017] [authz_core: error] [pid 2984] [client 172.68.10.17:22705] AH01630: client denied by server configuration: /home/admin/domains/nieuws-feiten.nl/public_html / 12-today-and-news-facts
[Sat Feb 18 17: 44: 28.858453 2017] [authz_core: error] [pid 2984] [client 172.68.10.17:22705] AH01630: client denied by server configuration: /home/admin/domains/nieuws-feiten.nl/public_html /403.shtml

Not one country is locked in Country Code Lists and Settings.
In Edit csf.allow, the IP address allow file (Currently: 17 permanent IP Allows) of Cloedflare added.

The error message is a coincidence of clooudflare.

What is going on here??

[john01]

Wow that's quite useful.

[martexx]

Config Server installed Firewall & Security, and uses the two options to run this plugin.

After several hours the plugin have put in, I get error messages in my log DirectAdmin,

Site Summary / Statistics / Logs> Error Log Web Full Error Log

Code: Select all

Sat Feb 18 17: 44: 25.414434 2017] [authz_core: error] [pid 4082] [client 172.68.10.101:21762] AH01630: client denied by server configuration: /home/admin/domains/nieuws-feiten.nl/public_html/ robots.txt
[Sat Feb 18 17: 44: 25.414567 2017] [authz_core: error] [pid 4082] [client 172.68.10.101:21762] AH01630: client denied by server configuration: /home/admin/domains/nieuws-feiten.nl/public_html /403.shtml
[Sat Feb 18 17: 44: 28.858357 2017] [authz_core: error] [pid 2984] [client 172.68.10.17:22705] AH01630: client denied by server configuration: /home/admin/domains/nieuws-feiten.nl/public_html / 12-today-and-news-facts
[Sat Feb 18 17: 44: 28.858453 2017] [authz_core: error] [pid 2984] [client 172.68.10.17:22705] AH01630: client denied by server configuration: /home/admin/domains/nieuws-feiten.nl/public_html /403.shtml

Not one country is locked in Country Code Lists and Settings.
In Edit csf.allow, the IP address allow file (Currently: 17 permanent IP Allows) of Cloedflare added.

The error message is a coincidence of clooudflare.

What is going on here??

Site Summary / Statistics / Logs> Error Log Web Full Error Log>>>>>>>>>>>>>>>>
>> means that the request came to your website and thus was not stopped by the firewall.

" The error message is a coincidence of clooudflare." is not understandable

Sat Feb 18 17: 44: 25.414434 2017] [authz_core: error] [pid 4082] [client 172.68.10.101:21762] AH01630: client denied by server configuration: /home/admin/domains/nieuws-feiten.nl/public_html/ robots.txt

>> one person using that IP 172.xxx treid to access tobots.txt and was denied acces to it by your server config.
i would check your .htaccess or server config

//ik snap je frustratie, maar zo een toontje help meestal niet

[john01]

Most IP addresses are cloudflare where access is blocked.

Now it might be, in your writing, that perhaps the reason for this could lie in the .htaccess file. This I also checked, it is quite possible that it is hidden therein.

##### Common hacking tools and bandwidth hoggers block -- BEGIN
SetEnvIf user-agent "WebBandit" stayout=1
SetEnvIf user-agent "webbandit" stayout=1
SetEnvIf user-agent "Acunetix" stayout=1
SetEnvIf user-agent "binlar" stayout=1
SetEnvIf user-agent "BlackWidow" stayout=1
SetEnvIf user-agent "Bolt 0" stayout=1
SetEnvIf user-agent "Bot mailto:craftbot@yahoo.com" stayout=1
SetEnvIf user-agent "BOT for JCE" stayout=1
SetEnvIf user-agent "casper" stayout=1
SetEnvIf user-agent "checkprivacy" stayout=1
SetEnvIf user-agent "ChinaClaw" stayout=1
SetEnvIf user-agent "clshttp" stayout=1
SetEnvIf user-agent "cmsworldmap" stayout=1
SetEnvIf user-agent "comodo" stayout=1
SetEnvIf user-agent "Custo" stayout=1
SetEnvIf user-agent "Default Browser 0" stayout=1
SetEnvIf user-agent "diavol" stayout=1
SetEnvIf user-agent "DIIbot" stayout=1
SetEnvIf user-agent "DISCo" stayout=1
SetEnvIf user-agent "dotbot" stayout=1
SetEnvIf user-agent "Download Demon" stayout=1
SetEnvIf user-agent "eCatch" stayout=1
SetEnvIf user-agent "EirGrabber" stayout=1
SetEnvIf user-agent "EmailCollector" stayout=1
SetEnvIf user-agent "EmailSiphon" stayout=1
SetEnvIf user-agent "EmailWolf" stayout=1
SetEnvIf user-agent "Express WebPictures" stayout=1
SetEnvIf user-agent "extract" stayout=1
SetEnvIf user-agent "ExtractorPro" stayout=1
SetEnvIf user-agent "EyeNetIE" stayout=1
SetEnvIf user-agent "feedfinder" stayout=1
SetEnvIf user-agent "FHscan" stayout=1
SetEnvIf user-agent "FlashGet" stayout=1
SetEnvIf user-agent "flicky" stayout=1
SetEnvIf user-agent "GetRight" stayout=1
SetEnvIf user-agent "GetWeb!" stayout=1
SetEnvIf user-agent "Go-Ahead-Got-It" stayout=1
SetEnvIf user-agent "Go!Zilla" stayout=1
SetEnvIf user-agent "grab" stayout=1
SetEnvIf user-agent "GrabNet" stayout=1
SetEnvIf user-agent "Grafula" stayout=1
SetEnvIf user-agent "harvest" stayout=1
SetEnvIf user-agent "HMView" stayout=1
SetEnvIf user-agent "ia_archiver" stayout=1
SetEnvIf user-agent "Image Stripper" stayout=1
SetEnvIf user-agent "Image Sucker" stayout=1
SetEnvIf user-agent "InterGET" stayout=1
SetEnvIf user-agent "Internet Ninja" stayout=1
SetEnvIf user-agent "InternetSeer.com" stayout=1
SetEnvIf user-agent "jakarta" stayout=1
SetEnvIf user-agent "Java" stayout=1
SetEnvIf user-agent "JetCar" stayout=1
SetEnvIf user-agent "JOC Web Spider" stayout=1
SetEnvIf user-agent "kmccrew" stayout=1
SetEnvIf user-agent "larbin" stayout=1
SetEnvIf user-agent "LeechFTP" stayout=1
SetEnvIf user-agent "libwww" stayout=1
SetEnvIf user-agent "Mass Downloader" stayout=1
SetEnvIf user-agent "Maxthon$" stayout=1
SetEnvIf user-agent "microsoft.url" stayout=1
SetEnvIf user-agent "MIDown tool" stayout=1
SetEnvIf user-agent "miner" stayout=1
SetEnvIf user-agent "Mister PiX" stayout=1
SetEnvIf user-agent "NEWT" stayout=1
SetEnvIf user-agent "MSFrontPage" stayout=1
SetEnvIf user-agent "Navroad" stayout=1
SetEnvIf user-agent "NearSite" stayout=1
SetEnvIf user-agent "Net Vampire" stayout=1
SetEnvIf user-agent "NetAnts" stayout=1
SetEnvIf user-agent "NetSpider" stayout=1
SetEnvIf user-agent "NetZIP" stayout=1
SetEnvIf user-agent "nutch" stayout=1
SetEnvIf user-agent "Octopus" stayout=1
SetEnvIf user-agent "Offline Explorer" stayout=1
SetEnvIf user-agent "Offline Navigator" stayout=1
SetEnvIf user-agent "PageGrabber" stayout=1
SetEnvIf user-agent "Papa Foto" stayout=1
SetEnvIf user-agent "pavuk" stayout=1
SetEnvIf user-agent "pcBrowser" stayout=1
SetEnvIf user-agent "PeoplePal" stayout=1
SetEnvIf user-agent "planetwork" stayout=1
SetEnvIf user-agent "psbot" stayout=1
SetEnvIf user-agent "purebot" stayout=1
SetEnvIf user-agent "pycurl" stayout=1
SetEnvIf user-agent "RealDownload" stayout=1
SetEnvIf user-agent "ReGet" stayout=1
SetEnvIf user-agent "Rippers 0" stayout=1
SetEnvIf user-agent "SeaMonkey$" stayout=1
SetEnvIf user-agent "sitecheck.internetseer.com" stayout=1
SetEnvIf user-agent "SiteSnagger" stayout=1
SetEnvIf user-agent "skygrid" stayout=1
SetEnvIf user-agent "SmartDownload" stayout=1
SetEnvIf user-agent "sucker" stayout=1
SetEnvIf user-agent "SuperBot" stayout=1
SetEnvIf user-agent "SuperHTTP" stayout=1
SetEnvIf user-agent "Surfbot" stayout=1
SetEnvIf user-agent "tAkeOut" stayout=1
SetEnvIf user-agent "Teleport Pro" stayout=1
SetEnvIf user-agent "Toata dragostea mea pentru diavola" stayout=1
SetEnvIf user-agent "turnit" stayout=1
SetEnvIf user-agent "vikspider" stayout=1
SetEnvIf user-agent "VoidEYE" stayout=1
SetEnvIf user-agent "Web Image Collector" stayout=1
SetEnvIf user-agent "Web Sucker" stayout=1
SetEnvIf user-agent "WebAuto" stayout=1
SetEnvIf user-agent "WebCopier" stayout=1
SetEnvIf user-agent "WebFetch" stayout=1
SetEnvIf user-agent "WebGo IS" stayout=1
SetEnvIf user-agent "WebLeacher" stayout=1
SetEnvIf user-agent "WebReaper" stayout=1
SetEnvIf user-agent "WebSauger" stayout=1
SetEnvIf user-agent "Website eXtractor" stayout=1
SetEnvIf user-agent "Website Quester" stayout=1
SetEnvIf user-agent "WebStripper" stayout=1
SetEnvIf user-agent "WebWhacker" stayout=1
SetEnvIf user-agent "WebZIP" stayout=1
SetEnvIf user-agent "Wget" stayout=1
SetEnvIf user-agent "Widow" stayout=1
SetEnvIf user-agent "WWW-Mechanize" stayout=1
SetEnvIf user-agent "WWWOFFLE" stayout=1
SetEnvIf user-agent "Xaldon WebSpider" stayout=1
SetEnvIf user-agent "Yandex" stayout=1
SetEnvIf user-agent "Zeus" stayout=1
SetEnvIf user-agent "zmeu" stayout=1
SetEnvIf user-agent "CazoodleBot" stayout=1
SetEnvIf user-agent "discobot" stayout=1
SetEnvIf user-agent "ecxi" stayout=1
SetEnvIf user-agent "GT::WWW" stayout=1
SetEnvIf user-agent "heritrix" stayout=1
SetEnvIf user-agent "HTTP::Lite" stayout=1
SetEnvIf user-agent "HTTrack" stayout=1
SetEnvIf user-agent "ia_archiver" stayout=1
SetEnvIf user-agent "id-search" stayout=1
SetEnvIf user-agent "id-search.org" stayout=1
SetEnvIf user-agent "IDBot" stayout=1
SetEnvIf user-agent "Indy Library" stayout=1
SetEnvIf user-agent "IRLbot" stayout=1
SetEnvIf user-agent "ISC Systems iRc Search 2.1" stayout=1
SetEnvIf user-agent "LinksManager.com_bot" stayout=1
SetEnvIf user-agent "linkwalker" stayout=1
SetEnvIf user-agent "lwp-trivial" stayout=1
SetEnvIf user-agent "MFC_Tear_Sample" stayout=1
SetEnvIf user-agent "Microsoft URL Control" stayout=1
SetEnvIf user-agent "Missigua Locator" stayout=1
SetEnvIf user-agent "panscient.com" stayout=1
SetEnvIf user-agent "PECL::HTTP" stayout=1
SetEnvIf user-agent "PHPCrawl" stayout=1
SetEnvIf user-agent "PleaseCrawl" stayout=1
SetEnvIf user-agent "SBIder" stayout=1
SetEnvIf user-agent "Snoopy" stayout=1
SetEnvIf user-agent "Steeler" stayout=1
SetEnvIf user-agent "URI::Fetch" stayout=1
SetEnvIf user-agent "urllib" stayout=1
SetEnvIf user-agent "Web Sucker" stayout=1
SetEnvIf user-agent "webalta" stayout=1
SetEnvIf user-agent "WebCollage" stayout=1
SetEnvIf user-agent "Wells Search II" stayout=1
SetEnvIf user-agent "WEP Search" stayout=1
SetEnvIf user-agent "zermelo" stayout=1
SetEnvIf user-agent "ZyBorg" stayout=1
SetEnvIf user-agent "Indy Library" stayout=1
SetEnvIf user-agent "libwww-perl" stayout=1
SetEnvIf user-agent "Go!Zilla" stayout=1
SetEnvIf user-agent "TurnitinBot" stayout=1
SetEnvIf user-agent "YandexImages" stayout=1
SetEnvIf user-agent "SiteExplorer" stayout=1
SetEnvIf user-agent "DOC" stayout=1
SetEnvIf user-agent "Zao" stayout=1
SetEnvIf user-agent "Zealbot" stayout=1
SetEnvIf user-agent "MSIECrawler" stayout=1
SetEnvIf user-agent "SiteSnagger" stayout=1
SetEnvIf user-agent "WebStripper" stayout=1
SetEnvIf user-agent "WebCopier" stayout=1
SetEnvIf user-agent "Fetch" stayout=1
SetEnvIf user-agent "Offline Explorer" stayout=1
SetEnvIf user-agent "Teleport" stayout=1
SetEnvIf user-agent "TeleportPro" stayout=1
SetEnvIf user-agent "WebZIP" stayout=1
SetEnvIf user-agent "linko" stayout=1
SetEnvIf user-agent "HTTrack" stayout=1
SetEnvIf user-agent "HTTrack" stayout=1
SetEnvIf user-agent "Xenu" stayout=1
SetEnvIf user-agent "larbin" stayout=1
SetEnvIf user-agent "libwww" stayout=1
SetEnvIf user-agent "ZyBORG" stayout=1
SetEnvIf user-agent "Download Ninja" stayout=1
SetEnvIf user-agent "wget" stayout=1
SetEnvIf user-agent "grub-client" stayout=1
SetEnvIf user-agent "k2spider" stayout=1
SetEnvIf user-agent "NPBot" stayout=1
SetEnvIf user-agent "WebReaper" stayout=1
SetEnvIf user-agent "Custo" stayout=1
SetEnvIf user-agent "Vagabondo" stayout=1
SetEnvIf user-agent "Walhello appie" stayout=1
SetEnvIf user-agent "Netcraft" stayout=1
SetEnvIf user-agent "SurveyBot" stayout=1
SetEnvIf user-agent "Voila" stayout=1
SetEnvIf user-agent "TencentTraveler" stayout=1
SetEnvIf user-agent "psbot" stayout=1
<IfModule !mod_authz_core.c>
deny from env=stayout
</IfModule>

It is strange that if all whitelisted IP addresses in the DirectAdmin as in Config Server Firewall & Security solutions are still blocked.

[martexx]

remove the htaccess file to test it, its way to extensive for others to speak about )))

[john01]

remove the htaccess file to test it, its way to extensive for others to speak about )))

The data contained in the .htaccess file, coming from Admin Tools.
I have had previous run-ins with Admin Tools that disable various functions in Joomla itself.

This is yet another collision with Admin Tools, and I can assume without testing it that Admin Tools also causes this error. Config Server Firewall & Security apparent even in large hosting companies, to be turned away. They indicate, you must install fail2ban. I have therefore once implemented, and soon found out that fail2ban totally does not work if the Config Server Firewall & Security.

With Config Server Firewall & Security. much more one can decide on their own, and it works 100% much better than any script I've ever used before. I could not even imagine that Config Server Firewall & Security.de cause of the error messages could be.

[john01]

Conclusion

The error that is mentioned in the first topic, is caused by Admin Tools. Admin Tools is possible to refuse access various USER-AGENT to your server / website.

The various user agent is also blocked all IP addresses from CloudFlare, as a consequence can be seen in the first topic. Even if you added all IP white list to the white list of the Config Server Firewall & Security, will not help.

Now after few hours later, all errors are gone, so it is advisable not to use Admin Tools when CloudFlare, CDN, DNS, DDoS protection and security for your website.

Would you still want to use Admin Tools, then we will put on a good idea to add a single user agent each time. Check every 30 minutes your Site Summary / Statistics / Logs> Web Full Error Log when errors occur.

This quick fix came about through martexx who made the reference to the .htaccess file.

[martexx]

jeeeeehhh ))) now tell me what is adin tools and why does it creste such a horrible big htaccess??

[john01]

You can read all about Admin tools here
https://www.akeebabackup.com/documentat ... maker.html

Scroll down to Block access from specific user agents