How to block an abusive bot? (majestic12.co)

[alexio]

We're getting hit hard by the infamous (and probably nefarious) "majestic12.co.uk/bot".

I've added the various IP's it uses to "Deny", but am wondering...:

Is there a way to block ALL requests coming from "majestic12.co.uk/bot"?

Thanks in advance...

[marcele]

Place the following file:
Centos / RHEL
/etc/httpd/conf.d/bad_bot.conf

Debian / Ubuntu
/etc/apache2/conf.d/bad_bot.conf

Content

Code: Select all

<IfModule mod_setenvif.c>
# SetEnvIfNoCase User-Agent ^$ bad_bot
SetEnvIfNoCase User-Agent "^MJ12bot" bad_bot
SetEnvIfNoCase User-Agent "^MJ12bot/v1.4.5" bad_bot

# Apache < 2.4
<IfModule !mod_authz_core.c>
  <FilesMatch "(.*)">
    Order allow,deny
    Allow from all
    Deny from env=bad_bot
  </FilesMatch>
</IfModule>

# Apache >= 2.4
<IfModule mod_authz_core.c>
  <FilesMatch "(.*)">
    <RequireAll>
      Require all granted
      Require not env bad_bot
    </RequireAll>
  </FilesMatch>
</IfModule>
</IfModule>

Restart Apache! This will block MJ12bot useragent.

[wonderwall]

Will this also work in an Nginx+Apache setup?

[marcele]

Will this also work in an Nginx+Apache setup?

Yes it would.

[wonderwall]

Great - thanks for the quick reply.