Maxmind Rate Limits?

[p4guru]

Seems this month Maxmind is enforcing it's API rate limits https://support.maxmind.com/hc/en-us/ar ... -Databases

Please note: Beginning in March 2024, MaxMind will begin enforcing its policies around API requests and endpoints. We will require use of the HTTPS protocol for all database download requests, and will require requests to use the proper hostname as listed below, and on our Developer Portal.

For my API downloads I now get too many request 429 when i hit their 30 downloads per day limit

Database download limits
Every account is limited to 2,000 total direct downloads (30 for GeoLite accounts) in a 24-hour period. If you have to distribute your databases across multiple servers, it is advisable that you download databases to a local repository on your network, and distribute them to other servers from there.

How is CSF Maxmind Geolite2 database downloads being handled? If I have 50 servers using CSF Firewall with Maxmind GeoLite2 database method configured using the same Maxmind account, does that mean there's a chance of hitting Maxmind's enforced rate limits?

For my own API requests, the workaround is to download one copy direct via API and have my applications download from that cached version. But CSF Firewall AFAIK doesn't allow setting my own cached download URLs for Maxmind GeoLite2 databases? Maybe a feature to add to CSF in future where I can set in csf.conf a GeoLite2 database URL that is self-hosted?

[p4guru]

Maybe a feature to add to CSF in future where I can set in csf.conf a GeoLite2 database URL that is self-hosted?

would this work by modify the Perl code in /usr/local/csf/lib/ConfigServer/Config.pm to use these variables instead of the hardcoded URLs so I could define in /etc/csf/csf.conf, CC_COUNTRY, CC_CITY and CC_ASN URLs?

Code: Select all

$config{cc_src} = "MaxMind";
$config{asn_src} = "MaxMind";
$config{cc_country} = $config{CC_COUNTRY};
$config{cc_city} = $config{CC_CITY};
$config{cc_asn} = $config{CC_ASN};

[AbduL Karim Hirbetak]

I am also trying to stop CSF from pulling requests from MaxMind servers.

I tried to remove the License Key from csf.conf and I'm encountering this error: "ERROR: Country Code Lookups setting MM_LICENSE_KEY must be set in /etc/csf/csf.conf to continue using the MaxMind databases."

Following MaxMind advice, I'm looking to avoid hitting the limit in the future by downloading each database to a local repository on my network.

I noticed the CSF option CC_SRC = 1 will pull MaxMind databases. How can I prevent CSF from making these requests? Should I set a false license key and then push the CSV database from the local repository? Do you have any recommendations?

[Saxtus]

I guess you could try changing providers.
Maybe this would help if you have many servers, but don't hold me accountable if something goes wrong:

Code: Select all

sed -i '/CC_SRC = "1"/c\CC_SRC = "2"' /etc/csf/csf.conf; csf -ra