Page 1 of 1

Huge server load

Posted: 15 Feb 2008, 07:05
by erick_p
Hi. I have being having outages since two days. My host has been looking at it. We tried putting my main host IP behind a Cisco guard etc, changed the MaxClients in Apache (and other tuning) but nothing works. I restart Apache and the server soon enough goes down again.

Yesterday, I was told I'm getting SYN attacks. So I enabled the syncookie and enabled the SYN option in LFD options -- with "5" as the "SYNFLOOD_RATE" option.

Today, I am told by my host that if the firewall (CSF) is running, that is when the server goes down. If we stop the firewall, the server seems fine. I verified this and this indeed seems to be the case!

GIven that I have changed ONLY the syn setting, what can I do? Should I disable the SYNFLOOD setting? Should i disable logging? Should I decrease the size of my mod_security rules? (I'm using chosen rules from the Apache2 version of suggestions from http://www.gotroot.com/mod_security+rules )

Any thoughts on what I can do?? Is this SYNFLOOD setting a bug in CSF?

Thanks

Posted: 15 Feb 2008, 15:00
by deadeye
erick_p wrote:Yesterday, I was told I'm getting SYN attacks. So I enabled the syncookie and enabled the SYN option in LFD options -- with "5" as the "SYNFLOOD_RATE" option.
If the volume and frequency of the SYN attacks is high enough then they will overload any software firewall. This could be what you are seeing. CSF is working, but the volume of attacks it is blocking are high enough that it starts using too many resources. If this is the case then your host needs to blocking this at their router or switch before the traffic ever gets to your server.

Posted: 16 Feb 2008, 08:46
by erick_p
Thanks Charles. Actually now I'm not getting so much traffic, but whenever I restart the CSF firewall, the load goes high and before I know it the server is down. Easily fixed by shutting down CSF/LFD. Any idea why this may be happening and how to debug this?

My host told me something about "hung semaphores" on my Centos 4.4. Any thoughts?

Posted: 16 Feb 2008, 18:16
by deadeye
I'm afraid I probably won't be much help there. I know I'm running CSF with no problems on CentOS 4.6.

If it were me I would start by watching /var/log/lfd.log and var/log/messages to see if I could spot something that was causing the load to rise. Barring that I would turn off as many of the features in CSF as I could and see if I still had the problem. If the problem goes away when you do that then start turning things back on one at a time until you start seeing the problem again. Then post back with which feature it is and maybe we can be of some help.

Like I said though, I'm taking a shot in the dark here, hopefully someone else will have a better suggestion.

Posted: 18 Feb 2008, 10:40
by chirpy
For starters, if you're running CentOS v4.4 it's out of date and you should upgrade your OS. Secondly, you should ensure you're running the latest kernel from the OS vendor. Lastly, make sure you're not running out of memory.