Page 1 of 1
Slow POP3 mail receiving
Posted: 23 Oct 2007, 21:37
by valkira
On two cPanel11 boxes with CSF we have identified a problem while receiving mail using POP3/IMAP.
During the send/receive process the client (Outlook, Thunderbird) shows the time remaining as 30+ mins for 50K e-mails, it happens randomly (not every send/receive) to random clients (but during some time all of them have experienced the problem at least once). We've tried and tested all possible options, but stopping csf is the only way to get the POP3/IMAP connections to work fine.
Client SMTP and server SMTP sessions are not affected, also none of our cPanel10 boxes with csf have any problems.
Has any one else experienced the same or similar problems?
Posted: 24 Oct 2007, 21:01
by JRKy
We have experienced an issue before with slow SMTP but that turned out to be the slow resolvers.
Perhaps you might want to take a peek at them jut to rule it out?
Posted: 25 Oct 2007, 18:10
by valkira
With resolvers you mean DNS resolving?
Posted: 25 Oct 2007, 18:42
by JRKy
Correct, I was referring to DNS resolvers.
Posted: 25 Oct 2007, 22:56
by valkira
No, the resolvers are not a problem.
Anyway, none of our cPanel10 servers with csf are affected, only cPanel11 boxes. Slow receiving happens on all connection speeds, it's also more likely to happen while receiving larger mails (with or without attachments like servers logwatch or backup completed emails).
Stopping the firewall resolves all problems with email (and of course opens the server to all trigger happy hackers
)
Just remembered: everything was fine until we did a csf upgrade, and I think the problem started after upgrading to 2.89 (from 2.87 I think) and escalated even more after upgrading to 2.90 - if it matters
Posted: 05 Nov 2007, 09:33
by chirpy
Nothing was changed between those versions that would affect iptables chains at all. I would guess that if anything has changed, it's the cPanel courier-imap configuration somehow. Have you check the messages log for any spurious iptables blocks when you're seeing the problem?
Posted: 21 Nov 2007, 15:28
by valkira
well here is one example, today on client had a complaint about slow POP3 connection, after checking the /var/log/lfd.log and /var/log/messages i found this in the messages log:
Nov 21 11:50:14 orion kernel: Firewall: *TCP_OUT Blocked* IN= OUT=eth0 SRC=213.147.96.57 DST=83.131.217.21 LEN=1440 TOS=0x00 PREC=0x00 TTL=64 ID=9943 DF PROTO=TCP SPT=110 DPT=1699 WINDOW=5840 RES=0x00 ACK URGP=0
Nov 21 11:52:14 orion kernel: Firewall: *TCP_OUT Blocked* IN= OUT=eth0 SRC=213.147.96.57 DST=83.131.217.21 LEN=1440 TOS=0x00 PREC=0x00 TTL=64 ID=9945 DF PROTO=TCP SPT=110 DPT=1699 WINDOW=5840 RES=0x00 ACK URGP=0
Nov 21 11:54:14 orion kernel: Firewall: *TCP_OUT Blocked* IN= OUT=eth0 SRC=213.147.96.57 DST=83.131.217.21 LEN=1440 TOS=0x00 PREC=0x00 TTL=64 ID=9947 DF PROTO=TCP SPT=110 DPT=1699 WINDOW=5840 RES=0x00 ACK URGP=0
Nov 21 11:56:14 orion kernel: Firewall: *TCP_OUT Blocked* IN= OUT=eth0 SRC=213.147.96.57 DST=83.131.217.21 LEN=1440 TOS=0x00 PREC=0x00 TTL=64 ID=9949 DF PROTO=TCP SPT=110 DPT=1699 WINDOW=5840 RES=0x00 ACK URGP=0
Nov 21 11:58:14 orion kernel: Firewall: *TCP_OUT Blocked* IN= OUT=eth0 SRC=213.147.96.57 DST=83.131.217.21 LEN=1440 TOS=0x00 PREC=0x00 TTL=64 ID=9951 DF PROTO=TCP SPT=110 DPT=1699 WINDOW=5840 RES=0x00 ACK URGP=0
Nov 21 12:00:14 orion kernel: Firewall: *TCP_OUT Blocked* IN= OUT=eth0 SRC=213.147.96.57 DST=83.131.217.21 LEN=1440 TOS=0x00 PREC=0x00 TTL=64 ID=9953 DF PROTO=TCP SPT=110 DPT=1699 WINDOW=5840 RES=0x00 ACK URGP=0
Nov 21 12:02:14 orion kernel: Firewall: *TCP_OUT Blocked* IN= OUT=eth0 SRC=213.147.96.57 DST=83.131.217.21 LEN=1440 TOS=0x00 PREC=0x00 TTL=64 ID=9955 DF PROTO=TCP SPT=110 DPT=1699 WINDOW=5840 RES=0x00 ACK URGP=0
Nov 21 12:04:14 orion kernel: Firewall: *TCP_OUT Blocked* IN= OUT=eth0 SRC=213.147.96.57 DST=83.131.217.21 LEN=1440 TOS=0x00 PREC=0x00 TTL=64 ID=9957 DF PROTO=TCP SPT=110 DPT=1699 WINDOW=5840 RES=0x00 ACK URGP=0
The DST address is his, the SRC address is our server (CentOS 4.5, WHM 11.2.0 cPanel 11.10.0-S16448) further investigations revealed blocked connections on several needed ports (21, 22, 25, 80 etc). I can also attach my csf.conf file if needed.
I assume there could be a problem with the configuration, because I've noticed that putting TESTING = "1" seem to reduce these problems (not confirmed by clients, just less calls by them
).
Posted: 10 Dec 2007, 10:00
by chirpy
Enabled TESTING disables the iptables configuration after 5 minutes, so the firewall is down.
Those log lines suggest that the email client to the server connection tracking is timing out and causing problems with iptables.
Posted: 15 Dec 2007, 10:13
by valkira
Now I have disabled CT_LIMIT and restarted csf. Then I browsed a couple of sites on the server and here is a part of /var/log/messages:
{pegasus} root #3 10:54 [/var/log]> grep 'xxx.xxx.xx.xxx' messages
Dec 15 10:55:18 pegasus kernel: Firewall: *INVALID* IN=eth1 OUT= MAC=00:1a:4b
41:02:00:15:62:4a:39:80:08:00 SRC=xxx.xxx.xxx.xxx DST=yyy.yyy.yyy.yyy LEN=52 TOS=0x00 PREC=0x00 TTL=118 ID=42240 DF PROTO=TCP SPT=3708 DPT=80 WINDOW=17520 RES=0x00 ACK URGP=0
Dec 15 10:55:18 pegasus kernel: Firewall: *INVALID* IN=eth1 OUT= MAC=00:1a:4b
41:02:00:15:62:4a:39:80:08:00 SRC=xxx.xxx.xxx.xxx DST=yyy.yyy.yyy.yyy LEN=52 TOS=0x00 PREC=0x00 TTL=118 ID=42242 DF PROTO=TCP SPT=3708 DPT=80 WINDOW=17520 RES=0x00 ACK URGP=0
Dec 15 10:55:18 pegasus kernel: Firewall: *INVALID* IN=eth1 OUT= MAC=00:1a:4b
41:02:00:15:62:4a:39:80:08:00 SRC=xxx.xxx.xxx.xxx DST=yyy.yyy.yyy.yyy LEN=52 TOS=0x00 PREC=0x00 TTL=118 ID=42260 DF PROTO=TCP SPT=3708 DPT=80 WINDOW=17520 RES=0x00 ACK URGP=0
Dec 15 10:55:18 pegasus kernel: Firewall: *INVALID* IN=eth1 OUT= MAC=00:1a:4b
41:02:00:15:62:4a:39:80:08:00 SRC=xxx.xxx.xxx.xxx DST=yyy.yyy.yyy.yyy LEN=52 TOS=0x00 PREC=0x00 TTL=118 ID=42263 DF PROTO=TCP SPT=3708 DPT=80 WINDOW=17520 RES=0x00 ACK URGP=0
Dec 15 10:55:19 pegasus kernel: Firewall: *INVALID* IN=eth1 OUT= MAC=00:1a:4b
41:02:00:15:62:4a:39:80:08:00 SRC=xxx.xxx.xxx.xxx DST=yyy.yyy.yyy.yyy LEN=52 TOS=0x00 PREC=0x00 TTL=118 ID=42266 DF PROTO=TCP SPT=3708 DPT=80 WINDOW=17520 RES=0x00 ACK URGP=0
where xxx.xxx.xxx.xxx is my IP address and yyy.yyy.yyy.yyy is servers IP address
Posted: 26 Dec 2007, 15:21
by chirpy
That's iptables rejecting the connection because it's receiving an ACK out of sequence. That suggests either a network or a client application, or perhaps a server resource issue. I've seen such issues if the servers is low on memory resources.