Memcache Connect Issues

[Jose Avila]

memcache_pconnect(): Can't connect to 10.2.10.1:1024, Unknown error (0)

I'm constantly experiencing memcache cant connect errors with the php  
client. I've verified that my host is accesible from my clients. Is  
there any specific OS settings, switch settings or network settings  
that might help reduce this problem, The memcache server is up and  
accesible at all time I have a separate script that is monitoring it  
every 10 seconds. As my traffic has ramped up so have the complains  
even though the host 10.2.10.1 is up. I have about 25 servers in my  
pool serving memcache and apache.

It also alternates on which server it shows down. It randomly reports  
them.


Things I've verified so we don't go over same thing.

Telnet from host reporting that cant connect to host reported down
telnet 10.2.10.1 1024
Escape character is '^]'.
stats
STAT pid 15803
STAT uptime 690148
STAT time 1203727021
STAT version 1.2.4
STAT pointer_size 32
STAT rusage_user 14166.857373
STAT rusage_system 25006.382801
STAT curr_items 1382805
STAT total_items 630076453
STAT bytes 474558385
STAT curr_connections 2156
STAT total_connections 236286
STAT connection_structures 3480
STAT cmd_get 573972717
STAT cmd_set 630031077
STAT get_hits 538547803
STAT get_misses 35424914
STAT evictions 10156980
STAT bytes_read 271917263446
STAT bytes_written 365614267814
STAT limit_maxbytes 536870912
STAT threads 4


Iptable is disabled in every host

iptables -L
Chain INPUT (policy ACCEPT)
target     prot opt source               destination

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination

I even unloaded the iptables module from the kernel to prevent  
connection status overload.