Memcached and maximum persistent connections

[Steve Mushero]

We are having similar problems, but our limit seems to be 10240 - we
have 60 Apache servers with up to 500 processes each, running PHP, so
that's a total of 30,000 possible persistent connections.  This works
well until the server connection count reaches about 10,240 then the
clients get connetion errors and then our DBs get slameed by the
non-cached SQL requests.  We ran non-persistent for a while, but the
connection rate is extremely high and consumes lots of CPU & effort; the
persistent system works well, but only blow this limit.

We don't see any hard limits in the docs or elsewhere, but would like
suggestions for raising the limit.  Servers are multi-core, 4-8GB RAM
machines running Linux 2.6.x, all very standard and quire powerful.  We
are in the process of breaking up the application to use more PHP and
Memcache farms which will reduce the overall process/connection count,
but we need a short-term solution to this problem.  Our current fix is
to limit the total number of Apache processes to 10-12,000 or so, but
that cuts our overall site capacity more than we'd like (there are many
other processes that run using other memcache & DB clusters).

Thanks.

Steve
------
CTO
Tudou.com - China's Largest Video Sharing Site


>Hi all,

>I'm trying to run memcached but had some trouble with the connection
limit.
>Already found that I need to use the '-c <num>' option to increase the
# of
>maximum simultaneous connections. However, I have 200 webservers each
with
>~64 apache childs running (there may be peaks). If each process would
create
>a persistent connection that would mean that each of my 10 memcached
nodes
>have to handle 12,800 simultaneous connections.
>I guess that shouldn't be a problem, but the low default maximum of
1024
>somewhat bothers me. Is it sane to start memcached with a limit of 40K
>connections, or is it likely that I'll run into other problems then?
>Your input is greatly appreciated.

>Thanks,

>Robin