Confused on scaling memcached & persistant connections (w/ the PECL client)

[Adrian Ziemkowski]

We're looking at a similar issue.  We don't use persistent connections
because of this.  We tried it and saw a significant speed up, right up
until the hard limit.  But then we died as we hit the ceiling.

I see that LiveJournal has 50+ servers and only a few memcached servers.
How is that being done efficiently?   I'm guessing by connection pooling
on each web server to avoid having so many idle persistent connections
(like we have with the PECL client)?  

Are there any plans for the PECL memcached client to pool the persistent
connections on the server in a more efficient manner?



Also, has anybody else noticed just how slow unserialize() is in PHP?
We spend more time unserializing complex objects we get from memcached
than getting them.  Using var_export() only cuts that time by 20%.  Is
there any way for the memcached PECL client to be extended to accept
arrays/objects and do this faster or is this just a limitation we face
with PHP?  Tested with and without ionCube accelerator.

Thanks,
 - Adrian Ziemkowski


-----Original Message-----
From: Brad Fitzpatrick [mailto:brad at danga.com]
Sent: Thursday, February 02, 2006 8:56 PM
To: Mike Lynn
Cc: memcached at lists.danga.com
Subject: Re: Confused on scaling memcached & persistant connections

Connections are really cheap nowadays.  If you're seeing a slowdown
after $n connections, you're not using epoll or kqueue it sounds like.
Are you using a modern Linux/FreeBSD?

- Brad


On Thu, 2 Feb 2006, Mike Lynn wrote:

> maybe somone can help me out - but I'm confused on scaling memcached.
> Right now I've got 7 web servers each with memcached on them.  All
webs
> are running apache/php/memcached.   with persistant connections to
> memcached.  Each memcached instance avg 2000 connections on avg - how
do
> i scale that?   Memcached is set at 10k connections max - but is seems
> to slow down after 3k connections.  Because the php client lib uses 
> persistant connections, which is good, it means that each new web
server
> addes 500 persistant connections to each memcached server - so I've
hit
> the max???  I don't see a really (good way) to do a truly global
cache.
> Perhaps a UDP based memcached....?  Thanks for any help in advance..
> mike
>
>