first time user with out of memory question

[Paul T]

19K hps certainly is "heavy pressure". In our current
setup we are not (yet) getting close to those numbers
with memcached. But then again - in our setup
memcached (latest stock memcached, built and started
with defaults) got crazy after 3 weeks. 

However. 

19K hps of reads *does*not* subsitute for a
stress-test consisting of 10K hps reads mixed with 5K
hps of writes (for example). In async application
under pressure just changing pconnect to connect can
make a (critical) difference - changing the memory
access patterns. 

Basically that was the only point I was trying to make
and it seems that we agreed on it several emails ago,
so I am sorry for being redundant.

Rgds.Paul.

BTW - the note that you made about USE_SYSTEM_ALLOC is
not 100% correct, because - from the code - the
USE_SYSTEM_ALLOC only affects part of the slabs system
(the 'heap' part, which is important part but it is
still - a part). 

It is not actually that easy to entirely eliminate
slabs from memcached codebase. 

Unfortunately, digging deeper will open another can of
worms. Linux is open source too - it envolves. And
frarmentation issues are with us since the days of
RT-11, so there is a good chance that memcached is
already fighting with the ghosts - in some setups. 

Thanks for the numbers.
 
--- Steven Grimm <sgrimm at facebook.com> wrote:

> Paul T wrote:
> > I think that nobody on this list is running
> memcached
> > under heavy pressure, though, so this bug can be 
> > happily ignored in many cases.
> >   
> 
> As I type this, we are running about 19,000 requests
> a second through 
> some of our production memcached instances.
> 
> There is a reason we spent time reworking
> memcached's network I/O to 
> reduce its CPU consumption: we were getting
> uncomfortably close to 
> running out of CPU capacity on our memcached servers
> (and these are not 
> slow old machines). There's also a reason we added
> UDP support: our 
> memcached instances were each handling tens of
> thousands of simultaneous 
> client connections, enough that the per-connection
> overhead was starting 
> to become significant. If that doesn't count as
> "heavy pressure" I'm not 
> sure what does -- what does your load look like?
> (And what hardware do 
> you run memcached on, that can support significantly
> heavier traffic 
> than that?)
> 
> -Steve
> 


__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com