first time user with out of memory question

[Paul T]

--- timeless <time at digg.com> wrote:

> 
> >> Our memcached nodes utilise 1/50% CPU at 400
> hits/sec. At volumes 
> >> around 400 hits/sec, the CPU:workload (C:W) ratio
> is <1. Assuming a 
> >> constant C:W ratio, this means we should utilise
> <<20% CPU at 400k 
> >> hits/sec.
> >
> > By the way, if anyone actually cared to follow
> that paragraph, I 
> > recommend you discard what you might've gleaned
> from it. I've gone 
> > back to check my work and found a faulty
> assumption.
> 
> Here's the revised set of calculations.
> 
> During a short period of 800 get hits/second, we
> used 20,000 CPU 
> microseconds/second[1]. That's 40 get hits per CPU
> millisecond, so we 
> should be able to sustain roughly 40,000 get
> hits/second[2][3][4].

 I bet that you would never get 40K hps on TCP (I
think that UDP would not be a solution either). At
some point the network latency will start taking it's
toll e t.c.

 You're dealing with several layers ( TCP / OS /
Libevent / Memcached (application) ), from some of my
libevent experiments I can already assure you that
starting at some level of throughtput the advantages
of using particular hashing alghoritm (and some other
stuff) in memcached are wiped out.

 800 hps is nothing - on such a load every layer
breeds freely. It is the bottlenecks that kill the
performance. That stuff is totallu non-linear.

 There are many things one can do to udnerstand the
limits of their memcached instance, but they all begin
with the stress test. 

Rgds.Paul.
 
> One thing that does remain true from the aforequoted
> paragraph is that 
> our CPU per Work ratio is <1 (that is, doing double
> the work requires 
> less than double the CPU). Probably the difference
> is just the overhead 
> of running memcached[5]. We can throw that
> consideration out.
> 
> --
> timeless
> 
> [1] Combined user+system
> [2] Assuming ratio of gets to sets doesn't change a
> lot.
> [3] ...and Assuming memcached really is O(1) for
> every operation
> [4] This is a 4-way SMP box, so the assumption that
> Memcached can have a 
> full CPU to itself seems reasonable.
> [5] The CPU usage at 800 get hits/sec is so low it's
> hard to correllate 
> CPU load to work.
> 
> 


__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com