Strange memcached behavior on stress test

[Garth Webb]

On Tue, 2006-06-13 at 09:39 +0200, Janning Vygen wrote:
> Am Montag, 12. Juni 2006 18:17 schrieb Brad Fitzpatrick:
> > On Mon, 12 Jun 2006, Janning Vygen wrote:
> > > > (though that depends on your usage pattern) but this is one condition
> > > > where you'll get a set problem and an out of memory error even if you
> > > > have space left in individual slabs.
> > >
> > > Thank you garth for this detailed explanation. Now i think i understood
> > > my problem. My session data is very different in size. So my allocated
> > > "slabs" are "full" and a new slab can't be allocated.
> >
> > Janning,
> >
> > This problem is fixed in two different ways in later releases.  Try out
> > this version for now:
> >
> > http://www.danga.com/memcached/dist/memcached-1.1.13-pre.tar.gz
> >
> > It pre-allocates a 1MB slab of each class on start, so people won't run
> > into this confusion.
> 
> Ok, i would run a newer version of memcached IF memcached is the right thing 
> for (PHP) session management. I am not sure about it anymore. Some people on 
> the list told me it is not the right thing because it saves it's session in 
> memory. But i dont think that this is a problem. But i think session 
> management with memcached can run into problems if you have a peak with lots 
> of pages as the oldest session data is moved out and you do not have a 
> garanteed session time anymore.
> 
> If i have a memcached instance with 2GB memory and my average session data is 
> 8 KByte (slab size) i can save 262144 items. As the oldest value is pushed 
> out, i will never have a guranteed session expiration time (in terms as you 
> are still logged in after 20 minutes of inactivity). A memory is quite 
> expensive it maybe does not make sense to use memcached as a session storage.

Exactly!  Memcached is *not* storage.  Memcached is meant to make your
*existing* storage solution (disk/mysql/etc) faster.  Let me outline a
situation.  It is very unlikely that all of your users will be logged in
at the same time.  Lets say you have 300,000 users.  I don't know what
your application is, but on most web apps, only a small percentage of
users are logged in at any given time.  Over any 20 minute period, maybe
there are 2000 users actively logged in.  You said that session is about
8K, this means that *you only need a memcached size of 16MB*.

Your session data can be stored in MySQL or SQL-lite or flatfiles or
whatever, it doesn't matter.  Why?  Because you are only going to access
that data *once* and load it into memcached.  At that point you will
access it from memcached.  The cost of loading that data from slow
storage will be unnoticeable because it only happens the first time it
is requested.  If the user continues to stay online, their data will
continue to stay in memcached because memcached keeps active data in the
cached.  If they leave and do something else for a while their data will
eventually be pushed out by other content.

> An i am now convinced that sharedance might do it much better, because it 
> uses disks to store session data.

Sharedance does not simply store data to disk.  It caches data in memory
just like memcached and THEN uses a file on disk for permanent storage.
Memcached will behave *exactly* like sharedance by adding the behavior I
mentioned above, ie:

1) Look for data in the cache
2) If its in the cache return it, if its not goto 3
3) Look on storage (disk, mysql, etc)
4) Write the data found to the cache
5) Return the data

With this algorithm, data is only loaded from slow storage once and then
kept in cache while its in use.  If the cache goes down, or data gets
pushed out it is transparent to the user because the system will always
fall back to the slow storage.  In the case of total cache failure, the
site maybe slower, but it will still continue to function.

Memcached is a very fast caching solution.  It can be used for steps 1
and 2 of the above process.  You must write code to work with your
permanent storage in steps 3, 4 and 5.  Its advantage is that it
integrates with your existing storage system.

Sharedance is a caching solution that ALSO implements a permanent
storage solution.  It does steps 1-5.  Its advantage is that you don't
have to choose a storage system.

So it sounds like you want sharedance because you don't want to do any
extra work to and you do not currently store data anywhere else (mysql,
disk, etc).

Garth

> Maybe i should take a look at sharedance which was always on my option list. 
> But memached rocks at the moment (beside the memory problem i encountered 
> once). And i feel that memcached is more used and better supported. 
> 
> kind regards,
> janning
>