Whats memcached all about?

[Lexington Luthor]

Malcolm Frazier wrote:
> I recently just heard about memcached and have a few questions;
> 
> 1. How much memory will memcached "really" use?
How much you tell it to use on the command line (-m flag) + the sizes of 
the largest concurrent commands it receives (e.g. If you have n clients 
each sending it m byte long commands line at the same time, add nm bytes).

> 2. What exactly does memcached cache? (What types of objects)
Arbitrary. It does not care. You give it a chunk of data and a name for 
it (and optionally a time at which to discard it), and later request it 
again. Aside from some support for incrementing and decrementing 
integers, memcached does not at all care what you give it, it will take 
it and happily give it back (unless it expires or is deleted to make 
room for other items, see the -M flag).

> 3. Does memcached cache objects per user?
It has a single key namespace. You give an "object" a key and retrieve 
it by that key. How you designate a key is entirely up to you. I assume 
the PHP client interface handles the hashing and server selection 
internally.

> 4. If I have a high volume of users hitting my sites with PHP code is 
> memcached going to eat up all my memory?
See 1.

If you expect a very high volume of activity, build your kernels (I 
assume Linux) with a more conservative overcommit algorithm (or simply 
disable overcommit entirely), and run memcache with the -k flag (to 
force it to mlock). If you are using a database server like PostgreSQL 
then do not run memcache or anything memory intensive on that machine at 
all, to make as much room as possible for the page cache.

> 5. Once I implement memcached will I see a a big jump in performance?
I use C with a custom memcache client, so I can't say how fast PHP will 
be, but memcached handles about 4000 keys per second on my modest 
hardware (I benchmark gets only, bear in mind there are thousands of 
sets and deletes taking place too). It is entirely network I/O bound, 
CPU usage is rarely above 20% even on the slowest machines. For best 
performance use a pool of permanently connected sockets for get requests 
and have a background thread (using a separate socket) handle the set 
requests. In my case the deletes are automatically issued by triggers on 
the database server, but its easy to handle these.

Using a PostgreSQL backend with the pgmemcache library to delete keys 
automatically when a SQL transaction affects them (to maintain 
transactional integrity), I saw a just from 800 transactions per minute 
to over 3000 transactions per minute with aggressive caching with 
memcached (as well as some application local caching).

> Sorry I have so many questions, I am also very interested in hearing 
> failed and success cases.
> Thank you to anyone who can help.
memcached is a brilliant way to reduce the load on your database server, 
but it is no silver bullet. You have to implement caching application 
wide and its not easy to maintain such an app without a highly 
customized data mapping layer (which is a lot of work, especially if you 
want to be database agnostic).

Regards,
LL