memcached debate

[Paul Querna]

Perrin Harkins wrote:
> On 4/19/07, andrew at flypublishing.com <andrew at flypublishing.com> wrote:
>> sure.  not ever gonna be as fast as normal mem... but my boss is
>> saying i'm crazy to try and access it super often.  says i'll flood
>> the network.
> 
> I guess the question is whether or not you need access to the data
> from multiple machines.  If you don't, a local storage system like
> BerkeleyDB is going to be faster and won't affect your network.  If
> you do, it will be hard to find anything significantly faster than
> memcached.

Actually, BerekleyDB won't always be faster.  It depends on your data 
size.  If your BDB databases are say, 30 gigs, and you only have 8 gigs 
of ram in a machine, even a 64 bit machine, it still going to seek a ton 
to find the data, and we have found in some uses cases the memcached can 
be significantly faster than relying upon BerkelyDB.

Whereas it is relatively trivial to get >30 gigs of memcached space, 
which means you could fit all of the data in RAM, which even across the 
network, can easily be faster than heavy disk seeking.

Of course, it all depends on your machines, datasets, and how you are 
using the data.

-Paul