Testing memcache vs db

[Jason Coene]

Hi David,

Memcached is _VERY_ fast.  On a high traffic server with a varying amount of
data, it can be thousands of times faster than executing real queries to an
RDBMS.  Of course, this all depends on your actual environment.  There are a
lot of people on this list who have immense amounts of varying data with
millions of users, and you'd be hard pressed to find someone that has a
problem with how fast memcached is.  The RDBMS on the other hand...

Anyways, in your test case, you're running a very simple query and with a
relatively small (in terms of data size, not rows) data set, and you're
likely getting a lot of filesystem cache hits.  Bottom line, your test case
data is few and cheap.

Memcached is much more valuable when it's storing many or expensive data.
If you're looking for how many times faster memcached is when compared to an
RDBMS, your answers will range from it being the same speed, to it being
thousands of times faster.  It's all about how it's used.

For me, I can offer these figures:

> 25GB of data in RDBMS, average query taking 200ms.

> 50,000 users at any given time, with 4-10 queries (cached in memcached)
and 2-4 pre-rendered components (cached in memcached) per page.

8GB of memcached capability

CMS that manages to "push" and "pull" from memcached properly.

Letting the CMS pull from memcached instead of the RDBMS isn't just helpful
for speed - it's helpful to keep the DB server from exploding into a
fireball.  If memcached weren't available, we're not talking "slow" - we're
talking "down".

So, you can say that with memcached, a piece of data takes 10ms to fetch.  A
page is delivered in 200ms (or closer to 1sec if there's a few pieces of
not-yet-cached data).

Without memcached, the first 2,000 requests will be fetched in between 1s
and 15s.  The rest won't be fetched at all, as the server will hit gridlock.

Hope that helps,

Jason

-----Original Message-----
From: memcached-bounces at lists.danga.com
[mailto:memcached-bounces at lists.danga.com] On Behalf Of David B
Sent: Friday, June 24, 2005 5:36 PM
To: memcached at lists.danga.com
Subject: Testing memcache vs db

Curious if any of you have some metrics of using memcache with
adb.performance numbers before and after?If not what would be a good way of
testing?
I can imagine a simplistic loop that runs the same query a milliontimes but
the db would cache it itself after the first select * fromcustomer where
customerid = 1And so the db will be faster on subsequent queries.
By the same token if I query where customerid = N and loop through nfrom
1-1million I will not find hits in db or in cache.
As a quick test I wrote a program that queried where cust = 1 for1million
passes.Memcache was 4times faster... I was expecting more like
50-100x...butI suspect it was because of the db caching result (all running
on samebox).
Thank you for any ideas or suggestions or pointers to existing results.