Why not let the fs handle this ??

[Constantin B]

Hi,

I have been working on high perf optimisations on a huge adult website (yes
this need power too),
well when we let the funny thing on the side the "girls" get around
800.000unique visitors by day
and around 5.000.000 view pages. So they get quite busy with mysql ...

I naturaly turned to memcache but then tested another simple solution , i
just maid a disk based
cache that do the same thing as memcache :

store(object)  write serialized object to disk
retrive(object)  retrive serialized object from disk

so
select * from a where b<now() - interval 30 min for example

becomes
$cache_key = '';
$cache_time = 30*60;

($age,$object) = retrive(object);
if($age > $cache_time){
  object = select * from a where b<now() - interval 30 min;
  store(object);
}

so this was running on all webservers and gently writing stuff to the disk .
unloading the database and speeding the girls up .
this resulted in around 5000 small files on the disk (if you count all
possible keys combinations)

this is the setup , now the remark .

They use linux + xfs there ...
and when i went to the webservers i realized after running the dstat that
they was not
reading anything on the disk ... all the small files was naturaly cached by
the os fs buffers in
a nice way . (i added a cron to clean up unused cached files rm where age >
5H) and thats all.

On the perfside it resulted to be all in ram so it was fast , it was trivial
to setup (compared to memcache)
and if a server reboot the cache still on disk and the os was dealing with
all the issues  no need  in any deamon
or whatever , so its realy trivial and robust .

The purpose of this email is to ask what are the future limitations of my
setup ? I mean for the moment it do the
job , but where is the gotcha ?

thanks
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.danga.com/pipermail/memcached/attachments/20060606/abfbeebe/attachment.html