file side memcached

[Andreas Vierengel]

Cahill, Earl wrote:
> It might sound like heresy, and I know about tugela cache
> 
>  
> 
> http://meta.wikimedia.org/wiki/Tugela_Cache
> 
>  
> 
> but I am wondering if anyone else would be interested in a file side
> cache that mimics the memcached spec.  Mostly I am thinking that it
> could work very well to be in front of Amazon's S3 storage service.
> 
>  
> 
> http://www.amazon.com/gp/browse.html/104-9358418-3750302?node=16427261
> 
>  
> 
> The problem for me is file size.  Maybe it wouldn't be an issue, but
> tugela uses Berkeley db and I am thinking that maybe that wouldn't work
> so hot for say photo storage.  
> 
>  
> 
> For me, having a nice terabyte box or two in front of S3 I think would
> work rather nicely.
> 
>  
> 
> Thanks,
> 
> Earl
> 
> 

I don't know all guts in Berkeley-DB, but if Berkeley-DB does not 
provide the concept of nonblocking-access to key/values, you must use 
threads to achieve this or a disk-bound query will stop all other 
"concurrent" requests. memcached as is uses async-IO with epoll, so you 
have to add some sort od a thread-pool to achieve nonblocking concurrency.
In my company we had/have plans to program/test a prototype in which we 
combine epoll with a thread-pool and berkeley-db as backend (BTREE or 
HASH), but currently we have no time to get in this more :(

--Andy