Greg Whalin gwhalin at
Tue Nov 30 11:15:42 PST 2004

Evan Martin wrote:
> On Tue, Nov 30, 2004 at 11:17:10AM -0500, Greg Whalin wrote:
>>Gregory Block wrote:
>>>The only, the *only* thing I can think of to wish for is a better 
>>>performing GzipOutputStream, and that's mostly because I haven't gone 
>>>looking for either a faster compression stream in Java, or something 
>>>backed by hardware to do that compression, and that's just a 
>>>nice-to-have rather than a killer.
>>This is the one thing I would like as well!  I played around with the 
>>basic compression streams in java and the performance was horrible.  Our 
>>solution was probably not the most elegant.  As the client supports a 
>>threshold object size, below which, no compression is attempted, we just 
>>set this value fairly high (128K) and store only our largest object 
>>compressed.   Given a typical memcached server is fairly cheap, we just 
>>bought more hardware to run as servers.
> Depending on your data, an alternative approach to squeezing more memory
> out of your memcached is using a tighter serialization format.  Not only
> does generic serialization have some overhead:
>   % perl -mStorable -e 'print Storable::freeze(\5)' | wc -c
>        14
>   % perl -mStorable -e 'print Storable::freeze([1,2])' | wc -c
>        22
> But also if you know the details of your data representation, you can
> use (for example) a single byte for small numbers.
> See, uh, get_userpic_info in

True. Though the default java serialization for most native object types 
is decent.  Seems only when you start serializing complex data 
structures that things get inefficient.  But yes, I agree that custom 
serialization can shrink things quite a bit.


More information about the memcached mailing list