Theoretical set assoc cache org performance boosts?

[Brian P Brooks]

Is it realistically possible for a small server project like memcached to ever pose the major bottleneck as the server's processing time rather than connections/networking/protocol?  Is this even possible for a lightweight server?  Could the binary protocol offer such a performance boost where server processing could be a competitor for major profiling bottlenecks?

Of course this is all in curiosity disregarding network speeds, API speeds, etc...

Brian Brooks
http://csel.cs.colorado.edu/~brooksbp/
Cell: (303)319-8663


---- Original message ----
>Date: Thu, 27 Dec 2007 22:18:47 -0800
>From: Steven Grimm <sgrimm at facebook.com>  
>Subject: Re: Theoretical set assoc cache org performance boosts?  
>To: Brian P Brooks <Brian.Brooks at Colorado.EDU>
>Cc: memcached <memcached at lists.danga.com>
>
>I doubt that would make the network round-trips any faster, and  
>network delay is, to be very conservative about it, four or five  
>orders of magnitude greater than the total request processing time  
>inside the server. You could reduce the server's processing time to  
>zero and it would have no measurable effect on response times or  
>throughput from the client's point of view.
>
>Of course, it's open source and you're welcome to experiment; nobody  
>would say no to a significant performance improvement. But I really  
>doubt there's much to be gained there.
>
>-Steve
>
>
>On Dec 27, 2007, at 9:59 PM, Brian P Brooks wrote:
>
>> To my understanding, at the server level, Memcached is implemented  
>> by a fully associative cache -- most likely using a LRU stack for  
>> overwriting comparisons.  Would it be theoretically beneficial if  
>> Memcached were to use a 2 or 4 way set associative cache?  Of course  
>> there would be some changes i.e. would have to statically alloc RAM  
>> so it could partition it's blocks.
>>
>> But, this would definitely help for apps that cache for speed rather  
>> than cache hit reliability.
>>
>> The only way I could see implementing any sort of direct mapped /  
>> set associative cache organization other than specifying the blocks/ 
>> partitions you write to in your application (ie spec'ing out the  
>> direct mapped design in your application).  Although, I could see  
>> how this could give you more control of the cache, and could  
>> probably result in faster caching performance (both reads and  
>> writes), but lower hit rates.
>>
>> Any thoughts?
>>
>> Brian Brooks
>> http://csel.cs.colorado.edu/~brooksbp/
>> Cell: (303)319-8663
>