Memcached implementation inquiry

[Michael Firsikov]

Marcus,

That is a solution, however, I can have potential searches that yield very
large resultset of matching IDs (excess of 1 million) and since memcached
can store only up to 1MB per key, I can not store 1 million 6 digit IDs in
one element and would have to fragment it further. (not that big of a
problem)

My main concern as this point that my Memcached setup is actually optimal.
Could anyone please do a test run on their test box to see how long would it
take to loop thru 500K+ records and get a value?

Thank you in advance,

Michael 


-----Original Message-----
From: Marcus Bointon [mailto:marcus at synchromedia.co.uk] 
Sent: Thursday, April 19, 2007 12:54 PM
To: Michael Firsikov
Cc: memcached at lists.danga.com
Subject: Re: Memcached implementation inquiry

On 19 Apr 2007, at 17:41, Michael Firsikov wrote:

> However, one of the main reasons to explore memcache for us, were the
> searches. (The complexity of searches in MySQL (a myriad of joins,  
> etc)
> resulted in sub-par performance). I am pretty certain it is against
> memcached best practices, but I have done a basic loop to get thru  
> roughly
> 600K records, get and check a value.

I think you have indeed got the wrong end of the stick. Memcached  
doesn't search, but it can store the results of searches that can be  
indexed and retrieved again. So say you do one of your expensive  
searches - and you are still going to have to do it at least once as  
this is a cache, not a DB - hash the SQL that was used to do it to  
make a key, and store the result as a series of IDs of the matching  
records. Then do a batch get for the value ids (which you have  
preloaded). You can store thousands of different searches this way.

Marcus
-- 
Marcus Bointon
Synchromedia Limited: Creators of http://www.smartmessages.net/
UK resellers of info at hand CRM solutions
marcus at synchromedia.co.uk | http://www.synchromedia.co.uk/