Altering queries?

[Clint Webb]

Theoretically, yes.   But you have to figure out the point where its no
longer worth it to store things in cache.  If it happens only once every now
and then, and its really not that expensive of a database operation, then
why bother caching it?

In my reality, a request for a list of entries higher than a few hundred
would be something that doesnt happen very often, I actually have a limit of
5,000 coded into the routine.  Anything over that, comes straight from the
database.  But since that has never actually happened, I don't know what is
best.  Considering that people only see 15 or 10 entries at a time, and the
fact that the DESIGN of the system causes the most relevant entries to be at
the top of the list it would take people a very long time to get past 10,000
entries.

Assuming that it takes them 5 seconds to click thru each set, it would take
over 7 hours of constant clicking to get past 5000 entries.

If my design was different and people DID browse thru older "pages" of
entries, then I might use a sliding window instead.  So instead of storing
the entire list each time, I would only store the subset.  For example,
store from 0-100, 101-200, 201-300, etc.

In giving my answers to you, I was trying to be generic... my only
recommendation is to try different scenarios and see what works best.  Do
some simple benchmarking.   It is ok to go straight to the database
sometimes.  Keep in mind that I do the lists this way because I've done a
bit of benchmarking, and looking at the design of the system and how it is
used, I see that its the most efficient.  A different way might be better
for you.

On 9/28/07, K J <sanbat at gmail.com> wrote:
>
>
>  Well, not really arrays in memory.  Lists in memcache.   It works well
> > because 99% of requests will always be within the 100.  That is why it
> > exists as apposed to just having a 500,1000,1500, etc.
> >
> > But lets say a user has browsed up to the 800th entry.  When getting the
> > key from memcache, it only gets the one (this one is called
> > "Articles:top:limit=1000").  Which contains all the ID's of the current
> > articles in the order that they would be displayed.  I skip the first 800
> > entries in that list, and display the next 15.  The actual contents of the
> > article itself is in another cache entry ("Article:aid=800" for example).
> > Depending on how I was accessing this information, I would produce either
> > xml or html and would generate a key called "xml:Article-list:800-814" or
> > "html:Article-list:800-814").
>
>
> Sorry for digging up this old topic, but I'm still not quite understanding
> how you are storing the Memcache lists.  You said you are storing the first
> 100 in one list, and the first 500 in another.  If someone were to request
> the 20,000-20,100 entries, what exactly do you do? Do you create a third
> list that has records 1-20,100?  Sorry if I'm a little slow.
>
>
>



-- 
"Be excellent to each other"
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.danga.com/pipermail/memcached/attachments/20070928/7459fc77/attachment.html