memcached debate

[andrew at flypublishing.com]

thanks everyone for your input, I think I will try and write something
to carefully measure the bandwidth used like Dustin said...

My boss will say though that until we release this to production
you'll never know if the network will handle it.  Says this is
basically impossible to test throughly with real world conditions.

More about the application...

I'm representing data that's different depending on from who's
perspective you are viewing it.  So I want to load all the actual
data, integers that point to other integers into memcached but then
for each request that comes in, look through several thousand of those
keys and assemble a response custom for the viewer.

I don't want to cache anything specific for a viewer because if any of
the real data changes, it invalidates what was previously generated
for him.

But this means TONS of key hits for each and every request.  The data
has to be accross many machines, each request could use a key from
anywhere in the set.  The list of integers each key holds are other
key values.  So when you get that list you then hit all those keys and
on and on and on until some stopping point.

My boss is saying cache the response for each viewer.  I'm saying,
let's just cache the actual data for everything and generate the
response for a viewer on the fly.  Since the data IS cached, in
memcached, won't each response be super fast?  He says since I'm
treating memcached like internal memory and hitting it way too much it
won't work...


On 4/19/07, Ask Bjørn Hansen <ask at develooper.com> wrote:
>
> > sure.  not ever gonna be as fast as normal mem... but my boss is
> > saying i'm crazy to try and access it super often.
>
> You didn't tell nearly enough about your application and your
> algorithms for us to say if he's right.
>
> As Paul said, "it depends".
>
>
>
>   - ask
>
> --
> http://develooper.com/ - http://askask.com/
>
>
>