key collisions

[mark]

I wrote to the memcache.py author from tummy with pointer to this
thread and this is what he had to say:
"""
On Thu, Feb 15, 2007 at 09:20:00AM -0800, mark wrote:
>It seems the problem is : using connections that persist across script
>runs (nothing wrong with that inherently) and aborting before reading
>an entire response. The next script comes along, sends its request,
>and reads the remainder of the response from the previous script.
>
>Can you please suggest how to fix this problem?

Sounds like you need to remove the connection from the persistent data and
only return it when the process completes.  It's possible that a change to
the memcached Python code could detect this and work around it, but it's
not something I'll have time to develop any time soon.
"""
Does anyone know how to do this on the client side?
thanks
mark

On 2/15/07, Steven Grimm <sgrimm at facebook.com> wrote:
> No clue -- I've never even looked at the Python client. It should be
> possible (I know Python has all the necessary system calls, etc.) but
> maybe someone else knows the specifics of that particular client and can
> give you some suggestion about where to look in the code.
>
> Starting a new connection for each request will cost you an extra
> network round-trip to establish the connection. Whether that's
> significant or not depends on your application -- if, for example,
> you're doing a complex database query or two on most of your pages, then
> the overhead of reconnecting to memcached will be statistical noise. But
> if you're making hundreds of memcached requests and no calls to other
> external resources then it'll be hugely significant.
>
> -Steve
>
>
> mark wrote:
> > Hi Steven,
> >
> > Is there some check that can be added to the memcache client which
> > takes care of this?
> > How to detect if there is some stale data sitting around?
> > It is not a good idea to start a new connection for every request right?
> > How to close an existing connection with memcache.py?
> >
> >
> > Thanks a lot!
> > mark
> >
> > On 2/14/07, Steven Grimm <sgrimm at facebook.com> wrote:
> >> A previous email on this topic had what seems like the most likely
> >> explanation to me: you are using connections that persist across script
> >> runs (nothing wrong with that inherently) and aborting before reading an
> >> entire response. The next script comes along, sends its request, and
> >> reads the remainder of the response from the previous script.
> >>
> >> If you're aborting your scripts during the middle of reading a response
> >> -- and note that it wouldn't necessarily *your* code doing this; it
> >> could be aborted by the Python interpreter if you run out of memory, for
> >> example -- then you need to close the memcached connections to guarantee
> >> you don't have any stale data sitting around.
> >>
> >> -Steve
> >>
> >>
> >> mark wrote:
> >> > Hi Dave,
> >> > I have not made much progress! I had the same problem of not being
> >> > able to reproduce in the test environment! But in production I
> >> > consistently get the problem!
> >> > So I moved to cmemcached http://gijsbert.org/cmemcache/index.html
> >> > which is a python wrapper for the libmemcache. It seemed to solve the
> >> > problem at the beginning, but what was happening was if there was a
> >> > collision the app died with this message:
> >> >
> >> > [FATAL at 1170222010.740685] mcm_fetch_cmd():1176: memcache(4) protocol
> >> > error: server sent data for key not in request
> >> >
> >> >
> >> > I have not fixed this problem yet! It seems to be the same problem as
> >> > before but the client here checking for consistency I think.
> >> >
> >> > It will be great if someone can find a fix for this!
> >> > thanks
> >> > mark
> >> >
> >> > On 2/14/07, Dave St.Germain <dcs at tabblo.com> wrote:
> >> >> hi, mark,
> >> >> I'm having the same problem with key collisions with the python
> >> >> binding, and I was wondering if you've made any progress.  I haven't
> >> >> been able to reproduce the problem in a test, but in production,
> >> >> we'll encounter the problem all of a sudden after a while (the only
> >> >> "solution" so far is to restart memcached).  The server returns the
> >> >> wrong value for the requested key.  Not only that, but if you try a
> >> >> get_many for key 'foo', you may get a response that has key 'bar' and
> >> >> its value.
> >> >> Since I haven't been able to reproduce this behavior in a test, I
> >> >> don't yet know if it's the python binding or memcached itself.
> >> >> Does anyone have any ideas?
> >> >>
> >> >> Dave
> >> >>
> >> >>
> >> >>
> >> >> > Hi
> >> >> > I am consistently getting key collisions. There are total of around
> >> >> > 2000 key
> >> >> > value pairs cached.
> >> >> > This is how I prepare the key for memcached: md5.new(key).hexdigest
> >> >> > () where
> >> >> > key is the unique
> >> >> > id for the database row that I want to cache.
> >> >> > When I do a 'get' for a specific key, I get a value of a different
> >> >> > key.
> >> >> > I am using python-memcached as the client.
> >> >> > This is causing lot of problems. If anyone has faced similar issues
> >> >> > or has
> >> >> > any suggestions do reply.
> >> >> > cheers
> >> >> > mark
> >> >>
> >> >>
> >>
> >>
>
>