key collisions

[Steven Grimm]

No clue -- I've never even looked at the Python client. It should be 
possible (I know Python has all the necessary system calls, etc.) but 
maybe someone else knows the specifics of that particular client and can 
give you some suggestion about where to look in the code.

Starting a new connection for each request will cost you an extra 
network round-trip to establish the connection. Whether that's 
significant or not depends on your application -- if, for example, 
you're doing a complex database query or two on most of your pages, then 
the overhead of reconnecting to memcached will be statistical noise. But 
if you're making hundreds of memcached requests and no calls to other 
external resources then it'll be hugely significant.

-Steve


mark wrote:
> Hi Steven,
>
> Is there some check that can be added to the memcache client which
> takes care of this?
> How to detect if there is some stale data sitting around?
> It is not a good idea to start a new connection for every request right?
> How to close an existing connection with memcache.py?
>
>
> Thanks a lot!
> mark
>
> On 2/14/07, Steven Grimm <sgrimm at facebook.com> wrote:
>> A previous email on this topic had what seems like the most likely
>> explanation to me: you are using connections that persist across script
>> runs (nothing wrong with that inherently) and aborting before reading an
>> entire response. The next script comes along, sends its request, and
>> reads the remainder of the response from the previous script.
>>
>> If you're aborting your scripts during the middle of reading a response
>> -- and note that it wouldn't necessarily *your* code doing this; it
>> could be aborted by the Python interpreter if you run out of memory, for
>> example -- then you need to close the memcached connections to guarantee
>> you don't have any stale data sitting around.
>>
>> -Steve
>>
>>
>> mark wrote:
>> > Hi Dave,
>> > I have not made much progress! I had the same problem of not being
>> > able to reproduce in the test environment! But in production I
>> > consistently get the problem!
>> > So I moved to cmemcached http://gijsbert.org/cmemcache/index.html
>> > which is a python wrapper for the libmemcache. It seemed to solve the
>> > problem at the beginning, but what was happening was if there was a
>> > collision the app died with this message:
>> >
>> > [FATAL at 1170222010.740685] mcm_fetch_cmd():1176: memcache(4) protocol
>> > error: server sent data for key not in request
>> >
>> >
>> > I have not fixed this problem yet! It seems to be the same problem as
>> > before but the client here checking for consistency I think.
>> >
>> > It will be great if someone can find a fix for this!
>> > thanks
>> > mark
>> >
>> > On 2/14/07, Dave St.Germain <dcs at tabblo.com> wrote:
>> >> hi, mark,
>> >> I'm having the same problem with key collisions with the python
>> >> binding, and I was wondering if you've made any progress.  I haven't
>> >> been able to reproduce the problem in a test, but in production,
>> >> we'll encounter the problem all of a sudden after a while (the only
>> >> "solution" so far is to restart memcached).  The server returns the
>> >> wrong value for the requested key.  Not only that, but if you try a
>> >> get_many for key 'foo', you may get a response that has key 'bar' and
>> >> its value.
>> >> Since I haven't been able to reproduce this behavior in a test, I
>> >> don't yet know if it's the python binding or memcached itself.
>> >> Does anyone have any ideas?
>> >>
>> >> Dave
>> >>
>> >>
>> >>
>> >> > Hi
>> >> > I am consistently getting key collisions. There are total of around
>> >> > 2000 key
>> >> > value pairs cached.
>> >> > This is how I prepare the key for memcached: md5.new(key).hexdigest
>> >> > () where
>> >> > key is the unique
>> >> > id for the database row that I want to cache.
>> >> > When I do a 'get' for a specific key, I get a value of a different
>> >> > key.
>> >> > I am using python-memcached as the client.
>> >> > This is causing lot of problems. If anyone has faced similar issues
>> >> > or has
>> >> > any suggestions do reply.
>> >> > cheers
>> >> > mark
>> >>
>> >>
>>
>>