Threadsafe Python client

[Philip Neustrom]

"Does anyone on the list have experience running the python memcache
client in a threaded environment, or know of / have a patch which
makes it threadsafe?"

I am also using 1.2_tummy5 and I've experienced major problems with
the module when using threads. I'm in Mac OS 10.4.2, python 2.4 and
latest memcached.  Doing something very simple like the following
(note that we are only calling in a thread once):

#!/usr/bin/python
import threading, os, time, memcache

def f():
 global the_obj
 mc = memcache.Client(['127.0.0.1:11211'], debug=1)
 t0 = time.time()
 obj = mc.get("images:cowmilking_2ejpg,front_20page")
 t = time.time() - t0
 if obj:
   print 'got obj in %.4f seconds' % t
 else:
   mc.set("images:cowmilking_2ejpg,front_20page", the_obj)
   print 'no obj, set it in memcache'

def g():
  while 1:
    threading.Thread(target=f).start()
    while threading.activeCount() > 1:
      time.sleep(.1)
    print 'done with set of threads'

m = memcache.Client(['127.0.0.1:11211'], debug=1)
import cPickle
fi = open('img_obj.pickle','r')
the_obj = cPickle.load(fi)

g()

Results in a cycle of successful and fast retrevials:

"got obj in 0.0032 seconds
done with set of threads
got obj in 0.0033 seconds
done with set of threads
got obj in 0.0029 seconds
done with set of threads"

And then it hits a slow one, "got obj in 4.9311 seconds."  The cycle
continues like this (good get()s and then a slow get()).  If we
replace threading.Thread.. with just f() it works fine and gives me
fast gets().  It seems to work so-so using different the_obj's.  I'm
using this particular one
(http://daviswiki.org/cool_files/img_obj.pickle) because it's where I
saw the problem (semi-large binary objects).  If I alter the_obj I get
results like "got obj in 0.1009 seconds" consitently -- no 4 second
hangs, but still generally laggy retrevial.

After looking at memcache.py, it looks like it's hanging on a
socket.recv().  Could this be because the server needs to flush()? 
Why does my problem occur with a single thread but not with just
normal function calls?

--Philip Neustrom