what guarantees can i expect from write ops?

[Paul G]

----- Original Message ----- 
From: "Richard 'toast' Russo" <russor at msoe.edu>
To: <memcached at lists.danga.com>
Sent: Sunday, October 02, 2005 1:53 AM
Subject: Re: what guarantees can i expect from write ops?


> Paul,
>
> That's a pretty small expiretime (1 second).  If I recall correctly,
> memcached uses a unixtime for expiration, so it's possible your set
> comes before the second jumps and the get after.

yep, this looks like the most likely explanation (it does use time()).
ideally we'd be using gettimeofday(), but for my planned use cases the ttl
will never be this low so i have little incentive to produce a patch (should
be easy though).

> Also, if your server is under (really) heavy use during the testing,
> it's possible your cache retention is really bad.

these are unit tests run on a dev server, so the conditions are controlled
(short: no).

> In the memcached is not a database, consistency is not assured camp,
> your expectation of getting the same data that you set back is pretty
> reasonable.

my expectation, precisely worded, is "i expect any subsequent get to return
the data supplied in a previous set if: a) the set succeeds and b) the key
hasn't expired." it appears that the b condition was failing and the wrong
expectation was about the precision of the expiration time rather than
consistency. imo, this should be either documented or fixed, since the
current implementation is obscenely imprecise wrt short expiration times
(approaching 0% precision for 1 sec ttl). of course, it was silly on my part
to confine troubleshooting to the verification of condition a ;]

> Doesn't work if some other process is writes to the same
> key, the key falls out of the cache, or if your client changes where
> it wants to hash that key between the write and the read.

.. of course. these concerns are being addressed locally by a) locking
implemented with incr/decr, b) oversizing memcached and c) controlling the
hashing process/not using multiple servers.

> Offloading writes doesn't seem like a safe use of memcached, though.
> Would be interested in the reasoning behind that.

i didn't elaborate on this sufficiently, since my use case is orthogonal to
my original question. it's only unsafe if the data needs to be persistent;
this requirement may not be present for all sorts of transient data such as
sessions, access statistics etc. my first use case is utilizing memcached as
a storage backend for session data (it is acceptable to lose it in case
memcached/box running memcached crashes or is restarted). if memcached holds
up given my current client implementation, i'll use it to store access
statistics data to generate caching candidacy reports; it is acceptable to
lose this on crash/restart as well. the last case i'm contemplating is using
memcached to hold sequence values required for a new db connection load
balancing algo (weighted round robin by connection class) to replace the
extremely simplistic (to the point of being harmful at times)
random+weighted one i inherited; this can be made not to care about loss (as
long as it's consistent) either.

as you can see, i'm using memcached as (lightweight) distributed shared
memory where my fault tolerance requirements are low enough to make using
something more lightweight than traditional implementations an advantageous
option. although i'm sure we're in the minority, i expect that there is a
number of memcached users who use memcached in a similar manner, as opposed
to the way god/danga intended ;)

-p