Few queries on atomicity of requests

[Daniel]

Hi

Thanks Dustin...

>       Adding further metadata to the cache and changing the protocol
to  
> return it isn't really an option at this point, but it's easy to add  
> to the data in your application.
> 
Yeah, unless someone really needs it. It seems like this feature could
be added to the current protocol easily enough just by adding an
"expired" flag to the return data, and perhaps a command line switch to
enable memcached to return expired data with a timestamp.


>       Why would you disable caching just because something's
writing?   
> There's always a last write.

Ok, first I better mention what I was referring to with "guaranteed"
current data.

Database programmers are fanatical about making sure the data is
consistent to any reader during a whole transaction. They get into all
sorts permutations with things like read committed, serialized, mvcc
processing etc to handle the issue. The problem is a truly thorny one,
which explains to me why databases can run so darn slowly!

>From my research into concurrent transactions, things just get too
hairy
to try to keep track of all the possible concurrent issues when caching
anything that's being updated in transactions.

If it has been done, is there anyone interested in sharing how they did
it, since the locking/transaction/caching issues can be hard to solve.

Speaking of this, it seems that a better solution would have the
database interface module include caching. It could create a huge
performance boost by integrating memcached functionality into the a
database interface module. Rather than having all database functionality
run on a core set of machines, you could have a database pre-processor
with memcached functionality running on the client machines.  The
pre-processor could handle all of the sql parsing, memcached server, and
communication to the core database machines.

If done right, the database interface wouldn't change a bit, but reads
from the database could occur so much faster.

To do this, the database programmers would have to be convinced that
they could provide "guaranteed" accurate cached data.

I've tried talking with the postgresql programmers, and they are stone
cold to the idea... Perhaps Sun could get a few guys working on it for
Mysql since they seem to now have an interest in both memcached and
mysql, and the technique could do so much better than the current
caching implemented in mysql!!!

>       Most of this is just tired ramblings of someone guessing at  
> requirements, though.  Once there are particular constraints for an  
> application, the mechanism to ensure correctness becomes more clear.

So, just in case my ramblings were not especially clear, has anyone
dealt with the constraints of caching data in the middle of multiple
on-going transactions in a way that is demonstrateably effective?

Is there any reason why Memcached couldn't be baked into a database
interface module?

Thanks

Daniel