brian at tangent.org
Thu Feb 14 02:19:17 UTC 2008
On Feb 14, 2008, at 5:05 AM, Dustin Sallings wrote:
> It's come up on the list a few times. Some have not been able to
> imagine dealing without keys, but it's really as simple as putting
> your keys in an array, and reserving the opaque space from the
> beginning to one past the end and processing values on the way out,
> looking up the key by response opaque - first opaque.
The array is going to cost memory allocation though, and then a lookup
to get my hands on it (which will be a scan on an array).
Let me show this below.
> It's especially a good thing with UDP (which has no
> implementation). You can pack more responses in fewer packets.
UDP to met is best use for tiny things... once you go above a single
packet, you might as well be using TCP.
>> So, presently, on a high level there is already an opaque mapping
>> of values to what the client wants. This is literally the key=value
>> mapping. The relation of key values to the application happens
>> outside of the library, so it's not a bother. If the lib wants to
>> hide the opaque=key mappingit has to do a lot of internal tracking,
>> which is slow and awkward in C.
> I'm thinking something near this (in some kind of weird cthon
> char * key_map;
> int start_opaque, i;
> start_opaque = gen_opaque();
> for key in keys:
This will require memory allocation per connection (aka in mgets() for
each key that hits a server).
You will have to take a shot in the dark for array size, and then
realloc() for growth in the array.
The memory allocations will kill any performance gain. Plus, for
embedding, its just not going to be that friendly to small client
> for res in responses:
> char *key = key_map[res.opaque - start_opaque];
Unless something changes, you won't know if all keys were retrieved or
not (aka you would have to fill up the network with NOT_FOUND packets).
This assumes order as well (so you can't optimize the internals of the
server to just send keys as quickly as they are processed). You create
implicit ORDER BY (yes... mentioning SQL). If you don't order... well
then you will need a hash or a double array. More memory, and if array
an array scan.
Array states, or hashes, are expensive.
Unless someone else has a way to avoid he ordering, and the state
usage... this is going to turn out to be a lot slower then the text
protocol in the overall architecture.
Brian "Krow" Aker, brian at tangent.org
http://krow.net/ <-- Me
http://tangent.org/ <-- Software
http://exploitseattle.com/ <-- Fun
You can't grep a dead tree.
More information about the memcached