libmemcache(3) 1.1.0rc4...
Sean Chittenden
sean at chittenden.org
Wed Dec 22 11:21:03 PST 2004
> Check out these stats for 5pm last night to 9am this morning:
>
> stats
> STAT pid 8269
> STAT uptime 57909
> STAT time 1103735953
> STAT version 1.1.11
> STAT rusage_user 3596:627229
> STAT rusage_system 8962:163544
> STAT curr_items 420162
> STAT total_items 68031447
> STAT bytes 91619292
> STAT curr_connections 1
> STAT total_connections 584
> STAT connection_structures 7
> STAT cmd_get 136062198
> STAT cmd_set 68031447
> STAT get_hits 135641850
> STAT get_misses 420348
> STAT bytes_read 16627870580
> STAT bytes_written 38062445503
> STAT limit_maxbytes 268435456
> END
Wholly crap! That's a 99.7% hit rate... you've got some damn cachable
data! What are your stats like during peak hours? Oh, and fwiw,
you're doing 3524 operations per second, not 2300: (get+set) / uptime.
:)
(136062198 + 68031447) / 57909 = 3524.385588
>> 1) identify data needs
>> 2) acquire data
>> 3) act on data
>> 4) display data
>>
>> Where in 1), you setup a bunch of callbacks. In 2), you do the actual
>> mc_get() and your callbacks are executed (which get the data from the
>> database in the event of a cache miss). And by 3) and 4),
>> libmemcache(3) is probably out of the picture. But, for step 1 and 2,
>> you may have removed a bunch of independent get requests and
>> consolidated them all into one get request.
>
> I'm going to look into this a bit today, right now I just to single
> gets. But I believe it is actually very easy for me to change my code
> to perform multi-gets on its own so the callbacks may not be needed in
> my situation.
Ah, ok... well, I was grouping all of my get's together into a single
get, then was parsing out the data independently... which was a huge
waste of CPU cycles and tedious to program. Using the callbacks I
saved a few hundred lines of code and managed to pick up a bit of a
boost for performance.
>>> [snip]
>
> All of this sounds great, but what I had in mind for now is just a
> simple mcm_reactivate_servers function or such that could be manually
> called and would go through the list of deactivated servers and attempt
> ro revive them.
Ah, yeah... that'd be easy to do too. I thought I had an AM flight
today, turns out it's a PM flight, so I'm grounded and here at my colo
banging out code... I'll add two functions: one that does a global
reactivate, the other that does a per server reactivate.
> My application isn't handling user requests, its a
> background process that never quits. As such I want to periodically
> say
> once every 5 minutes or something try to restore any dead servers, as
> we
> may sometimes need to take one down for maintenance or hardware may
> fail. If this happens we do not want to have to restart the daemon
> that
> is utilizing libmemcache to get it to see the server after it comes
> back.
Yup, I appreciate that.
> I know this presents problems with the location of data in the cache,
> but the way I see it the worst thing that happens is I get a miss when
> really it was cached on the other server, and then I restore the cache
> and in the future get hits. Some memory is wasted, but the extra copy
> will just never get accessed and as such rotate out of the LRU cache
> anyway. Theres no concern about stale data due to expiration times
> being
> properly set. Am I missing some major issue?
Not really... when a server goes down, your whole cache essentially
gets invalidated (well, not your whole cache... if you have 3 machines,
and one goes down, you have a 50% chance of a cache hit... if you have
10 machines, one goes down, you have a 11% chance of a hit, etc). If
you have enough load and enough memcache machines, if one goes down,
your database(s) could see a nice spike in load until the cache reaches
saturation again.... until the next server list change. That
fluctuation could be trivial or could be disastrous. You've been
warned. ;~)
> I'm probably going to add this either today (or late next week, as I
> have some time off for xmas). I think this would likely be useful to
> others as well until the full solution you outlined above is
> implemented.
Yup... I'll see if I can knock out a 1.1.1 release before 3pm PST
that'd include this feature. I'm in the midst of sucking in the
relevant parts of sys/queue.h into memcache.h that way I can remove
this external dependency which has been an issue for folks at
Amazon.com on various leenox distro's. *shrug* BSD++ *grin*
-sc
--
Sean Chittenden
More information about the memcached
mailing list