strange performance differences of 1.2.2 vs 1.2.5

[Sean Chittenden]

> I was wondering if anyone here has experience with FreeBSD 6.2? I  
> was compiling 1.2.5 for one of the groups at Yahoo and they saw  
> dramatic performance decreases with 1.2.5 over 1.2.2. Has there been  
> an introduction of code I am forgetting that may cause this?

Summary: For 90% of all cases: "clock_gettime(CLOCK_MONOTONIC, ...)"  
== good.  "gettimeofday(2)" == bad.


I haven't benched 1.2.5, but I'd bet a beer that the culprit is the  
following 1.2.5 change:

        * Use gettimeofday(2) instead of time(2).


For background on why gettimeofday(2) is more expensive, see the  
following thread:

	http://lists.freebsd.org/mailman/htdig/freebsd-current/2005-October/057280.html

or for a brief discussion of this in a white paper format:

	http://dsd.lbl.gov/publications/imc-2003.pdf

Essentially, gettimeofday(2) is supposed to be highly accurate  
according to POSIX, FreeBSD guarantees accuracy and Linux makes no  
such guarantees (hence the performance hit when going from time(2) to  
gettimeofday(2) for BSD and not Linux).  Proper implementation of  
gettimeofday(2) requires synchronization across CPUs, whereas time(2)  
does not.  memcached *should not* use gettimeofday(2) unless highly  
accurate and sub-second resolution is a requirement.  If highly  
precise, but inaccurate time keeping is acceptable,  
clock_gettime(CLOCK_MONOTONIC, ...) should be used instead (using a  
system call that requires CPU synchronization is the opposite of good  
for a performance oriented daemon).

The above thread has all the gory details, but the performance of time  
system calls is an FAQ that should be widely understood at this point.

-sc

-- 
Sean Chittenden
sean at chittenden.org