freeBSD 5.3 and memcached

[Ted Schundler]

Interesting. It *might* be a FreeBSD bug as it seems to be tied to the
interface's MTU. A packet will be at most 1500 bytes before it has to
be split up on the ethernet frame level. It seems the 2nd packet isn't
being sent immediately - maybe a weird side effect of some packet
scheduling code.

More interesting - it coredumps at 1407-1409 byte packages and also at
2855-2856. Which are basically at the border of the size ranges where
there is a speed issue. And at similar values when decreasing the
NIC's mtu.
But it doesn't coredump in any socket function - it coredumps when
copying the incomming content...

Ted

On Sun, 27 Mar 2005 14:41:05 +0400, Al Ermolaew <alermo at bk.ru> wrote:
> Hi All!
> 
> At once I shall apologize for my English :)
> 
> I had very strange problem with work of the memcached on
> freeBSD. I have two servers with freeBSD 5.3 and on both
> servers problem has repeated. I have a local machine with
> Linux and on it problems have not arisen.....
> 
> Short example - data between 1400 and 2800 bytes would be
> transferred very slowly
> 
> su-2.05b# ./memd_test 1248
> memd_test: mc_aget result: <0.0001540>
> 
> su-2.05b# ./memd_test 1440
> memd_test: mc_aget result: <0.0998720>
> 
> su-2.05b# ./memd_test 10440
> memd_test: mc_aget result: <0.0005540>
> 
> 1440 bytes are transferred from memcached at 0.09 seconds,
> but 10440 bytes at 0.0005......
> 
> I upgrade em driver to version 2.05, but it has not
> helped...
> On lo device (127.0.0.1) all OK.
> 
> I test it with libevent 0.9, 1.0, 1.0b...
> I test memcached 1.1.11, 1.1.12cr1, 1.1.9-snapshot
> I using libmemcached for C test and Cached::Memcached for
> perl...
> 
> Probably problem in a freeBSD kernel or em driver... I don't
> know :(
> 
> Any ideas?
> 
> Regards,
>     Alermo
> 
> ------ simple memd_test.c based on regress.c from
> libmemcached ------
> 
> /*
> 
> memcached test on freeBSD 5.3
> data between 1400 and 2800 bytes would be transferred very
> slowly - 0.1 sec
> device em (Intel(R) PRO/1000 Gigabit Ethernet driver)
> 
> on device lo all OK.... Very strange.
> 
> */
> 
> #include <err.h>
> #include <sysexits.h>
> #include <stdio.h>
> #include <stdlib.h>
> #include <string.h>
> #include <unistd.h>
> #include <sys/time.h>
> #include <errno.h>
> 
> #include <memcache.h>
> 
> int
> main(int argc, char *argv[]) {
>   struct memcache *mc = NULL;
>   u_int32_t long_string_size = 0;
>   char *long_string;
>   u_int32_t i;
> 
>   if (argc > 1)
>     long_string_size = strtol(argv[1], NULL, 10);
> 
>   if (long_string_size == 0)
>     long_string_size = 2750;
> 
>   mc = mc_new();
>   if (mc == NULL)
>     err(EX_OSERR, "Unable to allocate a new memcache
> object");
> 
>   mc_server_add(mc, "192.168.1.1", "11211");
> 
>   long_string = malloc(long_string_size);
> 
>   for (i = 0; i < long_string_size; ++i)
>     long_string[i] = '0';
> 
>  mc_set(mc, "long_poisoned_string",
> MCM_CSTRLEN("long_poisoned_string"), long_string,
> long_string_size, 0, 0);
> 
>   struct timeval t0,t1;
>   int res0,res1;
>   void *val;
> 
>   res0 = gettimeofday(&t0,NULL);
>   if(res0 < 0)
>     warnx("Error found: %s\n",errno);
> 
>  // for (i = 0; i < 100; ++i )
>      val = mc_aget(mc, "long_poisoned_string",
> strlen("long_poisoned_string"));
>  // warnx("res: %s\n",val);
> 
>   res1 = gettimeofday(&t1,NULL);
>   if(res1 < 0)
>     warnx("Error found: %s\n",errno);
> 
>   double tdif,sdif;
>   sdif = t1.tv_usec-t0.tv_usec;
>   sdif = sdif/1000000;
>   tdif = t1.tv_sec-t0.tv_sec;
>   tdif = tdif+sdif;
> 
>   warnx("mc_aget result: <%.7f>\n", tdif );
> 
>   mc_free(mc);
> 
>   return EX_OK;
> }
> 
>