memcached memory usage

Denis Antipov antipovd at gmail.com
Wed Dec 7 07:23:02 UTC 2005


Hello, folks

I've been continuing to test memcached and trying to go about my problem
with overwritten/lost data in memcached without the use of a database or
some other solid data source. Yes I'm stubborn. Anyway. I've written a
script that shows, IMHO, that memcached doesn't utilize memory efficiently,
either due to it's queue and class slab allocation algorithm, or some other
unknown to me reason. See the PHP script below. I've used Apache Bench with
50000 requests on a memcached with 1Gb maxbytes. The script outputs a CSV
line for each data "set" in memcached. It records time, current items, total
items, the key, and the data size that was stored for that key (the amount
of data is random for each key). For my run, I noticed that at about 15K
items no more bytes were "used", meaning that no more memory was allocated
and no more items can be stored, the overwriting has begun. The problem is
that bytes kept staying at 560Mb which means it's using 56% of the allowed
memory. I've read about memory space inefficiency with the classes and such,
but 50%!!! Isn't it a bit low?!!! Can we address this issue somehow, was
there anything done besides the class reasignment script in scripts, which
I'm not sure how would solve this problem anyway?
Thanks,

Denis

<snippet>

<?php
session_start();
session_regenerate_id();
$sess = session_id();
ignore_user_abort(true);

$data_file = "data.csv";
$fail_file = "conn_fail.log";
$flag = 'a';

$memcache = new Memcache;
if(!$memcache->connect('192.168.1.253', 11213)){
 if(file_exists($fail_file)){
   $flag = 'a';
 } else {
   $flag = 'w';
 }
   $ffh = fopen($fail_file, $flag) or die ("unable to open ".$fail_file);
   while(1) {
     if (flock($ffh, LOCK_EX)) {
     fwrite($ffh, $sess."\n") or die ("failed to write to ".$fail_file);
         flock($ffh, LOCK_UN);
     break;
     }
   }
 fclose($ffh);

} else {
  for($i=rand(1,13);$i>0;$i--){ // 13 is no not go over the 1MB limit
    $sess.=$sess;
  }

  $size = strlen($sess)*2; // 2 bytes per character

  $stats = $memcache->getStats();
  reset($stats);

  if($memcache->set(session_id(), $sess, false, 0)){
    if(file_exists($fail_file)){
      $flag = 'a';
    } else {
      $flag = 'w';
    }

    $dh = fopen($data_file,$flag) or die ("unable to open ".$data_file);
    while(1) {
      if (flock($dh, LOCK_EX)) {
     fwrite($dh,
$stats['time'].",".$stats['curr_items'].",".$stats['total_items'].",".$stats['bytes'].",".session_id().",".$size."\n")

      or die ("failed to write to ".$data_file);
         flock($fdh, LOCK_UN);
     break;
      }
    }
    fclose($dh);
  }
}

?>

<\snippet>

--
"To a person with a hammer, all problems look like nails"
someone on Usenet
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.danga.com/pipermail/memcached/attachments/20051207/b58b935f/attachment.htm


More information about the memcached mailing list