successive writes are lost
Jan Pfeifer
pfjan at yahoo.com.br
Wed Jan 24 22:51:01 UTC 2007
Sorry for the confusion. Actually I was doing something wrong on the pre-filling loop ...
So I guess I can't reproduce any problems then :)
----- Original Message ----
From: Jan Pfeifer <pfjan at yahoo.com.br>
To: Steven Grimm <sgrimm at facebook.com>
Cc: memcached list <memcached at lists.danga.com>; Peter Yen <peteryen at yahoo-inc.com>
Sent: Wednesday, January 24, 2007 2:09:52 PM
Subject: Re: successive writes are lost
Cool, thanks :)
So I'm using the 1.2.1, on Ubuntu Edgy (Linux 2.6).
I'm sending the code to you (and my friend) only, as it is part of a bigger test program, and I don't want to consume the list bandwidth -- but feel free to send to the list if you prefer
cheers,
- jan
----- Original Message ----
From: Steven Grimm <sgrimm at facebook.com>
To: Jan Pfeifer <pfjan at yahoo.com.br>
Cc: memcached list <memcached at lists.danga.com>; Peter Yen <peteryen at yahoo-inc.com>
Sent: Wednesday, January 24, 2007 2:00:53 PM
Subject: Re: successive writes are lost
Any chance you can send your test program to me? If I can reproduce
that, I'll definitely work on fixing it. We haven't noticed that
behavior but I suppose if it's a rare condition, it might have been
happening infrequently enough to escape our attention.
Which version of memcached are you running against?
-Steve
Jan Pfeifer wrote:
> hi,
>
> I was re-running some performance test, after a friend commented that memcache lost some writes in some cases.
>
> I
> noticed that if I do a program to write 50k entries (keys are the
> number 1 to 50000 in string format, and values are a 100 chars string)
> in a 500mb cache (enough space), I only get ~65% of them written
> (according to "stats items" and to the cache miss ratio I get after
> writting).
>
> But, if I do a 10microsec pause in between each write, they all get through without a problem.
>
> I
> edited the server, and adding a few printf's I noticed that item_alloc
> and slab_alloc are called the 50k times, and return an ok results
> ("STORED").
>
>
> any ideas of what is happening, and how can we avoid this ?
>
>
> thanks :)
>
> - jan
>
>
>
>
>
>
>
>
>
> ____________________________________________________________________________________
> Don't pick lemons.
> See all the new 2007 cars at Yahoo! Autos.
> http://autos.yahoo.com/new_cars.html
>
____________________________________________________________________________________
Sucker-punch spam with award-winning protection.
Try the free Yahoo! Mail Beta.
http://advision.webevents.yahoo.com/mailbeta/features_spam.html
#include <iostream>
#include <boost/smart_ptr.hpp>
#include <boost/lexical_cast.hpp>
#include <boost/thread/thread.hpp>
#include <boost/random.hpp>
#include "MemCache.hh"
using namespace std;
using namespace Answers;
using namespace boost;
using namespace boost::random;
boost::mt19937 rng;
boost::uniform_real<> uni_dist(0,1);
boost::variate_generator<mt19937&, boost::uniform_real<> > rand_real(rng, uni_dist);
struct CacheTestSet {
unsigned long num_of_keys;
string value;
time_t stop_time;
unsigned long average_sleep_useconds;
unsigned long count_reads, count_writes;
unsigned long total_waiting_read_time_useconds, total_waiting_write_time_useconds;
mutex m;
};
int a = 0;
class CacheTester
{
public:
CacheTester( shared_ptr<CacheTestSet> test_set ) : test_set( test_set ), cache() {}
void run()
{
while ( !test_end() )
{
process_key();
update();
if ( test_end() ) break;
sleep();
}
}
private:
shared_ptr<CacheTestSet> test_set;
MemCache cache;
boost::thread *thrd;
unsigned long read_time, write_time;
void process_key()
{
string key = lexical_cast<string>( (long) ( rand_real() * test_set->num_of_keys ) );
string value;
struct timeval tv_begin, tv_end;
gettimeofday( &tv_begin, 0 );
cache.get( key, value );
gettimeofday( &tv_end, 0 );
read_time = (tv_end.tv_sec - tv_begin.tv_sec) * 1000000L + tv_end.tv_usec - tv_begin.tv_usec;
if ( rand_real() < 0.1 ) {
gettimeofday( &tv_begin, 0 );
cache.set( key, test_set->value, 0, 0 );
gettimeofday( &tv_end, 0 );
write_time = (tv_end.tv_sec - tv_begin.tv_sec) * 1000000L + tv_end.tv_usec - tv_begin.tv_usec;
} else {
write_time = 0;
}
}
bool test_end()
{
return ( time(0) >= test_set->stop_time );
}
void sleep()
{
if ( test_set->average_sleep_useconds <= 0 ) return;
unsigned long sl = (unsigned long) ( rand_real() * 2 * test_set->average_sleep_useconds );
usleep( sl );
}
void update()
{
mutex::scoped_lock( test_set->m );
if ( read_time > 0 ) {
test_set->count_reads ++;
test_set->total_waiting_read_time_useconds += read_time;
}
if ( write_time > 0 ) {
test_set->count_writes ++;
test_set->total_waiting_write_time_useconds += write_time;
}
}
};
class Runner
{
public:
Runner( CacheTester *ct ) : ct(ct) {}
Runner() : ct(0) {}
void operator()()
{
ct->run();
}
void set( CacheTester *_ct ) { ct = _ct; }
private:
CacheTester *ct;
};
//
// Start up threads
int main( int argc, char *argv[] )
{
try {
// help
if ( argc != 7 || argv[1] == "-h" || argv[1] == "--help" ) {
cout << argv[0]
<< " <# of simultaneous threads>"
<< " <# of seconds running>"
<< " <average # of microseconds between each request per thread>"
<< " <# of different keys to use>"
<< " <value size>"
<< " <fill cache? (0|1)>"
<< endl;
return 1;
}
// get arguments
int seconds_to_run = lexical_cast<int>( argv[2] );
shared_ptr<CacheTestSet> test_set( new CacheTestSet );
int num_threads = lexical_cast<int>( argv[1] );
test_set->stop_time = time(0) + seconds_to_run;
test_set->average_sleep_useconds = lexical_cast<unsigned long>( argv[3] );
test_set->num_of_keys = lexical_cast<unsigned long>( argv[4] );
test_set->value = string( lexical_cast<unsigned>( argv[5] ), 'a' );
cout << "Value being used (" << lexical_cast<unsigned>( argv[5] ) << ") : " << test_set->value << endl;
int prefill = lexical_cast<bool>( argv[6] );
// prefill if requested
if ( prefill ) {
MemCache cache;
struct timeval tv_begin, tv_end;
struct timespec pause = { 0, 1000 };
gettimeofday( &tv_begin, 0 );
for ( unsigned i = 0; i < test_set->num_of_keys; i++ ) {
string key = lexical_cast<string>( (long) ( rand_real() * test_set->num_of_keys ) );
cache.set( key, test_set->value, 0, 0 );
// nanosleep( &pause, 0 );
}
gettimeofday( &tv_end, 0 );
unsigned long write_time = (tv_end.tv_sec - tv_begin.tv_sec) * 1000000L + tv_end.tv_usec - tv_begin.tv_usec;
cout << "Cache prefilled, " << (double) test_set->num_of_keys / (double) write_time * 1e6 << " writes per second" << endl;
gettimeofday( &tv_begin, 0 );
unsigned failed = 0;
for ( unsigned i = 0; i < test_set->num_of_keys; i++ ) {
string key = lexical_cast<string>( (long) ( rand_real() * test_set->num_of_keys ) );
string v;
if ( ! cache.get( key,v ) ) failed ++;
else if ( v != test_set->value ) cerr << "returned wrong value!!!" << endl;
// nanosleep( &pause, 0 );
}
gettimeofday( &tv_end, 0 );
unsigned long read_time = (tv_end.tv_sec - tv_begin.tv_sec) * 1000000L + tv_end.tv_usec - tv_begin.tv_usec;
cout << "Cache prefill checked, " << (double) test_set->num_of_keys / (double) read_time * 1e6 << " reads per second, "
<< failed << " failed" << endl;
return 0;
}
CacheTester *testers[num_threads];
int i;
for ( i = 0 ; i < num_threads; i ++ ) testers[i] = new CacheTester( test_set );
Runner runners[ num_threads ];
for ( i = 0 ; i < num_threads; i ++ ) runners[i].set( testers[i] );
thread *threads[ num_threads ];
try {
for ( i = 0 ; i < num_threads; i ++ ) threads[i] = new thread( runners[i] );
} catch ( ... ) {
cerr << "Threads created: " << i << endl;
throw;
}
for ( i = 0 ; i < num_threads; i ++ ) { threads[i]->join(); delete threads[i]; threads[i] = 0; }
for ( i = 0 ; i < num_threads; i ++ ) { delete testers[i]; testers[i] = 0; }
// print statistics
cout << "Rate of read requests: " << (double)test_set->count_reads / (double)seconds_to_run << " reqs/s, " << test_set->count_reads << " total" << endl;
cout << "Rate of write requests: " << (double)test_set->count_writes / (double)seconds_to_run << " reqs/s," << test_set->count_writes << " total" << endl;
cout << "Total time passed waiting for requests (useconds): " << test_set->total_waiting_read_time_useconds << endl;
cout << "Average reading time (useconds/request): "
<< (double) test_set->total_waiting_read_time_useconds / (double)test_set->count_reads << endl;
cout << "Average writing time (useconds/request): "
<< (double) test_set->total_waiting_write_time_useconds / (double)test_set->count_writes << endl;
return 0;
} catch (exception &e) {
cerr << "Exception: " << e.what() << endl;
return 1;
}
}
#ifndef MEMCACHED_HH
#define MEMCACHED_HH
extern "C" {
#include <memcache.h>
}
#include <string>
#include <vector>
#include <stdexcept>
#include <boost/lexical_cast.hpp>
namespace Answers {
class MemCache
{
public:
MemCache();
~MemCache();
inline void set( const std::string &key, const std::string &value, const time_t expire, const u_int16_t flags );
inline bool get( const std::string &key, std::string &value );
//bool get( const std::vector<const std::string> &keys, std::vector<std::string> &values,
// std::vector<bool> &found = std::vector<bool>());
private:
struct ::memcache *mc;
};
////////////////////////////////////////////////////////////////////////////////
//
// Implementation
//
////////////////////////////////////////////////////////////////////////////////
inline void MemCache::set( const std::string &key, const std::string &value, const time_t expire, const u_int16_t flags )
{
if ( key.empty() ) return;
/* Add a key */
int ret = ::mc_set(mc, (char *) key.c_str(), key.length(), value.c_str(), value.length(), expire, flags);
if ( ret != 0 ) throw std::runtime_error( "MemCache::MemCache(): mc_put failed "+boost::lexical_cast<std::string>(ret) );
}
inline bool MemCache::get( const std::string &key, std::string &value )
{
struct memcache_req *req = mc_req_new();
if ( ! req ) throw std::bad_alloc(); // "MemCache::get(): can't create memcache_req object with mc_req_new()" );
struct memcache_res *res = ::mc_req_add(req, (char*) key.c_str(), key.length());
::mc_get(mc, req);
if ( ! mc_res_found( res ) ) {
::mc_req_free(req);
return false;
}
value = std::string( (char*)res->val, (char*)res->val+res->size );
::mc_req_free(req);
return true;
}
}
#endif
#include "MemCache.hh"
using namespace std;
Answers::MemCache::MemCache() {
mc = mc_new();
if ( !mc ) throw std::bad_alloc(); // "Can't create memcached object" );
/* Add a few servers */
int ret = mc_server_add(mc, "127.0.0.1", "11211");
if ( ret != 0 ) throw std::runtime_error( "MemCache::MemCache(): mc_server_add failed" );
//mc_server_add(mc, "127.0.0.1", "11212");
//mc_server_add4(mc, "127.0.0.1:11213");
}
Answers::MemCache::~MemCache() {
if ( mc ) mc_free(mc);
mc = 0;
}
/*
mc_server_stats_free(s);
// Storage commands:
mc_add(mc, key, key_len, val, bytes, expire, flags);
mc_replace(mc, key, key_len, val, bytes, expire, flags);
mc_set(mc, key, key_len, val, bytes, expire, flags);
// Delete commands:
mc_delete(mc, key, key_len, hold_timer);
// Atomic opts:
mc_incr(mc, key, key_len, 1);
mc_decr(mc, key, key_len, 1);
*/
____________________________________________________________________________________
The fish are biting.
Get more visitors on your site using Yahoo! Search Marketing.
http://searchmarketing.yahoo.com/arp/sponsoredsearch_v2.php
More information about the memcached
mailing list