Performance when changing the length of a keys value. (It gets really bad.)

[Alex Stapleton]

OK, Here are my tests running the tummy3 fork. Summary: It's a lot  
better than the danga version.

I restarted memcached between each run

Writing at least 4 MB of data in 5000, 1 KB blocks
Took 0.572039842606 seconds
8740.64991212 t/s
0.000114407968521 s/t
17.0715818596 MB/s

Writing at least 78 MB of data in 5000, 16 KB blocks
Took 1.12143111229 seconds
4458.58862413 t/s
0.000224286222458 s/t
139.330894504 MB/s

Writing at least 156 MB of data in 5000, 32 KB blocks
Took 1.65031003952 seconds
3029.73373503 t/s
0.000330062007904 s/t
189.35835844 MB/s

My memcached has 256 megs of ram to play with, and doesn't swap.

Extrapolating from those 3 samples (it draws a pretty linear line if  
you do blocksize vs absolute time) you would expect it to take about  
2.5 seconds for 64kb blocks.

It takes a lot longer. Doing tests with smaller samples (like 10-50  
instead of 5000) it suggests that it gets >100 times slower when your  
block size gets that large.




On 14 Jul 2005, at 10:21, Alex Stapleton wrote:

> I still seem to get hugely slow performance using large data chunks  
> as well. It's great until I get up over 32k ish. I expect this is  
> probably a python problem though.
>
> On 14 Jul 2005, at 08:30, Alex Stapleton wrote:
>
>
>
>> Ah that's great, thanks.
>>
>> On 14 Jul 2005, at 05:14, Evan Martin wrote:
>>
>>
>>
>>
>>> See also
>>> http://cheeseshop.python.org/memcached/1.2_tummy3
>>>
>>> On 7/13/05, Alex Stapleton <alexs at advfn.com> wrote:
>>>
>>>
>>>
>>>
>>>> I saw the reference to potential slowness readline() in the python
>>>> client. I will try and fix that and submit my changes, and the
>>>> results of any changes it has on my performance problem.
>>>>
>>>> I couldn't recreate the problem on my last batch of tests, but I
>>>> think I was using a smaller sample size. It definitely wasn't the
>>>> same as my original run.
>>>>
>>>> On 13 Jul 2005, at 18:10, Brad Fitzpatrick wrote:
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>> On Wed, 13 Jul 2005, Alex Stapleton wrote:
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>> Here is my test script. It is written in Python using the danga
>>>>>> memcached module. I hope nothing mangles the formatting.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>> Iirc, the Danga python module is/was inefficient, doing one  
>>>>> syscall
>>>>> per
>>>>> byte read/written.  Hopefully that's not what's biting you.
>>>>>
>>>>> If somebody has time, I'd love to see this in C or Perl (clients I
>>>>> trust),
>>>>> before I go chasing possibly non-existent bugs in the server.
>>>>>
>>>>> - Brad
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>>
>>>
>>
>>
>>
>>
>
>
>