Is incr/decr atomic in multithread mode?

[dormando]

The delta add can't really use atomic routines in this case, but the 
mutex could be split up more finely (we're only really locking that 
_item_, not the entire cache).

There *are* a number of places where atomic tests (from gcc and from 
solaris) would allow us to remove some locks. I think at least a few 
people from the list know where these parts are, and folks are just 
looking for time.

-Dormando

Paul van den Bogaard wrote:
> if there are too many mutex calls these can become scalability issues. 
> Solaris 10 has some atomic routines that could be helpfull.
> See 
> http://cvs.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/uts/common/sys/atomic.h 
> 
> 
> for the routines.
> 
> Of course if multiple operations are needed that need the protection of 
> a mutex than a atomic operation is no alternative, besides a CPU 
> consuming polling device. And sometimes these come in handy too.
> 
> --Paul
> 
> 
> On 25-feb-2008, at 10:53, Steve Chu wrote:
> 
>> On Mon, Feb 25, 2008 at 4:28 PM, dormando <dormando at rydia.net> wrote:
>>> Steve Chu wrote:
>>>> On Mon, Feb 25, 2008 at 2:51 PM, dormando <dormando at rydia.net> wrote:
>>>>> I think this is fine?
>>>>>
>>>>>  Since the operations are independently atomic, it's perfectly 
>>>>> valid for
>>>>>  two threads to incr the same value.
>>>>>
>>>>
>>>> But for client api, they seems NOT atomic, because when Thread A just
>>>> has 'item_get' done but before 'add_delta', Thread B can still do
>>>> item_get.
>>>>
>>>> X-> set "foo" 1
>>>> A-> item_get "foo"
>>>> B-> item_get "foo"
>>>> A-> add_delta "foo" by 1
>>>> B-> add_delta "foo" by 1
>>>>
>>>> then A and B both hold value 2, which one should be stored?
>>>>
>>>> And the exact value we need is 3, right? I am not quit sure, correct
>>>> me if i am wrong:)
>>>
>>>  The result gets copied into the thread-local 'temp' buffer, then copied
>>>  into that connection (implicitly atomic)'s output buffer.
>>>
>>>  So actually I read the code wrong the first time. This _should_ do the
>>>  right thing:
>>>
>>>  We have threads (A, B, X):
>>>
>>>  X -> set 'foo' 1
>>>
>>>  A -> process_arithmetic_command()
>>>  note thread local storage (char temp[])
>>>  B -> same.
>>>
>>>  A -> it = item_get(key, nkey)
>>>  (atomically increments the refcount for that item, returns a reference
>>>  to it)
>>>  B -> same, same time.
>>>  (same)
>>>
>>>  A -> out_string(c, add_delta(it, incr, delta, temp));
>>>  add_delta runs atomically, copies the result of the addition into the
>>>  thread local 'temp' buffer and _returns the 'temp' buffer_ (which
>>>  contains 2)
>>>  B -> out_string(c, add_delta(it, incr, delta, temp));
>>>  Gets serialized behind A, and its 'temp' then holds a value one higher
>>>  than A's (containing 3)
>>
>> Here I am confused. Every thread has a local var 'temp' and here
>> 'temp' should hold 2.  'temp' is independent, and doesn't share
>> between threads. So thread B should overwrite the value in the
>> slabber. Am I right?
>>
>>>  A -> runs out_string(c, temp), with 'temp' having the atomically
>>>  incremented value.
>>>  A -> inside out_string, copy's the contents of the original 'temp'
>>>  buffer into the connection's local write buffer, then sets the
>>>  connection into write mode (which will flush its buffer before doing
>>>  further processing). Then returns.
>>>  A -> returns to the state machine, 'temp' goes to the wolves, etc.
>>>  A -> sends value '2' to its client.
>>>
>>>  B -> Same thing. Client should receive a '3'.
>>>
>>>  Apologies to the original patch author and the list for the 
>>> confusion :)
>>>  I had mentally confused "normal" responses and "get" responses, which
>>>  operate differently.
>>>
>>>  -Dormando
>>>
>>>
>>>
>>>>>  It's possible that with two threads (A, B):
>>>>>
>>>>>  X -> set "foo" 1
>>>>>  A -> incr "foo" by 1
>>>>>  B -> incr "foo" by 1
>>>>>
>>>>>  A -> returns 3
>>>>>  B -> returns 3
>>>>>
>>>>>  The only guarantee the slabber makes for returning values, is that 
>>>>> they
>>>>>  won't get deleted before being transferred through the network.
>>>>>
>>>>>  Could be wrong, but that's my impression from checking the sources.
>>>>>
>>>>>  -Dormando
>>>>>
>>>>>
>>>>>
>>>>>  Steve Chu wrote:
>>>>>> In 'process_arithmetic_command' function:
>>>>>> We first do item_get from slabs and then do add_delta to incr/decr 
>>>>>> the value.
>>>>>>
>>>>>> My question is:
>>>>>> Is this atomic in multithread mode? Item_get is atomic and so is
>>>>>> add_delta because they are protected by mutex. But the entire two
>>>>>> operations seems NOT atomic. Two threads can both do item_get with
>>>>>> same key, then both incr it, and write back.
>>>>>>
>>>>>> Anybody can tell me whether it is a bug or not?
>>>>>>
>>>>>> Regards,
>>>>>>
>>>>>> Steve Chu
>>>>>
>>>>>
>>>>
>>>
>>>
>>
>>
>>
>> -- 
>> Steve Chu
>> http://stvchu.org
> 
> --------------------------------------------------------------------------------------------- 
> 
> Paul van den Bogaard                               
> Paul.vandenBogaard at sun.com
> ISV-E  -- ISV Engineering, Opensource Engineering group
> 
> Sun Microsystems, Inc                              phone:        +31 334 
> 515 918
> Saturnus 1                                                 extentsion: x 
> (70)15918
> 3824 ME Amersfoort                                 mobile:       +31 651 
> 913 354
> The Netherlands                                        fax:            
> +31 334 515 001
>