Inconsistent output of 'mogadm domain list' encountered

[dormando]

That's really odd... I had been testing and reloading and it wasn't 
doing that for me at all.

I see I did confuse the way of_dmid works when I placed the singleton 
cache clear... Thought the name2id check was above the singleton check 
(which calls reload_domains if the name map fails). The issue also 
wasn't happening again with my patch :\

It's entirely possible I goofed my test setup, so I'll fix this later 
today. Would be nice to figure out what I did wrong with the testing.

The changes to Class's caching issue should be good though. You test those?

-Dormandou

Chaos Wang wrote:
> Emm...I'm afraid making modifications only in
> MogileFS::Domain::reload_domains() won't be enough...
> 
> The MogileFS::Domain::delete() method called
> MogileFS::Domain::invalidate_cache() instead of reload_domains(), so if
> you deleted a domain, then try to retrieve domain instance through
> MogileFS::Domain::of_dmid() with the deleted domain's ID, the issue will
> come up again.
> 
> I have no idea about the reason of having cache invalidation code both
> in reload_domains() and invalidate_cache(), but it seems to be a better
> solution to keep that code only in invalidate_cache() and have
> reload_domains() calling it at first.
> 
> This issue don't need parallel environment to be revealed, here is the
> corresponding testing snippet I appended in
> mogilefs/server/t/domains-classes.t :
> 
> MogileFS::Domain->create("bar");
> my $tmp_dom=MogileFS::Domain->of_namespace("bar");
> my $tmp_dmid=$tmp_dom->{dmid};
> $tmp_dom->delete;
> ok(!defined(MogileFS::Domain->of_dmid($tmp_dmid)), "no domain info
> remained in cache after deletion");
> 
> dormando wrote:
>> Fixed both instances in trunk...
>>
>> They were trivial changes (although I put the cache clear in a different
>> place), but I might not have totally understood the original goals of
>> the singleton caches. Feel free to berate my fail :)
>>
>> Also haven't figured a decent test, since the issue tends to happen in
>> parallel processes. Any ideas?
>>
>> -Dormando
>>
>> Frank Ng wrote:
>>   
>>> We see the same thing in our test mogilefs env.  i made a change of
>>> mindevcount from 2 to 3 and ran mogadm class list a few times, it's show
>>> alternating results.
>>>  
>>> [root at xxx mogdata]# mogadm class list
>>>   domain               class                mindevcount
>>> -------------------- -------------------- -------------
>>>  snf_test01           default                   2
>>>  snf_test01           snf_hr                    2
>>>  snf_test01           snf_lr                    3 <--
>>>  
>>> [root at xxx mogdata]# mogadm class list
>>>  domain               class                mindevcount
>>> -------------------- -------------------- -------------
>>>  snf_test01           default                   2
>>>  snf_test01           snf_hr                    2
>>>  snf_test01           snf_lr                    2 <--
>>>  
>>> [root at xxx mogdata]# mogadm domain list
>>>  domain               class                mindevcount
>>> -------------------- -------------------- -------------
>>>  snf_test01           default                   2
>>>  snf_test01           snf_hr                    2
>>>  snf_test01           snf_lr                    2 <--
>>>  
>>> [root at xxx mogdata]# mogadm domain list
>>>  domain               class                mindevcount
>>> -------------------- -------------------- -------------
>>>  snf_test01           default                   2
>>>  snf_test01           snf_hr                    2
>>>  snf_test01           snf_lr                    3 <--
>>>
>>>  
>>> On Wed, Mar 12, 2008 at 3:58 AM, Chaos Wang <chaoslawful at gmail.com
>>> <mailto:chaoslawful at gmail.com>> wrote:
>>>
>>>     Hi,
>>>
>>>     I use the MogileFS code from official svn trunk to do some test. But
>>>     when I do some domain addition/deletion operations, the output of
>>>     'mogadm domain list' becomes inconsistent between different calls.
>>>     This phenomenon can be repeated by following steps:
>>>
>>>        1. mogadm domain add xxx
>>>        2. mogadm domain delete xxx
>>>        3. mogadm domain add yyy
>>>        4. mogadm domain list (several times)
>>>
>>>     I thinks the problem is in the invalidate_cache() method in
>>>     MogileFS::Domain module, because it forget to clean up the hash map
>>>     %singleton (of_dmid(), which is called in domains(), used this hash
>>>     map before trying %id2name map). After made the following changes in
>>>     MogileFS::Domain::invalidate_cache(), the phenomenon doesn't show up
>>>     again:
>>>
>>>     --code begin--
>>>     sub invalidate_cache {
>>>     ...
>>>     %id2name=();
>>>     %name2id=();
>>>     %singleton=(); # XXX: this is newly added
>>>     ...
>>>     --code end--
>>>
>>>     And I have another question: does classes belong to certain domain?
>>>     If so, why delete a domain does not clean up the classes under it?
>>>
>>>
>>>     
>>
>>   
>