Delete Objects based on REGEX.

Thu Jul 26 13:39:53 UTC 2007

Prem,

Honestly I cannot be helpful on this or other topics about memcached:
I'm a project manager, former C#/.NET developer, and I don't have the
knowledge to work on the Perl project to implement new features or fix
bugs.
My discussions are generally related to abstract concepts, that are
true on all the systems.

The idea of tags it's a good one since it gives a fast way to access
shared portions of memory, joined by the definition of common elements
(tags): in this way, for instance, if you have stored 2 versions of a
serialized user profile, one including more data, the other
simplified, removing a commo tag will remove both the versions, while
removing a specific tag will delete just the version specified by the
tag itself.

Cheers.
Antonello

On 7/26/07, P R E M <prem_somanin at yahoo.com> wrote:
> Hi,
>
> If REGEX implementation is going to affect the performance, then we have to opt for another solution. The TAG implementation is also not clear to me.
>
> How about maintaining the relationship - the main key with all related keys[which are the actual keys] in a HashMap, within the application. So each time when I want to delete the set, I simply give the main key and obtain all other related keys from the hash, then delete the cache one by one.
>
> eg : Key-1 will be mapped as,
>
> key-1
>        |
>        ` ------ key-1-aaa
>         ` ------ key-1-bbb
>         ` ------ key-2-ccc
>
> And by a lookup into my local map with key as "key-1", I will be able the actual set of keys "key-1-aaa...bbb...ccc"
>
>
> Am I into right track. Please letme know.
>
> regards,
> Prem
>
>
> ----- Original Message ----
> From: Antonello Provenzano <antonello at deveel.com>
> To: Steven Grimm <sgrimm at facebook.com>
> Cc: "BUSTARRET, Jean-francois" <jfbustarret at wat.tv>; P R E M <prem_somanin at yahoo.com>; memcached at lists.danga.com
> Sent: Thursday, July 26, 2007 3:05:31 PM
> Subject: Re: Delete Objects based on REGEX.
>
> Steven,
>
> > I actually implemented regex deletes quite some time back. But the
> > problem is that I did it in a brute-force way, nothing elegant or
> > efficient, and if you have a large cache the whole thing will freeze up
> > for many seconds, chewing 100% of your CPU while the code walks through
> > all the items comparing them against the regex. We wanted it for very
> > occasional administrative operations where the cache freezing up for a
> > short while would be less disruptive than a full flush, but I think
> > we've used it maybe two or three times at most because it's so nasty.
>
> That's what I was trying to say. I created a project for Mono project
> (named Deveel Regex) for the compilation and execution of RegEx
> expressions on strings and I know what I'm speaking about: although
> Deveel Regex is quite fast (faster than the built in solution in
> Microsoft .NET and Mono), the parsing, compilation and execution of a
> RegEx is a quite stressing process, specially if repeated thousands of
> times per second.
>
> Maybe the amount of memory and CPU used by Perl is much smaller than
> Mono or .NET (which are behind a virtual machine as Java), since I've
> never worked with it I have no benchmarks for this, I still believe
> that RegEx in this kind of context could be a killing feature that
> would stress the machine memcached would run on and cause problems in
> long terms.
>
> Antonello
>
>
>
> On 7/26/07, Steven Grimm <sgrimm at facebook.com> wrote:
> > I actually implemented regex deletes quite some time back. But the
> > problem is that I did it in a brute-force way, nothing elegant or
> > efficient, and if you have a large cache the whole thing will freeze up
> > for many seconds, chewing 100% of your CPU while the code walks through
> > all the items comparing them against the regex. We wanted it for very
> > occasional administrative operations where the cache freezing up for a
> > short while would be less disruptive than a full flush, but I think
> > we've used it maybe two or three times at most because it's so nasty.
> >
> > Brad's generation-based idea makes much more sense as a long-term
> > approach to this feature, but honestly I like tags a lot better. I'd
> > rate tags a 10 and regex deletes, even well-implemented ones, at most a
> > 6 or 7 because they're so inflexible and require that all consumers of
> > objects in the cache be aware of all the metadata you can use to address
> > a particular object (because it's all part of the key and you need the
> > whole key to query the cache.) Tags are IMO far superior because you can
> > tag an item with metadata that some of your clients are blissfully
> > unaware of.
> >
> > -Steve
> >
> >
> > BUSTARRET, Jean-francois wrote:
> > > IMHO, regexp deletes would be great. I would prefer tags, but on a scale of 10 (0 being the current state), tags would be at 10, and regexps at 9.
> > >
> > > I my case, it would solve the most important cache coherency problems.
> > >
> > > I'm willing to help (I would be unable to code the most complex part of it, but still willing to help).
> > >
> > >
> > >> -----Message d'origine-----
> > >> De : memcached-bounces at lists.danga.com
> > >> [mailto:memcached-bounces at lists.danga.com] De la part de
> > >> Dustin Sallings
> > >> Envoyé : mercredi 25 juillet 2007 23:14
> > >> À : dormando
> > >> Cc : P R E M; memcached at lists.danga.com
> > >> Objet : Re: Delete Objects based on REGEX.
> > >>
> > >>
> > >> On Jul 25, 2007, at 12:59 , dormando wrote:
> > >>
> > >>
> > >>> Seems interesting, don't recall if anyone was specifically
> > >>>
> > >> against it.
> > >>
> > >>> The implementation details are a bit foggy, and someone has
> > >>>
> > >> to go do
> > >>
> > >>> it still.
> > >>>
> > >>      The biggest complaints are that people feel that it may
> > >> not solve actual problems people have.  The alternative is
> > >> tags, but that seems more complicated (to use) to me.
> > >>
> > >>      In the end, if nobody's problems are solved, people
> > >> probably aren't going to be writing any code for it.
> > >>
> > >> --
> > >> Dustin Sallings
> > >>
> > >>
> > >>
> > >>
> >
> >
> >
>
>
>
>
>
>
>
>       ____________________________________________________________________________________
> Fussy? Opinionated? Impossible to please? Perfect.  Join Yahoo!'s user panel and lay it on us. http://surveylink.yahoo.com/gmrs/yahoo_panel_invite.asp?a=7
>
>
>