list_keys listing all keys

[Brad Fitzpatrick]

Really what we need is a:

   list_keys after=&limit=1000

Which means "return up to 1000 keys with key after ''".

Then we get to "foo" as the 1000th item in the response, and client can
re-issue:

   list_keys after=foo&limit=1000

And so on.

And prefix can still live as an additional filter, in addition to after=.

Care to write a patch?


On Mon, 20 Nov 2006, Brad Fitzpatrick wrote:

> The whole server method is flawed because there's no way to iteratively
> return stuff via the protocol.  It's single-line one-response one-answer,
> so if your request is "All keys please!" the answer might be a 2GB single
> line, which would probably break dozens of layers as it traverses stuff.
>
> So the list_keys limitation in the server is really a "I bet this isn't
> what you want." type of thing.
>
>
> On Mon, 20 Nov 2006, Daniel Burke wrote:
>
> >
> > Hi MogileFSers,
> >
> > If list_keys is passed an empty prefix, it will return an error:
> >
> >   $mogc->list_keys("", undef);
> >
> > I had expected all keys to be returned but is this an intentional
> > design decision?
> >
> > If I apply the attached patch to Worker/Query.pm::cmd_list_keys:476
> >
> >   return $self->err_line("no_key") unless $prefix;
> >
> > To become:
> >
> >   return $self->err_line("no_key") unless defined $prefix;
> >
> > list_keys in the Client API will now list all keys if given an empty
> > prefix.
> > Will anything bad happen? I ask because I think there must be a good
> > reason for this design decision and I want to be careful.
> >
> > Thanks!
> > Daniel
> >
> >
> >
>
>