Namespaces

[Skylos]

It probably does make sense to store a structure of some sort for each
zip, since you're querying all that each time.

zipcode => { table1 => { state => Y, fetchversion => 112 }, table2 ->
{ state => N, fetchversion => 113}}

And so on.  You just check your versions, or timestamps, or wahtever
to see if the data for that particular table is valid.  if its not,
requery, store.

And you don't have to ask the db for table version, OR run a diff
memcached for it.  Just store that in the same memcached too.

table1-version => 112
table2-version => 112

it would fetch the table2 data over, but not hte table 1 data over in
this example case.

Of course, if your updates aren't made by an application that can
update the memcached, you might want to create a lil daemon that runs
a query at an acceptable interval to see if it has.   Or,
alternatively, set the table1-version key to expire after this
acceptable interval, which will force the application to do the query.
 (the problem being, of course, that multiple processes could see it
not there and try and run the update simultaneously.)

TIMTOWTDI, after all.

Skylos


On 6/23/05, Eamon Daly <edaly at nextwavemedia.com> wrote:
> I didn't really provide information about my particular
> application, so let me outline that and perhaps someone will
> have a workaround: I have 100 tables, each containing 20,000
> rows, with zipcode as the PK. When a zipcode comes in, our
> application checks each table and reports which tables
> contain that zipcode.
> 
> With namespaces, the table name is the namespace name, each
> zipcode requested is a key, and the response is the value.
> Cache maintenance is simple: I delete the namespace when the
> underlying table is changed.
> 
> Your suggestion is a good one, but unless I store a complex
> object in the value:
> 
> 94110 -> { 'table_1' => 'Y', 'table_2' => 'N' ... }
> 
> it breaks down in my particular case. Also, I'd still have
> to ask the database for the table version on every request,
> or maintain a separate cache of table updates and query
> against that. That's certainly cheaper than going to the
> database, but still a lot of activity.
> 
> I appreciate the response! Any other ideas?
> 
> ____________________________________________________________
> Eamon Daly
> 
> 
> 
> ----- Original Message -----
> From: "Brad Fitzpatrick" <brad at danga.com>
> To: "Eamon Daly" <edaly at nextwavemedia.com>
> Cc: <memcached at lists.danga.com>
> Sent: Thursday, June 23, 2005 10:50 AM
> Subject: Re: Namespaces
> 
> 
> > You can just cache a table version number in each row's day.  So
> >
> > 94110 -> "1/San Francisco, CA"
> >
> > And then you fetch 94110, split the version/city apart, see if the version
> > matches your table version.
> >
> >
> > On Thu, 23 Jun 2005, Eamon Daly wrote:
> >
> >> Any feel for when namespaces and clearing by namespace might
> >> be implemented? We want to cache some really large tables of
> >> zipcodes, but it doesn't make sense to cache entire tables
> >> when only 5-10% of the rows are ever hit. We'd like to cache
> >> rows as they're requested, but then there's no way to delete
> >> all of them when the underlying table is updated.
> >>
> >> ____________________________________________________________
> >> Eamon Daly
> 
>