Multiget/intelligent generic PHP wrapper function... thoughts/advice wanted

[Gavin M. Roy]

How do you invalidate your cache when there are database changes?

On 10/31/07, mike <mike503 at gmail.com> wrote:
>
> I have a functional implementation of a generic cache wrapper that
> supports multiget transparently. Since it mixes both cache calls and
> database calls, this now becomes a full out "data access layer" type
> function I believe.
>
> I'm wondering if there are any additional optimizations I can do here,
> or if anyone has some pointers. I could have missed something VERY
> obvious and overengineered this :)
>
> Primarily, I use a key prefix of "tablename:" for the row information,
> for example. I suppose this is where namespaces would come in handy,
> so I don't have to iterate through the array to strip off the key
> prefix when doing the last array_diff_key() to look for the ones
> missing from the memcache_get().
>
> Has anyone else done anything like this and have it work properly with
> minimal overhead?
>
> I know of the the following caveats:
> - Still not reusable as it could be, I think.
> - I could add some typeof($var) to determine the key type. Right now I
> assume it is numeric.
>
> I just hate having to swap back and forth removing key prefixes and
> looping through the arrays. I don't think there is any way in PHP to
> rename array keys without that (unless it's just some array_walk type
> callback, but even then it's still a loop, maybe just less internal
> overhead?)
>
> Any help is appreciated. Thank you :)
>
> ~~~
>
> echo "<pre>\n";
> $foo = episode_get(Array(1000000004, 1000000006, 1000000014, 1000000011));
> var_dump($foo);
> echo "</pre>\n";
>
> function episode_get($keys) {
>         $table = "episodes";
>         $column = "ID";
>         return db_cache_get($table, $column, $keys);
> }
>
> function db_cache_get($table, $column, $requested_keys) {
>
>         $prefix = $table.':';
>
>         if(!isset($GLOBALS['ch'])) {
>                  cache_open();
>         }
>
>         if(!is_array($requested_keys)) {
>                 if(!$result = memcache_get($GLOBALS['ch'],
> $requested_keys)) {
>                         $q = db_query("SELECT * FROM $table WHERE
> $column=$requested_keys");
>                         list($result) = db_rows($q);
>                         db_free($q);
>                 }
>                 return $result;
>         }
>
> echo "key request array:\n";
> var_dump($requested_keys);
> echo "\n";
>         // build the key list
>         foreach(array_values($requested_keys) as $key) {
>                  $cache_keys[$prefix.$key] = $prefix.$key;
>         }
>
> echo "cache key array:\n";
> var_dump($cache_keys);
> echo "\n";
>
>         // do a multiget
>         $cache_array = memcache_get($GLOBALS['ch'], $cache_keys);
>
>         // if the result count is the same we have no need to go to the
> database
>         if(count($cache_keys) != count($cache_array)) {
>
> echo "cache returned array:\n";
> var_dump($cache_array);
> echo "\n";
>
>                 $need_array = array_diff_key($cache_keys, $cache_array);
>
>                 if(count($need_array) > 0) {
>                         $query_keys = '';
>                         foreach($need_array as $need_key) {
>                                 $k = str_replace($prefix, '', $need_key);
>                                 if(!empty($k)) { $query_keys .= $k.','; }
>                         }
>                         $query_keys = substr($query_keys, 0,
> strlen($query_keys)-1);
>                         $q = db_query("SELECT * FROM $table WHERE
> $column IN($query_keys)");
>                         while($r = db_rows_assoc($q)) {
>                                 $k = $prefix.$r[$column];
>                                 cache_set($k, $r);
>                                 $cache_array[$k] = $r;
>                         }
>                         db_free($q);
>                 }
>         }
>
>         $return = array();
>         foreach(array_keys($cache_array) as $item) {
>                 $newindex = str_replace($prefix, '', $item);
>                 $return[$newindex] = $cache_array[$item];
>                 unset($cache_array[$item]);
>         }
>         return $return;
> }
>
> function cache_set($key, $value, $flags = 0, $expiry = 2592000) {
>         if(!isset($GLOBALS['ch'])) {
>                  cache_open();
>         }
>         memcache_set($GLOBALS['ch'], $key, $value, $flags, $expiry);
>         return false;
> }
>
> function cache_open() {
>         $GLOBALS['ch'] = memcache_pconnect('localhost', 11211);
> }
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.danga.com/pipermail/memcached/attachments/20071031/5ce4abba/attachment.htm