Patch to enable caching reverse proxy with mogstored

Nathan Schmidt nschmidt at
Fri Jan 5 07:45:15 UTC 2007

We're looking at storing and reading lots of small invariant files  
(wiki page revisions). It would be nice if they were cacheable on a  
local squid instance since our frontend machines have disks for  
bootstrapping but they're otherwise idle.

I measured in the 200-300/sec read rate from a Gig-E-adjacent  
mogstored instance with a 2kb file size, and then with this patch and  
an untuned squid process we got up to a 1500/sec read rate. On the  
client side we have a localhost-only memcache instance for key  
stashing. Between those two most requests for 'hot' keys never leave  
the local machine, and for the hottest they're purely memory-resident.

This particular change should be pretty innocuous and solves a  
problem we had, might be helpful for you too.

-Nathan Schmidt / PBwiki

--- mogstored   (revision 642)
+++ mogstored   (working copy)
@@ -149,6 +149,18 @@
harvest_dead_children();  # every 2 seconds, it reschedules itself
+# set headers on fids so they'll be retained indefinitely by a  
caching reverse proxy
'modify_response_headers', sub {
+                                               my $svc = shift;
+                                               if(200 == $svc-> 
{res_headers}->response_code()) {
+                                                 $svc->{res_headers}- 
 >header("Expires", "Tue, 19 Jan 2038 00:00:00 GMT");
+                                                 # epoch 2038
+                                                 $svc->{res_headers}- 
 >header("Cache-Control", "max-age=" . (2147483647 - time()));
+                                               }
+                                               # we don't want to  
interrupt sending this file, so return 0
+                                               return 0;
+                                             });
# now start the main loop

More information about the mogilefs mailing list