The main reason I can't just rely on expires times that are included in the individual fragments is to support the Invalidation Spec<br><a href="http://www.w3.org/TR/esi-invp">http://www.w3.org/TR/esi-invp</a><br><br>The use cases for when you would want this kind of invalidation is for times when you have an object that needs to be invalidated. I'm actually in favor of not supporting this case, since IMO you can always store the unique information in the key required to invalidate. But there are some edge cases where invalidating becomes useful...
<br><br><div><span class="gmail_quote">On 9/10/07, <b class="gmail_sendername">Tres Seaver</b> <<a href="mailto:email@example.com">firstname.lastname@example.org</a>> wrote:</span><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
-----BEGIN PGP SIGNED MESSAGE-----<br>Hash: SHA1<br><br>Todd Fisher wrote:<br>> Hi all,<br>><br>> I've been working on implementing a ESI (Edge Side Include) Proxy caching<br>> server. The idea is very similar to SSI (Server Side Include), except the
<br>> includes are remote instead of local.<br>><br>> There's a very nice write up of using SSI in this context with nginx and<br>> memcached here =><br>> <a href="http://blog.kovyrin.net/2007/08/05/using-nginx-ssi-and-memcache-to-make-your-web-applications-faster/">
http://blog.kovyrin.net/2007/08/05/using-nginx-ssi-and-memcache-to-make-your-web-applications-faster/</a><br><br>Cool idea: I was the lead for the project which funded adding ESI to<br>squid3, and I'm in favor of multiple implementations.
<br><br>> My project is mongrel-esi (<a href="http://code.google.com/p/mongrel-esi">http://code.google.com/p/mongrel-esi</a>) and I'd<br>> like to use it as an example of how one can use memcache and still<br>> invalidate URLs using regular expressions.
<br>><br>> The basic idea is to store the regular expressions that have been picked up<br>> by processing an invalidation rule in memcache. Before retrieving a key<br>> from memcache checking if the key matches any of the regular expressions in
<br>> the invalidation-key and that the object being cached is older then the<br>> invalidation rule that matched. We can probably be clever about how we<br>> name the invalidation-key so that it too can expire. So long as the
<br>> invalidation-key doesn't expire before a predetermined max-expiration time<br>> that all keys get. Then we can be certain that either before we request a<br>> key, or the keys expiration time the objects marked to expire by the regex
<br>> will expire as requested by surrogate.<br>><br>> For me this realization is great because it means I can finish implementing<br>> the proxy server using memcached as the cache storage. Hopefully, it also
<br>> will be a helpful solution for others looking to access all the keys. And<br>> of course, I may be missing something important, so please correct me if I<br>> have.<br>><br>> With this solution I think it's important to get the clever part about
<br>> making sure the expire rules have an expire time on them that is greater<br>> then or equal to the expire time of all keys. There may also be some ways<br>> to name the keys based on the specific domain to have more then one
<br>> invalidation key to avoid letting it grow into a large dataset that needs to<br>> be retrieved everytime. It also means you have two hits on memcache per<br>> key lookup instead of the one before... There are probably other
<br>> techniques for solving this problem... Just hoping it makes sense and is<br>> useful to others...<br><br>I'm puzzled: why don't you store the fragments into memcache using<br>expiration times which correspond to the HTTP cache headers which
<br>wrapped them? One of the reasons I was excited about ESI was that I<br>could cache different sections of the page with different policies,<br>which meant I could drop the whole idea of purging pages from the Squid<br>
cache: the "hot" sections of the page would just time out naturally.<br>Most pages themselves would have *long* expiration times, in this<br>scenario (e.g., 1 day); a given zone on the page (e.g., "top 5 news
<br>stories") might have a very short expiration (e.g., 30 seconds).<br><br><br>Tres.<br>- --<br>===================================================================<br>Tres Seaver +1 540-429-0999 <a href="mailto:email@example.com">
firstname.lastname@example.org</a><br>Palladion Software "Excellence by Design" <a href="http://palladion.com">http://palladion.com</a><br>-----BEGIN PGP SIGNATURE-----<br>Version: GnuPG v1.4.6 (GNU/Linux)<br>Comment: Using GnuPG with Mozilla -
<a href="http://enigmail.mozdev.org">http://enigmail.mozdev.org</a><br><br>iD8DBQFG5VAi+gerLs4ltQ4RAvheAJ4zpfI6sdcpRm7ZMZDfE08DuNl9UgCeLwXI<br>ivd4FE5pyvC4isPO7/OtVcw=<br>=6TD/<br>-----END PGP SIGNATURE-----<br></blockquote>