<div>
<blockquote class="gmail_quote" style="PADDING-LEFT: 1ex; MARGIN: 0px 0px 0px 0.8ex; BORDER-LEFT: #ccc 1px solid">However, memcached semantics don't quite give you what you want.<br>Depending on whether you can reasonably get a configuration to do what
<br>you want, it might be easier to think of memcached as a bloom filter<br>than as a set in this case. That is, if you negatively cache things<br>that *aren't* part of your list, then the presence of a key will tell
<br>you for certain that a particular key is not a member, but the absence<br>of a key would mean that you don't know (or perhaps memcached *did*<br>know, but had since forgotten).</blockquote></div>
<div> </div>
<div>I had thought of doing a bloom filter on it as well. The problem here is, the membership list might change sometimes, and reading info on bloom filters, it's not really well-suited for dynamically changing lists?
</div>
<div> </div>
<blockquote class="gmail_quote" style="PADDING-LEFT: 1ex; MARGIN: 0px 0px 0px 0.8ex; BORDER-LEFT: #ccc 1px solid">You could optionally preload objects that are likely to be used if<br>you think the natural population wouldn't do it effectively (you can
<br>measure this with stats).<br></blockquote>
<div>Suppose I cache 10,000 recently-logged in members. Also, suppose 50% of traffic actually come from these users. Then, this cache would have a high hit ratio when testing for membership.</div>
<div> </div>
<div>However, what about the non-members? For instance let's say 40% of the traffic come from non-members. This would mean there'd need to be a full listing of members to check?</div>
<div> </div>
<div>Hmmm an interesting thought did just come across my mind. Let me hear your thoughts:</div>
<ul>
<li>Cache 10,000 most recently logged-in members</li>
<li>Bloom filter on the entire list</li></ul>
<div>This way, you can test for negatives (bloom filter), and if there's a positive, check the 10,000 most recently logged-in users. If that still yields nothing, then do a database query. In effect, only a small minority of checks would require a trip to the DB.
</div>
<div> </div>
<div> </div>
<div><br> </div>
<blockquote class="gmail_quote" style="PADDING-LEFT: 1ex; MARGIN: 0px 0px 0px 0.8ex; BORDER-LEFT: #ccc 1px solid">
<div></div>
<div class="Wj3C7c"><br>On Nov 4, 2007, at 23:32, J A wrote:<br><br>> I have a fairly large members list that I want to keep in memcache.<br>> What I do with this list is query it against particular user IDs to<br>
> see if they are a member of that list or not. If they are they get<br>> certain priviledges.<br>><br>> The problem is, this list has gotten to the point of saturating the<br>> PHP's memory when fetching the MySQL query the first time.
<br>><br>> Is there a way to do this more effectively, for instance,<br>> partitioning the list into separate smaller lists, grouped by time<br>> of login? I'm thinking of this, as users who have logged in in the
<br>> past 3 months are more likely to be in the list anyway.<br><br><br></div> It'd be easier to not think of it as a list if you're just testing<br>for membership. All you want to know is if a particular object is an
<br>element of a particular set. You could do this by key convention if<br>you batch populate the records.<br><br> However, memcached semantics don't quite give you what you want.<br>Depending on whether you can reasonably get a configuration to do what
<br>you want, it might be easier to think of memcached as a bloom filter<br>than as a set in this case. That is, if you negatively cache things<br>that *aren't* part of your list, then the presence of a key will tell
<br>you for certain that a particular key is not a member, but the absence<br>of a key would mean that you don't know (or perhaps memcached *did*<br>know, but had since forgotten).<br><br> You could, of course, record the status either way so as to tell the
<br>difference between not knowing and knowing whether it's a member or<br>not. This is probably best suited to your needs.<br><br> You could optionally preload objects that are likely to be used if<br>you think the natural population wouldn't do it effectively (you can
<br>measure this with stats).<br><br>--<br><font color="#888888">Dustin Sallings<br><br><br><br></font></blockquote><br>