OT using crc32 as key for objects

Steven Grimm sgrimm at facebook.com
Fri Oct 6 08:42:31 UTC 2006


You could use SHA-1 or even MD5 and the chances of a collision would be 
infinitesimal. That would be much more secure (against collisions) than 
concatenating CRC32s.

-Steve


Jed Reynolds wrote:
> Sorry if this is too far off topic, but since memcached is essentially 
> a very large hash table, discussion of hashing might be appropriate here.
>
> When caching query results, mysql uses the query as the key in its 
> query cache. Since a sql query is not a legal memcache key, nor is the 
> base64 of a query legal (often too long), the hash of the query is the 
> next logical choice.
>
> I'd like to avoid using truncated MD5 or SHA1 hashes of queries as 
> that seems overkill. Using a crc32 of a query is more risky but still 
> improbable to collide within a hundred thousand queries, but I don't 
> know the math that well. It would seem to me that concatenating the 
> crc32s of a query and a salted version of the query would very likely 
> yield a pretty unique key for a sql query. By the same logic, 
> concatenating the crc32 of the base64 encoding of a query seems 
> reasonably safe, too.
>
> Any opinions?
>
> Jed Reynolds



More information about the memcached mailing list