Regarding Consistent Hashing....

[dormando]

Ajinkya Nahar wrote:
> Hi,
> 
>  
> 
> Basically, we are using Amazon's EC2 servers as memcached servers.
> 
>  
> 
> So as and when we see an increase in traffic/load, we expect to add more 
> EC2 servers to the memcached cloud. And when the traffic/load subsides 
> (lets say after a week or so), we might remove a few servers from the 
> cloud.
> 
>  
> 
> In such a scenario, if using normal hashing, I believe we would have to 
> flush everything when adding/removing memcached servers.
> 
>  
> 
> Do you think doing this might cause the system to go in a spin and might 
> be a risky proposition, and so we should rather consider using 
> consistent hashing?
> 
>  
> 
> Or do you think this is not a big deal and we should rather use the 
> regular hashing (which is more stable)?
> 
>  
> 
> Please let me know your comments on this.
> 
>  
> 
> Regards,
> 
> Ajinkya

Are your application servers local to EC2 as well? If not, I can't 
imagine the latency being any good for the system, and it'd have to be 
over fat CPU intensive encrypted links. Memcached's not very CPU heavy, 
so I don't really see it being that great of a use for EC2. It thrives 
on fast local links with massive gobs of RAM.

Just use consistent hashing if you're worried. There should be enough 
libraries to work, or you can test the pecl client, or use libketama if 
you use php. Hard to go wrong with it.

-Dormando