mogilefs usage and performance
dormando at rydia.net
Fri Aug 11 19:41:19 UTC 2006
> If configured with 'memcached before get_paths':
> perlbal->mod_perl->memcached->perlbal->mogstored: 250req/sec
> mod_perl->memcached (no real file contents, just get x-reproxy): 1500
> req/sec (memcached works fast, not an invention)
> I dont want to store actual file contents in memcached, so didn't test it.
> So, (1500req/sec of mod_perl/memcached + 1170 req/sec mogstored) via
> perlbal = 250req/sec overall performance for 10kb file downloading in
> 10 concurrent connections.
Paths don't usually change that often, so I just give them a five minute
timeout. Very occasionally we'd see a blip, but it would correct itself.
Also if you use perlbal's ability to X-Reproxy to a list of paths it'll
almost always work even if one of the paths dies.
So what you're saying is that:
Just directly from mod_perl->memcached to fetch a path but not actually
reproxying runs at 1500 requests/sec?
How're you getting the requests/sec to mogstored? Directly querying it?
Then putting the whole chain together ends up with 250req/sec? Are you
monitoring the box during your 'ab' tests and seeing if any one
component starts nailing IO, page thrashing, or maxing out CPU?
Also, is your perlbal -> mod_perl step using backend keepalives? Not
having that working adds a lot of overhead to perlbal.
Unfortunately I have over 24 perlbal instances spreading out my load
balancing load, so I don't have anything doing more than 150
requests/sec to examine :)
More information about the mogilefs