mogilefs usage and performance

[dormando]

> 
> If configured with 'memcached before get_paths':
> 
> perlbal->mod_perl->memcached->perlbal->mogstored: 250req/sec
> 
> mod_perl->memcached (no real file contents, just get x-reproxy): 1500
> req/sec (memcached works fast, not an invention)
> 
> I dont want to store actual file contents in memcached, so didn't test it.
> So, (1500req/sec of mod_perl/memcached + 1170 req/sec mogstored) via
> perlbal = 250req/sec overall performance for 10kb file downloading in
> 10 concurrent connections.

Paths don't usually change that often, so I just give them a five minute 
timeout. Very occasionally we'd see a blip, but it would correct itself. 
Also if you use perlbal's ability to X-Reproxy to a list of paths it'll 
almost always work even if one of the paths dies.

So what you're saying is that:

Just directly from mod_perl->memcached to fetch a path but not actually 
reproxying runs at 1500 requests/sec?

How're you getting the requests/sec to mogstored? Directly querying it?

Then putting the whole chain together ends up with 250req/sec? Are you 
monitoring the box during your 'ab' tests and seeing if any one 
component starts nailing IO, page thrashing, or maxing out CPU?

Also, is your perlbal -> mod_perl step using backend keepalives? Not 
having that working adds a lot of overhead to perlbal.

Unfortunately I have over 24 perlbal instances spreading out my load 
balancing load, so I don't have anything doing more than 150 
requests/sec to examine :)

-Dormando