filename - hashkey

sanados sanados at failure.at
Fri Jan 4 08:08:12 UTC 2008


Mark Smith wrote:
>> any chance that i could name them
>>
>> /dev1/0/000/000/0000000011_5e2bc.jpg  for example.
>>
>> so i prevent that user could browse through the server and get all files.
>> and i would also need to rewrite
>> http://127.0.0.1:7500/dev1/0/000/000/0000000011.fid
>>     
>
> It's not really designed to allow you to expose your file servers
> directly to the Internet.  By doing this you lose a lot of the
> advantages of the MogileFS system, such as the ability to gracefully
> handle disk/machine failures.  If you give the end-user a URL that
> points directly at one of your storage nodes, then you have to hope
> that node/disk never goes away.
>   
Do know about this.
Though the tracker should know about failures and should always deliver 
the right url.
I did not intend to make the URL hardcoded and the fileserver 
transparent to the outside.
Request could be handled by apache2 which runs a c-wrapper that ask one 
of the tracker where to find the files and then rewrites the request and 
hides the origin.
So i guess the httpd can hide the origin filename too.
We have too much request to let them all run through php.
That would kill our performance.
>   
>> Intern (servers) should still use this, though to deliver the files
>> outside i would love to change the url to:
>>
>> http://fs1.files.domain.com/0/000/000/0000000011.fid (those request will
>> be handled by normal http server (apache,lighttpd, ...))
>>
>> (furthermore resulting in:
>> http://fs1.files.domain.com/0/000/000/0000000011_5e2bc.jpg)
>>
>>
>> i do know that i could rewrite the domain and port in my scripts (php in
>> this case).
>> Though i don't want to give any logic to the Website.
>> (still looking forward that anyone gets the FUSE up and running)
>>     
>
> The web site should have some logic here.  If you use Perlbal in front
> then the amount of logic you have to write is vanishingly small.  Look
> it up, return it.  Then configure Perlbal to do reproxying and caching
> and you don't really have to worry about anything.
>   
Thanks for the advice on Perlbal ... didn't know about it.
Will look into it.
Seems this Perlbal does exactly what our c-wrapper on Apache was 
designed to do.

Can you point me to a __HOWTO__ how to combine mogilefs with perlbal?


> By doing it this way, you address your iteration concern as well,
> since presumably your files are stored in MogileFS by the appropriate
> filename so looking them up is easy.
>
> You also don't have to worry about nodes going down or becoming
> unavailable.  It's really just a better design pattern all around, for
> both you and your users.
>
> If you -really- insist on doing it the way you're proposing, you will
> have to hack on MogileFS to get it to save files as you are
> suggesting.  I really think it will be more work than just
> implementing the "proper" logic in your application.
>   
I do not intend to 'hack' mogilefs as it is an awesome piece of software 
already.
Just doing some work on the php extension mogileFS of capoune.net which 
is working well but i could still use more functions.
>
>   


You don't have any new information on FUSE on any chance?


More information about the mogilefs mailing list