replication policy update..

[Brad Fitzpatrick]

(Just a quick note to say I'm going out of town tomorrow for a week, so an
in-depth reply may not come until then, depending on net access and idle
time on my part.)


On Sun, 15 Oct 2006, Eric Lambrecht wrote:

> We're currently making some mods to mogile to help us push content
> around to different machines within our pool to deal with the fact that
> we've got some files that are extremely small and frequently accessed
> (thumbnails) and some other files that are extremely big and much less
> frequently accessed (original full size videos). Its somewhat pricey to
> get machines that can handle the gamut of stuff we store, so this is our
> attempt to split things around to different classes of machines and
> optimize how we store things.
>
> Clearly we could separate out mogile into different clusters, but having
> them all in the same cluster with the ability to tweak where things go
> really simplifies our frontend when it accesses mogile.
>
> Take a look at the description of how I want to tweak things and let me
> know if anybody has complaints/suggestions/comments. The implementation
> will fit within the new ReplicationPolicy framework and be completely
> backwards compatible.
>
> --
>
> Devices are now labelled with class preferences. If you specify
> 'foo', then that device will only store files in the 'foo' class.  A
> device can prefer to hold multiple classes or any class. If you
> specify 'foo; bar', then this device will only hold files in either
> the 'foo' or 'bar' classes. A device with no label, or with the label
> 'any', will hold files from any class, just as the existing system
> does.
>
> We use this to store files that have different traffic and filesize
> patterns on different hosts. Thumbnails are heavily hit by anonymous
> visitors and very small, so we store them on a small set of very fast
> (and expensive) machines with lots of RAM and not much storage in
> them. Full length movies are very large files and are accessed
> infreqently by a much smaller number of people who fork over their
> credit card information, so they are stored on machines with much
> denser (and slower, and cheaper) storage on them.
>
> When determining where to put a file or a replica of a file, we first
> try to store it on a machine that prefers the class of the file being
> submitted. If that fails, we try to store the file on a machine that
> will accept files of any class.
>
> Additionally, we want to influence where we store the replicas of
> files on different hosts. One reason for this is to make sure
> that multiple copies of files are stored on machines on different
> power strips. It does us no good if copy 1 and copy 2 are stored on
> two hosts that are hooked up to the same dead power supply.
>
> We specify where replicas of files should be stored by annotating the
> devices with which replica they would like to store: '1', '2', '3',
> e.t.c. A device labelled '1' would like to store the original instance
> of a file. A device labelled '3' would like to store the 3rd instance
> of a file. A device labelled '1 3 5' would like to store the 1st, 3rd,
> and 5th instance of a file. A device with no label will store any
> instance of a file.
>
> By labelling all hosts on power strip A with '1 3 5' and all hosts on
> power strip B with '2 4 6', we can be confident that if one of the
> power strips goes away, we'll still have access to our files.
>
> We can combine the two annotation methods to even further massage our
> content: 'foo 1 2' is a host that wants the first and second copy of
> 'foo' files, and 'foo 3 4' is a host that wants the third and fourth
> copies of 'foo' files.
>
> More examples:
>
> foo 1 2                 store the first and second instances
> 				of 'foo' files
> bar                     store any copy of 'bar' files
> any 1                   store the first copy of any file
> 1			store the first copy of any file
> any                     store any copy of any file
> foo 1; bar 2            store the first copy of 'foo' files
> 				and the second copy of 'bar' files
> 1 3                     store the first and third copy of any files
>
> Specifically, the algorithm works like this, when trying to determine
> where to store replica X of a file in class CLASS:
>
> 1. try to store the file in a device requesting files in class CLASS,
> replica X
> 2. try to store the file in a device requesting files in class CLASS
> 3. try to store the file in a device requesting replica X of any class
> 4. try to store the file in any device with no class or replica preferences
>
>
> (TBD: if we're at step 1 and there is only one device that we could
> store the file on, but it is dead or unreachable, do we fall through to
> #2 or do we fail and try again later?)
>
> Eric...
>
>