FlashNet Amalgamation Workflow
The FlashNet Amalgamation Workflow optimises archiving performance by consolidating multiple small file archive requests into a single archive process.
Tape archiving technology is based on placing a required tape into a drive, and then locating the correct position on the tape before content can be written to that tape. This process naturally requires a finite period of time, and is required whether the content to be stored is large or small. The larger the file, the higher the ratio between tape write and tape positioning, the more efficient the archiving process.
However, while many media files do require large amounts of storage, projects will often also require the storing of much smaller files: JPEGs, XMLs, and so on. If each such file is archived separately then the process of writing to tape becomes much less efficient.
FlashNet Amalgamation resolves this problem, by consolidating multiple small files into one set of files, all of which will be archived at the same time without any additional tape positioning.
Amalgamation runs directly within FlashNet. This means that two simple configuration settings are all that are required for set-up:
- A size threshold: Files are accumulated as a single job on the FlashNet queue until the size threshold is reached, when the job is submitted and processed
- A time threshold: If a specified duration is reached (for instance, 20 minutes) and the accumulated files have not reached the size threshold, launch the job anyway
Set up each Group as Needed
The accumulation size threshold can be configured separately for each tape group and disk volume.
Though overall throughput of jobs is accelerated significantly, the archiving of a single small file will seem to take a little longer – it needs to wait for the amalgamation process to complete before the archiving is done.
However, the FlashNet Queue Monitor enables the progress of amalgamation jobs to be easily tracked. The Monitor presents key information on the job, such as:
- How much time has elapsed, enabling comparing to the time threshold
- The current status of the job
- What percentage of the threshold has been reached
- The various files that are in the amalgamation job