Using the Right Verification Behavior for Your Data Wrangling Scenario — 4 Examples 5 min read

Copying and verifying source material is one of the most essential tasks of data wranglers and DITs alike. Silverstack provides various ways and many levels of detail to control and complete copy and verification tasks: It helps entry level users by reducing complexity, as well as advanced users by giving them more options. While Silverstack by default copies and verifies assets together to ensure maximum security, there are many possibilities to also have more detailed control over these activities.

In the following we would like to introduce you to these different levels of control in Silverstack with the help of 4 use cases that describe possible copy and verification scenarios.

Use Case 1: Simple & Secure — Copy and Verification Combined

Here is a potential user description of the requirements in this case:

“I want highest security and a simple configuration of the copy process. Time is not a big issue. We have to do MD5 checksums.”

This basic scenario should be known to anyone who is responsible for the backup of shooting data and has to make sure a verified copy is made.  The key requirements in this case are simplicity paired with highest security.

Use Case 1: Copy and verify together

Silverstack solves this requirement with its default verification behavior. When opening the application, the default verification behavior “Included in Copy Job” makes sure that all destinations and the source get verified during the offload. The selected default checksum method is set to MD5, as it is often required by productions (though it might limit the copy speed).

In order to meet the requirements of this first, most common use case, all the user has to do is open Silverstack, keep all default settings as they are, add his copy destinations and then start offloading. By the time the offload finishes, he has fully verified copies of his assets.

Use Case 2: Fast Dailies — Copy First, Transcode, Verify Afterwards

Here is a potential user description of the requirements in this case:

“I primarily need fast copy completion to start transcoding from the fast destination as soon as possible. Verification is important but has to be done after that.”

Whereas in use case 1 copy and verification are automatically done together, the order is different in this scenario. The primary target is to copy the files to the destination and start transcoding right away. Verification – though still important – is postponed until the primary task of copying and transcoding has been finished.

Use Case 2: Copy, transcode, verify

To support such a scenario, Silverstack allows to split the copy and verification process in two parts by creating a separate copy and verification job for one offload. In the first step, the files are only copied and checksums are created during offload. The checksums are stored in the library, but are not verified yet for the copies of the file. Second, after all files have been copied, the verification takes place.

As in this use case the user wants to transcode first after copying, he would suspend verification and resume it later after transcoding is done. He first copies the files, then starts transcoding from the destination source, and afterwards resumes verification to obtain fully verified copies.

As all tasks, including copy, verification, and transcoding are done in the same interface, the single tasks can be timed in the most efficient way to allow for an even faster dailies creation process. This is a major feature of Silverstack Lab.

Use Case 3: Fast Copy — Postpone Verification

Here is a potential user description of the requirements in this case:

“We have a high shooting ratio and only one machine dedicated to copying and verifying the files. We can’t afford the time to verify right away, but we want to have a solid verification at the end of the day, or during the night. We have enough camera media so we can wait to erase them after they have been verified successfully.”

This scenario is particularly challenging, as verification is postponed. Even though postponing verification is never recommended, special circumstances on set can require this step. Generally to any user faced with this situation, we recommend to get your hardware environment up to speed (e.g. more machines, etc.) to handle a solid verification already on set. If you are forced to postpone verification, Silverstack  supports you in the following way:

Use Case 3: Copy first, verify later

You can start all jobs with separate copy and verification and suspend the verification jobs automatically (verification behavior: Separate (per Job)). When all copy jobs are finished and the verification jobs are waiting in the queue (suspended), Silverstack allows you to shut down your machine without loosing this status. You can then re-build your setup away from set and resume all suspended verification jobs.

Use Case 4: Advanced Copy Control — Automation & Different Levels of Security

Here’s a potential user description of the requirements in this case:

“My initial copy should be verified immediately. The copies to the travel drives should then rather be completed as soon as possible. Verifying those second copies should be postponed and, in the worst case, skipped if there’s not enough time.”

This scenario is the most complex, as it requires the automation of a copy job after the initial offload is finished. It can be solved with Silverstack’s “Cascading Copy” functionality.

The “Cascading Copy” option helps the user to automate the process of offloading to one destination, and then copying from this destination to another. Combining this functionality with copy and verification in separate jobs allows the user to control every part of the process. The second copy process starts after the verification of the offload is done successfully. For this second copy, where no verification is required right away, verification can be stopped already when scheduling the cascading copy process.