Update: 13th January 2020; Added section: ARRIRAW and ProRes on One Mag
HDE encoding for ARRIRAW clips is a new technology that has been introduced just recently, and we received quite a few questions concerning the data management of such clips. Those questions concern the overall workflow, the general understanding how to properly handle HDE .ARX sequences in a secure backup process, and how this is covered in our software products for data management.
Our data management software Silverstack is capable of securely backing up Codex HDE ARRIRAW .ARX sequences since version 6.5, and we want to clarify the bits and pieces that come with this.
Let’s start with quickly sorting out the purpose of HDE and what it is, then take a look at the challenges that come with it in a backup context, and finally take a concrete look on how Silverstack handles HDE .ARX sequences.
What is HDE and What’s its purpose?
HDE stands for High Density Encoding and is a new technology for RAW workflows by Codex. It is a completely lossless encoding technique that allows to reduce file size by around 40%. Currently it can be used explicitly with ARRIRAW images of any resolution.
The drastic reduction in file size helps to decrease necessary storage space, transfer times, and therefore cost. As the images can be decoded again to a full bit-perfect representation of the original RAW file there’s no quality loss involved at all.
Challenges When Working with HDE
In high-density encoded (HDE) ARRIRAW clip sequences the files for each frame have a “.ARX” file extension instead of an “.ARI” extension. When enabling HDE with the Codex Device Manager you will see a representation of the content as a .ARX file sequence in the digital volume that appears when mounting the camera mag. However those file representations have a size of 0KB. That is because the encoding to HDE .ARX files will happen during offload.
This has the consequence that .ARX file sequences cannot be copied with macOS Finder, but only with a copy application that explicitly supports ARRIRAW HDE files, like for example Pomfort Silverstack. The encoding process takes place on the fly within Codex’s virtual file system driver and will create HDE-encoded .ARX sequences, now with a valid file size, on the destination.
Please be aware that you can actually trigger an invalid copy of a .ARX sequence in Finder. This copy process will be disproportionally fast and will leave you with invalid, empty .ARX files on the destination drive. So with .ARX sequences you will need to use a backup or offload application that understands this specific issue in order to avoid data loss!
Working with .ARX Sequences in Silverstack for Secure Backup
The backup process in Silverstack is not different from handling regular, non-HDE, .ARI sequences.
After setting up the Codex VFS (Virtual File System) source to present .ARX sequences, the drive can be selected as usual in Silverstack and the offload wizard will detect it automatically as an ARRIRAW sequence.
At this point it makes sense to understand that the displayed file size can only be projected as it cannot be identified at that point. It cannot be pre-determined because the final HDE size depends on the image content. After the copy process, the file sizes will be updated with the appropriate sizes from the HDE-encoded .ARX frames.
Now you can start a regular multi destination backup process and ingest the clips into the library. All regular functionality in Silverstack (like e.g. playback) is now available as for other camera formats and regular .ARI sequences.
After the secure backup process is finished, the HDE-encoded .ARX sequences are located on the destination drive, now with their final HDE-encoded file size.
Also the Silverstack Offload Manager supports HDE ARRIRAW .ARX sequences since version 1.1.
ARRIRAW and ProRes on One Mag
To understand the way HDE is currently intended to work becomes very important if ARRIRAW and ProRes clips have been recorded to the same mag. Due to the nature of HDE, only the ARRIRAW clips will be present on the HDE volume.
For example when working with Alexa Mini LF and the dedicated HDE volume that shows up once HDE is enabled, it is crucially important to offload both, the regular volume and the HDE volume. The ProRes clips are not present on the HDE volume and will be lost if only this volume should be offloaded. So make sure to check recording formats and which clips you would expect.