We are living in interesting times for digital video preservation (we are living in interesting times for other reasons too, of course).
For many years digital video preservation has been a confusing area of audiovisual archiving. To date there is no settled standard that organisations, institutions and individuals can unilaterally adopt. As Peter Bubestinger-Steindl argues, ‘no matter whom you ask [about which format to use] you will get different answers. The answers might be correct, but they might not be the right solution for your use-cases.’
The aim of CELLAR is to standardise three lossless open-source audiovisual formats – Matroska, FFV1 and FLAC – for use in archival environments and transmission.
To date the evolution of video formats has largely been driven by broadcast, production and consumer markets. The development of video formats for long term archival use has been a secondary consideration.
The work on the Matroska container, FFV1 video codec and FLAC audio codec is therefore hugely significant because they have, essentially, been developed by audiovisual archivists for audiovisual archivists.
Other key points to note is that Matroska, FFV1 and FLAC are:
2. Lossless compression. Simply put, lossless compression makes digital video files easier to store and transmit: file size is decreased without damaging integrity.
Managing large file sizes has been a major practical glitch that has held back digital video preservation in the past. The development of effective lossless compression for digital video is therefore a huge advance.
Here they explain that ‘the Matroska wrapper is organized into top-level sectional elements for the storage of attachments, chapter information, metadata and tags, indexes, track descriptions, and encoding audiovisual data.’
Each of these elements has a checksum associated with it, which means that each part of the file can be checked at a granular level. If there is an error in the track description, for example, this can be specifically dealt with. Matroska enables digital video preservation to become targeted and focused, a very useful thing given the complexity of video files.
It is also possible to embed technical and descriptive metadata within the Matroska container, rather than alongside it in a sidecar document.
This will no doubt make Matroska attractive to archivists who dream of a container-format that can store additional technical and contextual information.
Yet as Peter B. Hermann Lewetz and Marion Jaks argue, ‘keeping everything in one video-file increases the required complexity of the container, the video-codec – or both. It might look “simpler” to have just one file, but the choice of tools available to handle the embedded data is, by design, greatly reduced. In practice this means it can be harder (or even impossible) to view or edit the embedded data. Especially, if the programs used to create the file were rare or proprietary.’
While it would seem that embedding metadata in the container file is currently not wholly practical, developing tools and systems that can handle such information must surely be a priority as we think about the long term preservation of video files.
FFV1 and FLAC are also designed with archival use in mind. FFV1, Rice and Blewer explain, uses lossless compression and contains ‘self-description, fixity, and error resilience mechanisms.’ ‘FLAC is a lossless audio codec that features embedded checksums per audio frame and can store embedded metadata in the source WAVE file.’
Milestones for Digital Video Preservation
By the end of 2016 the CELLAR working group will have submitted standard and information specifications to the Internet Engineering Steering Group (IESG) for Matroska, FFV1, FLAC and EBML, the binary XML format the Matroska container is based on.
Outside of CELLAR’s activities there are further encouraging signs of adoption among the audio visual preservation community.
Austrian-based media archive management company NOA, for example, ‘chose to provide FFV1 as a native option for encoding within its FrameLector products, as they see it has many benefits as a lossless, open source file format that is easy to use, has low computational overheads and is growing in adoption.’
We’ll be keeping an eye on how the standardisation of Matroska, FFV1 and FLAC unfolds in 2017. We will also share our experiences with the format, including whether there is increased demand and uptake among our customer base.
In the last Greatbear article we quoted sage advice from the International Association of Audiovisual Archivists: ‘Optimal preservation measures are always a compromise between many, often conflicting parameters.’ [1]
While this statement is true in general for many different multi-format collections, the issue of compromise and conflicting parameters becomes especially apparent with the preservation of digitized and born-digital video. The reasons for this are complex, and we shall outline why below.
Lack of standards (or are there too many formats?)
Carl Fleischhauer writes, reflecting on the Federal Agencies Digitization Guidelines Initiative (FADGI) research exploring Digital File Formats for Videotape Reformatting (2014), ‘practices and technology for video reformatting are still emergent, and there are many schools of thought. Beyond the variation in practice, an archive’s choice may also depend on the types of video they wish to reformat.’ [2]
We have written in depth on this blog about the labour intensity of digital information management in relation to reformatting and migration processes (which are of course Greatbear’s bread and butter). We have also discussed how the lack of settled standards tends to make preservation decisions radically provisional.
In contrast, we have written about default standards that have emerged over time through common use and wide adoption, highlighting how parsimonious, non-interventionist approaches may be more practical in the long term.
The problem for those charged with preserving video (as opposed to digital audio or images) is that ‘video, however, is not only relatively more complex but also offers more opportunities for mixing and matching. The various uncompressed-video bitstream encodings, for example, may be wrapped in AVI, QuickTime, Matroska, and MXF.’ [3]
What then, is this ‘mixing and matching’ all about?
It refers to all the possible combinations of bitsteam encodings (‘codecs’) and ‘wrappers’ that are available as target formats for digital video files. Want to mix your JPEG2000 – Lossless with your MXF, or ffv1 with your AVI? Well, go ahead!
What then is the difference between a codec and wrapper?.
As the FADGI report states: ‘Wrappers are distinct from encodings and typically play a different role in a preservation context.’ [4]
The wrapper or ‘file envelope’ stores key information about the technical life or structural properties of the digital object. Such information is essential for long term preservation because it helps to identify, contextualize and outline the significant properties of the digital object.
Information stored in wrappers can include:
Content (number of video streams, length of frames),
Context (title of object, who created it, description of contents, re-formatting history),
Video rendering (Width, Height and Bit-depth, Colour Model within a given Colour Space, Pixel Aspect Ratio, Frame Rate and Compression Type, Compression Ratio and Codec),
Audio Rendering – Bit depth and Sample Rate, Bit Rate and compression codec, type of uncompressed sampling.
Codecs, on the other hand, define the parameters of the captured video signal. They are a ‘set of rules which defines how the data is encoded and packaged,’ [5] encompassing Width, Height and Bit-depth, Colour Model within a given Colour Space, Pixel Aspect Ratio and Frame Rate; the bit depth and sample rate and bit rate of the audio.
Although the wrapper is distinct from the encoded file, the encoded file cannot be read without its wrapper. The digital video file, then, comprises of wrapper and at least one codec, often two, to account for audio and images, as this illustration from AV Preserve makes clear.
Diagram taken from AV Preserve’s A Primer on Codecs for Moving Image and Sound Archives
Pick and mix complexity
Why then, are there so many possible combinations of wrappers and codecs for video files, and why has a settled standard not been agreed upon?
Fleischhauer at The Signal does an excellent job outlining the different preferences within practitioner communities, in particular relating to the adoption of ‘open’ and commercial/ proprietary formats.
Compellingly, he articulates a geopolitical divergence between these two camps, with those based in the US allegedly opting for commercial formats, and those in Europe opting for ‘open.’ This observation is all the more surprising because of the advice in FADGI’s Creating and Archiving Born Digital Video: ‘choose formats that are open and non-proprietary. Non-proprietary formats are less likely to change dramatically without user input, be pulled from the marketplace or have patent or licensing restrictions.’ [6]
One answer to the question: why so many different formats can be explained by different approaches to information management in this information-driven economy. The combination of competition and innovation results in a proliferation of open source and their proprietary doubles (or triplets, quadruples, etc) that are constantly evolving in response to market ‘demand’.
Impact of the Broadcast Industry
An important area to highlight driving change in this area is the role of the broadcast industry.
Format selections in this sector have a profound impact on the creation of digital video files that will later become digital archive objects.
In the world of video, Kummer et al explain in an article in the IASA journal, ‘a codec’s suitability for use in production often dictates the chosen archive format, especially for public broadcasting companies who, by their very nature, focus on the level of productivity of the archive.’ [7] Broadcast production companies create content that needs to be able to be retrieved, often in targeted segments, with ease and accuracy. They approach the creation of digital video objects differently to how an archivist would, who would be concerned with maintaining file integrity rather ensuring the source material’s productivity.
Furthermore, production contexts in the broadcast world have a very short life span: ‘a sustainable archiving decision will have to made again in ten years’ time, since the life cycle of a production system tends to be between 3 and 5 years, and the production formats prevalent at that time may well be different to those in use now.’ [8]
Take, for example, H.264/ AVC ‘by far the most ubiquitous video coding standard to date. It will remain so probably until 2015 when volume production and infrastructure changes enable a major shift to H.265/ HEVC […] H.264/ AVC has played a key role in enabling internet video, mobile services, OTT services, IPTV and HDTV. H.264/ AVC is a mandatory format for Blu-ray players and is used by most internet streaming sites including Vimeo, youtube and iTunes. It is also used in Adobe Flash Player and Microsoft Silverlight and it has also been adopted for HDTV cable, satellite, and terrestrial broadcasting,’ writes David Bull in his book Communicating Pictures.
HEVC, which is ‘poised to make a major impact on the video industry […] offers to the potential for up to 50% compression efficiency improvement over AVC.’ Furthermore, HEVC has a ‘specific focus on bit rate reduction for increased video resolutions and on support for parallel processing as well as loss resilience and ease if integration with appropriate transport mechanisms.’ [9]
Increased compression
The development of codecs for use in the broadcast industry deploy increasingly sophisticated compression that reduce bit rate but retain image quality. As AV Preserve explain in their codec primer paper, ‘we can think of compression as a second encoding process, taking coded information and transferring or constraining it to a different, generally more efficient code.’ [10]
The explosion of mobile, video data in the current media moment is one of the main reasons why sophisticated compression codecs are being developed. This should not pose any particular problems for the audiovisual archivist per se—if a file is ‘born’ with high degrees of compression the authenticity of the file should not ideally, be compromised in subsequent migrations.
Nevertheless, the influence of the broadcast industry tells us a lot about the types of files that will be entering the archive in the next 10-20 years. On a perceptual level, we might note an endearing irony: the rise of super HD and ultra HD goes hand in hand with increased compression applied to the captured signal. While compression cannot, necessarily, be understood as a simple ‘taking away’ of data, its increased use in ubiquitous media environments underlines how the perception of high definition is engineered in very specific ways, and this engineering does not automatically correlate with capturing more, or better quality, data.
Like error correction that we have discussed elsewhere on the blog, it is often the anticipation of malfunction that is factored into the design of digital media objects. These, in turn, create the impression of smooth, continuous playback—despite the chaos operating under the surface. The greater clarity of the visual image, the more the signal has been squeezed and manipulated so that it can be transmitted with speed and accuracy. [11]
MXF
Staying with the broadcast world, we will finish this article by focussing on the MXF wrapper that was ‘specifically designed to aid interoperability and interchange between different vendor systems, especially within the media and entertainment production communities. [MXF] allows different variations of files to be created for specific production environments and can act as a wrapper for metadata & other types of associated data including complex timecode, closed captions and multiple audio tracks.’ [12]
The Presto Centre’s latest TechWatch report (December 2014) asserts ‘it is very rare to meet a workflow provider that isn’t committed to using MXF,’ making it ‘the exchange format of choice.’ [13]
We can see such adoption in action with the Digital Production Partnership’s AS-11 standard, which came into operation October 2014 to streamline digital file-based workflows in the UK broadcast industry.
While the FADGI reports highlights the instability of archival practices for video, the Presto Centre argue that practices are ‘currently in a state of evolution rather than revolution, and that changes are arriving step-by-step rather than with new technologies.’
They also highlight the key role of the broadcast industry as future archival ‘content producers,’ and the necessity of developing technical processes that can be complimentary for both sectors: ‘we need to look towards a world where archiving is more closely coupled to the content production process, rather than being a post-process, and this is something that is not yet being considered.’ [14]
The world of archiving and reformatting digital video is undoubtedly complex. As the quote used at the beginning of the article states, any decision can only ever be a compromise that takes into account organizational capacities and available resources.
What is positive is the amount of research openly available that can empower people with the basics, or help them to delve into the technical depths of codecs and wrappers if so desired. We hope this article will give you access to many of the interesting resources available and some key issues.
As ever, if you have a video digitization project you need to discuss, contact us—we are happy to help!
References:
[1] IASA Technical Committee (2014) Handling and Storage of Audio and Video Carriers, 6.
[2] Carl Fleischhauer, ‘Comparing Formats for Video Digitization.’http://blogs.loc.gov/digitalpreservation/2014/12/comparing-formats-for-video-digitization/.
[3] Federal Agencies Digital Guidelines Initiative (FADGI), Digital File Formats for Videotape Reformatting Part 5. Narrative and Summary Tables. http://www.digitizationguidelines.gov/guidelines/FADGI_VideoReFormatCompare_pt5_20141202.pdf, 4.
[4] FADGI, Digital File Formats for Videotape, 4.
[5] AV Preserve (2010) A Primer on Codecs for Moving Image and Sound Archives& 10 Recommendations for Codec Selection and Management, www.avpreserve.com/wp-content/…/04/AVPS_Codec_Primer.pdf, 1.
[6] FADGI (2014) Creating and Archiving Born Digital Video Part III. High Level Recommended Practices, http://www.digitizationguidelines.gov/guidelines/FADGI_BDV_p3_20141202.pdf, 24.
[7] Jean-Christophe Kummer, Peter Kuhnle and Sebastian Gabler (2015) ‘Broadcast Archives: Between Productivity and Preservation’, IASA Journal, vol 44, 35.
[8] Kummer et al, ‘Broadcast Archives: Between Productivity and Preservation,’ 38.
[9] David Bull (2014) Communicating Pictures, Academic Press, 435-437.
[10] Av Preserve, A Primer on Codecs for Moving Image and Sound Archives, 2.
[11] For more reflections on compression, check out this fascinating talk from software theorist Alexander Galloway. The more practically bent can download and play with VISTRA, a video compression demonstrator developed at the University of Bristol ‘which provides an interactive overview of the some of the key principles of image and video compression.
[12] ‘FADGI, Digital File Formats for Videotape, 11.
[13] Presto Centre, AV Digitisation and Digital Preservation TechWatch Report #3, https://www.prestocentre.org/, 9.
[14] Presto Centre, AV Digitisation and Digital Preservation TechWatch Report #3, 10-11.
In archiving, the simple truth is formats matter. If you want the best quality recording, that not only sounds good but has a strong chance of surviving over time, it needs to be recorded on an appropriate format.
Most of us, however, do not have specialised knowledge of recording technologies and use what is immediately available. Often we record things within limited budgets, and need to make the most of our resources. We are keen to document what’s happening in front of us, rather than create something that will necessarily be accessible many years from now.
At the Great Bear we often receive people’s personal archives on a variety of magnetic tape. Not all of these tapes, although certainly made to ensure memories were recorded, were done on the best quality formats.
Recently we migrated a recording of a wedding service from 1970 made on C-120 audio cassette.
Image taken using a smart phone @ 72 dpi resolution
C60 and C90 tapes are probably familiar to most readers of this blog, but the C-120 was never widely adopted by markets or manufacturers because of its lesser recording quality. The C-120 tape records for an hour each side, and uses thinner tape than its C90 and C60 counterparts. This means the tape is more fragile, and is less likely to produce optimum recordings. Thinner tapes is also more likely to suffer from ‘print-through‘ echo.
As the Nakamichi 680 tape manual, which is pretty much consulted as the bible on all matters tape in the Great Bear studio, insists:
‘Choosing a high quality recording tape is extremely important. A sophisticated cassette deck, like the 680, cannot be expected to deliver superior performance with inferior tapes. The numerous brands and types of blank cassettes on the market vary not only in the consistency of the tape coating, but in the degree of mechanical precision as well. The performance of an otherwise excellent tape is often marred by a poor housing, which can result in skewing and other unsteady tape travel conditions.’
The manual goes on to stress ‘Nakamichi does not recommend the use of C-120 or ferrichrome cassettes under any circumstances.’ Strong words indeed!
It is usually possible to playback most of the tape we receive, but a far greater risk is taken when recordings are made on fragile or low quality formats. The question that has to be thought through when making recordings is: what are you making them for? If they are meant to be a long term record of events, careful consideration of the quality of the recording format used needs to be made to ensure they have the greatest chance of survival.
Such wisdom seems easy to grasp in retrospect, but what about contemporary personal archives that are increasingly ‘born digital’?
A digital equivalent of the C-120 tape would be the MP3 format. While MP3 files are easier to store, duplicate and move across digital locations, they offer substantially less quality than larger, uncompressed audio files, such as WAVs or AIFFs. The current recommended archival standard for recording digital audio is 24 bit/ 48 kHz, so if you are making new recordings, or migrating analogue tapes to digital formats, it is a good idea to ensure they are sampled at this rate
‘in the midst of an amazing revolution in computer technology, there is a near total lack of systems designed with digital preservation in mind. Instead, we have technology seemingly designed to work against digital preservation. The biggest single issue is that we are encouraged to scatter content so broadly among so many different and changing services that it practically guarantees loss. We need programs to automatically capture, organize and keep our content securely under our control.’
The issue of format quality also comes to the fore with the type of everyday records we make of our digital lives. The images and video footage we take on smart phones, for example, are often low resolution, and most people enjoy the flexibility of compressed audio files. In ten years time will the records of our digital lives look pixelated and poor quality, despite the ubiquity of high tech capture devices used to record and share them? Of course, these are all speculations, and as time goes on new technologies may emerge that focus on digital restoration, as well as preservation.
Ultimately, across analogue and digital technologies the archival principles are the same: use the best quality formats and it is far more likely you will make recordings that people many years from now can access.
Screenshot of software encoding a file to MP3 used at the Great Bear
After we have migrated your analogue or digital tape to a digital file, we offer a range of delivery formats.
For video, using the International Association of Sound & Audiovisual Archives Guidelines for the Preservation of Video Recordings, as our guide, we deliver FFV1 lossless files or 10-bit uncompressed video files in .mkv or QuickTime compatible .mov containers. We add viewing files as H264 encoded .mp4 files or DVD. We’ll also produce any other digital video files, according to your needs, such as AVI in any codec; any MacOS, Windows or GNU/Linux filesystem (HFS+, NTFS or EXT3.
For audio we offer Broadcast WAV (B-WAV) files on hard drive or optical media (CD) at 16 bit/44.1 kHz (commonly used for CDs) or 24 bit/96 kHz (which is the minimum recommended archival standard) and anything up to 24 bit / 192 kHz. We can also deliver access copies on CD or MP3 (that you could upload to the internet, or listen to on an ipod, for example).
Why are there so many digital file types and what distinguishes them from each other?
The main difference that is important to grasp is between an uncompressed digital file and a compressed one.
On the JISC Digital Media website, they describe uncompressed audio files as follows:
‘Uncompressed audio files are the most accurate digital representation of a soundwave, but can also be the most resource-intensive method of recording and storing digital audio, both in terms of storage and management. Their accuracy makes them suitable for archiving and delivering audio at high resolution, and working with audio at a professional level, and they are the “master” audio format of choice.’
Why uncompressed?
As a Greatbear client you may wonder why you need a large, uncompressed digital file if you only want to listen to your old analogue and digital tapes again. The simple answer is: we live in an age where information is dynamic rather static. An uncompressed digital recording captured at a high bit and kHz rate is the most stable media format you can store your data on. Technology is always changing and evolving, and not all types of digital files that are common today are safe from obsolescence.
It is important to consider questions of accessibility not only for the present moment, but also for the future. There may come a time when your digitised audio or video file needs to be migrated again, so that it can be played back on whatever device has become ‘the latest thing’ in a market driven by perpetual innovation. It is essential that you have access to the best quality digital file possible, should you need to transport your data in ten, fifteen or twenty years from now.
Compression and compromise?
Uncompressed digital files are sound and vision captured in their purest, ‘most accurate’ form. Parts of the original recording are not lost when the file is converted or saved. When a digital file is saved to a compressed, lossy format, some of its information is lost. Lossy compression eliminates ‘unnecessary’ bits of information, tailoring the file so that it is smaller. You can’t get the original file back after it has been compressed so you can’t use this sort of compression for anything that needs to be reproduced exactly. However it is possible to compress files to a lossless format, which does enable you to recreate the original file exactly.
In our day to day lives however we encounter far more compressed digital information than uncompressed.
There would be no HD TV, no satellite TV channels and no ipods/ MP3 players without compressed digital files. The main point of compression is to make these services affordable. It would be incredibly expensive, and it would take up so much data space, if the digital files that were streamed to televisions were uncompressed.
‘Every so often I’ll get the proper CD version of an album I’ve fallen in love with as a download, and I’ll get a rude shock when confronted by the sense of dimension and spatiality in the music’s layers, the sculpted force of the drums, the sheer vividness of the sound. The difference between CD and MP3 is similar to that between “not from concentrate” orange juice and juice that’s been reconstituted from concentrate. (In this analogy vinyl would be ‘freshly squeezed, perhaps). Converting music to MP3 is a bit like the concentration process, and its done for much the same reason: it’s much cheaper to transport concentrate because without the water it takes up a lot loss volume and it weighs a lot less. But we can all taste the difference.’
As a society we are slowly coming to terms with the double challenge of hyper consumption and conservation thrown up by the mainstreaming of digital technology. Part of that challenge is to understand what happens to the digital data we use when we click ‘save as,’ or knowing what decisions need to be made about data we want to keep because it is important to us as individuals, or to wider society.
At Greatbear we can deliver digital files in compressed and uncompressed formats, and are happy to offer a free consultation should you need it to decide what to do with your tape based digital and analogue media.
