We are living in interesting times for digital video preservation (we are living in interesting times for other reasons too, of course).
For many years digital video preservation has been a confusing area of audiovisual archiving. To date there is no settled standard that organisations, institutions and individuals can unilaterally adopt. As Peter Bubestinger-Steindl argues, ‘no matter whom you ask [about which format to use] you will get different answers. The answers might be correct, but they might not be the right solution for your use-cases.’
The aim of CELLAR is to standardise three lossless open-source audiovisual formats – Matroska, FFV1 and FLAC – for use in archival environments and transmission.
To date the evolution of video formats has largely been driven by broadcast, production and consumer markets. The development of video formats for long term archival use has been a secondary consideration.
The work on the Matroska container, FFV1 video codec and FLAC audio codec is therefore hugely significant because they have, essentially, been developed by audiovisual archivists for audiovisual archivists.
Other key points to note is that Matroska, FFV1 and FLAC are:
2. Lossless compression. Simply put, lossless compression makes digital video files easier to store and transmit: file size is decreased without damaging integrity.
Managing large file sizes has been a major practical glitch that has held back digital video preservation in the past. The development of effective lossless compression for digital video is therefore a huge advance.
Here they explain that ‘the Matroska wrapper is organized into top-level sectional elements for the storage of attachments, chapter information, metadata and tags, indexes, track descriptions, and encoding audiovisual data.’
Each of these elements has a checksum associated with it, which means that each part of the file can be checked at a granular level. If there is an error in the track description, for example, this can be specifically dealt with. Matroska enables digital video preservation to become targeted and focused, a very useful thing given the complexity of video files.
It is also possible to embed technical and descriptive metadata within the Matroska container, rather than alongside it in a sidecar document.
This will no doubt make Matroska attractive to archivists who dream of a container-format that can store additional technical and contextual information.
Yet as Peter B. Hermann Lewetz and Marion Jaks argue, ‘keeping everything in one video-file increases the required complexity of the container, the video-codec – or both. It might look “simpler” to have just one file, but the choice of tools available to handle the embedded data is, by design, greatly reduced. In practice this means it can be harder (or even impossible) to view or edit the embedded data. Especially, if the programs used to create the file were rare or proprietary.’
While it would seem that embedding metadata in the container file is currently not wholly practical, developing tools and systems that can handle such information must surely be a priority as we think about the long term preservation of video files.
FFV1 and FLAC are also designed with archival use in mind. FFV1, Rice and Blewer explain, uses lossless compression and contains ‘self-description, fixity, and error resilience mechanisms.’ ‘FLAC is a lossless audio codec that features embedded checksums per audio frame and can store embedded metadata in the source WAVE file.’
Milestones for Digital Video Preservation
By the end of 2016 the CELLAR working group will have submitted standard and information specifications to the Internet Engineering Steering Group (IESG) for Matroska, FFV1, FLAC and EBML, the binary XML format the Matroska container is based on.
Outside of CELLAR’s activities there are further encouraging signs of adoption among the audio visual preservation community.
Austrian-based media archive management company NOA, for example, ‘chose to provide FFV1 as a native option for encoding within its FrameLector products, as they see it has many benefits as a lossless, open source file format that is easy to use, has low computational overheads and is growing in adoption.’
We’ll be keeping an eye on how the standardisation of Matroska, FFV1 and FLAC unfolds in 2017. We will also share our experiences with the format, including whether there is increased demand and uptake among our customer base.
On a recent trip to one of Britain’s most significant community archives, I was lucky enough to watch a rare piece of digitised video footage from the late 1970s.
As the footage played it raised many questions in my mind: who shot it originally? What format was it originally created on? How was it edited? How was it distributed? What was the ‘life’ of the artefact after it ceased to actively circulate within communities of interest/ use? How and who digitised it?
As someone familiar with the grain of video images, I could make an educated guess about the format. I also made other assumptions about the video. I imagined there was a limited amount of tape available to capture the live events, for example, because a number of still images were used to sustain the rolling audio footage. This was unlikely to be an aesthetic decision given that the aim of the video was to document a historic event. I could be wrong about this, of course.
When I asked the archivist the questions flitting through my mind she had no answers. She knew who the donor of the digital copy was, but nothing about the file’s significant properties. Nor was this important information included in the artefact’s record.
This struck me as a hugely significant problem with the status of digitised material – and especially perhaps video – in mixed-content archives where the specificities of AV content are not accounted for.
Due to the haphazard and hand-to-mouth way mixed-content archives have acquired digital items, it seems more than likely this situation is the rule rather than the exception: acquired bit by bit (no pun intended), maintaining access is often privileged over preserving the context and context of the digitised video artefact.
As a researcher I was able to access the video footage, and this of course is better than nothing.
Yet I was viewing the item in an ahistoric black hole. It was profoundly decontextualised; an artefact extracted to its most barest of essences.
Standard instabilities
This is not in any way a criticism of the archive in question. In fact, this situation is wholly understandable given that digital video are examples of ‘media formats that exist in crisis.’
Video digitisation remains a complex and unstable area of digital preservation. It is, as we have written elsewhere on this blog, the final frontier of audiovisual archiving. This seems particularly true within the UK context where there is currently no systematic plan to digitise video collections, unlike film and audio.
There are signs, however, that file-format stabilities are emerging. The No Time to Wait: Standardizing FFV1 & Matroska for Preservation symposium (Berlin, July 2016) brought together software developers and archivists who want to make the shared dream of an open source lossless video standard, fit for archival purpose, a reality.
It seems like the very best minds are working together to solve this problem, so Greatbear are quietly optimistic that a workable, open source standard for video digital preservation is in reach in the not too distant future.
Metadata
Yet as my experience in the archive makes clear, the challenge of video digitisation is not about file format alone.
There is a pressing need to think very carefully about the kind of metadata and other contextual material that need to be preserved within and alongside the digitised file.
Due to limited funding and dwindling technical capacity, there is likely to be only one opportunity to transfer material currently recorded on magnetic tape. This means that in 2016 there really can be no dress rehearsal for your video digitisation plans.
‘Digitization is preservation…For audiovisual materials. And this bears repeating over and over because the anti-digitization voice is much stronger and generally doesn’t include any nuance in regards to media type because the assumption is towards paper. When we speak about digitization for audio and video, we now are not speaking about simple online access. We are speaking about the continued viability, about the persistence and the existence of the media content.’
What information will future generations need to understand the digitised archive materials we produce?
An important point to reckon with here is that not all media are the same. The affordances of particular technologies, within specific historical contexts, have enabled new forms of community and communicative practice to emerge. Media are also disruptive (if not deterministic) – they influence how we see the world and what we can do.
On this blog, for example, Peter Sachs Collopy discussed how porta-pak technology enabled video artists and activists in the late 1960s/ early 1970s to document and re-play events quickly.
Les prostituées documents a wave of church occupations by feminist activists in France.
The film demonstrates how women used emergent videotape technology to transmit footage recorded within the church onto TV screens positioned outside. Here videotape technology, and in particular its capacity to broadcast uni-directional messages, was used to protect and project the integrity of the group’s political statements. Video, in this sense, was an important tool that enabled the women – many of whom were prostitutes and therefore without a voice in French society – to ‘speak’.
Peter’s interview and Les prostituées de Lyon parlent are specific examples of how AV formats are concretely embedded within a social-historical and technical context. The signal captured – when reduced to bit stream alone – is simply not an adequate archival source. Without sufficient context too much historical substance is shed.
In this respect I disagree with Ranger’s claim that ‘all that really may be needed moving ahead [for videotape digitisation] is a note in the record for the new digital preservation master that documents the source.’ To really preserve the material, the metadata record needs to be rich enough for a future researcher to understand how a format was used, and what it enabled users to do.
‘Rich enough’ will always be down to subjective judgement, but such judgements can be usefully informed by understanding what makes AV archive material unique, especially within the context of mixed-content archives.
Moving Forward
So, to think about this practically. How could the archive item I discuss at the beginning of the article be contextualised in a way that was useful to me, as a researcher?
At the most basic level the description would need to include:
The format it was recorded on, including details of tape stock and machine used to record material
When it was digitised
Who digitised it (an individual, an institution)
In an ideal world the metadata would include:
Images of the original artefact – particularly important if digital version is now the only remaining copy
Storage history (of original and copy)
Accompanying information (e.g., production sheets, distribution history – anything that can illuminate the ‘life’ of artefact, how it was used)
These suggestions may seem obvious, but it is surprising the extent to which they are overlooked, especially when the most pressing concern during digitisation is access alone.
In every other area of archival life, preserving the context of item is deemed important. The difference with AV material is that the context of use is often complex, and in the case of video, is always changing.
As stressed earlier: in 2016 and beyond you will probably only get one chance to transfer collections stored on magnetic tape, so it is important to integrate rich descriptions as part of the transfer.
Capturing the content alone is not sufficient to preserve the integrity of the video artefact. Creating a richer metadata record will take more planning and time, but it will definitely be worth it, especially if we try to imagine how future researchers might want to view and understand the material.
In the last Greatbear article we quoted sage advice from the International Association of Audiovisual Archivists: ‘Optimal preservation measures are always a compromise between many, often conflicting parameters.’ [1]
While this statement is true in general for many different multi-format collections, the issue of compromise and conflicting parameters becomes especially apparent with the preservation of digitized and born-digital video. The reasons for this are complex, and we shall outline why below.
Lack of standards (or are there too many formats?)
Carl Fleischhauer writes, reflecting on the Federal Agencies Digitization Guidelines Initiative (FADGI) research exploring Digital File Formats for Videotape Reformatting (2014), ‘practices and technology for video reformatting are still emergent, and there are many schools of thought. Beyond the variation in practice, an archive’s choice may also depend on the types of video they wish to reformat.’ [2]
We have written in depth on this blog about the labour intensity of digital information management in relation to reformatting and migration processes (which are of course Greatbear’s bread and butter). We have also discussed how the lack of settled standards tends to make preservation decisions radically provisional.
In contrast, we have written about default standards that have emerged over time through common use and wide adoption, highlighting how parsimonious, non-interventionist approaches may be more practical in the long term.
The problem for those charged with preserving video (as opposed to digital audio or images) is that ‘video, however, is not only relatively more complex but also offers more opportunities for mixing and matching. The various uncompressed-video bitstream encodings, for example, may be wrapped in AVI, QuickTime, Matroska, and MXF.’ [3]
What then, is this ‘mixing and matching’ all about?
It refers to all the possible combinations of bitsteam encodings (‘codecs’) and ‘wrappers’ that are available as target formats for digital video files. Want to mix your JPEG2000 – Lossless with your MXF, or ffv1 with your AVI? Well, go ahead!
What then is the difference between a codec and wrapper?.
As the FADGI report states: ‘Wrappers are distinct from encodings and typically play a different role in a preservation context.’ [4]
The wrapper or ‘file envelope’ stores key information about the technical life or structural properties of the digital object. Such information is essential for long term preservation because it helps to identify, contextualize and outline the significant properties of the digital object.
Information stored in wrappers can include:
Content (number of video streams, length of frames),
Context (title of object, who created it, description of contents, re-formatting history),
Video rendering (Width, Height and Bit-depth, Colour Model within a given Colour Space, Pixel Aspect Ratio, Frame Rate and Compression Type, Compression Ratio and Codec),
Audio Rendering – Bit depth and Sample Rate, Bit Rate and compression codec, type of uncompressed sampling.
Codecs, on the other hand, define the parameters of the captured video signal. They are a ‘set of rules which defines how the data is encoded and packaged,’ [5] encompassing Width, Height and Bit-depth, Colour Model within a given Colour Space, Pixel Aspect Ratio and Frame Rate; the bit depth and sample rate and bit rate of the audio.
Although the wrapper is distinct from the encoded file, the encoded file cannot be read without its wrapper. The digital video file, then, comprises of wrapper and at least one codec, often two, to account for audio and images, as this illustration from AV Preserve makes clear.
Diagram taken from AV Preserve’s A Primer on Codecs for Moving Image and Sound Archives
Pick and mix complexity
Why then, are there so many possible combinations of wrappers and codecs for video files, and why has a settled standard not been agreed upon?
Fleischhauer at The Signal does an excellent job outlining the different preferences within practitioner communities, in particular relating to the adoption of ‘open’ and commercial/ proprietary formats.
Compellingly, he articulates a geopolitical divergence between these two camps, with those based in the US allegedly opting for commercial formats, and those in Europe opting for ‘open.’ This observation is all the more surprising because of the advice in FADGI’s Creating and Archiving Born Digital Video: ‘choose formats that are open and non-proprietary. Non-proprietary formats are less likely to change dramatically without user input, be pulled from the marketplace or have patent or licensing restrictions.’ [6]
One answer to the question: why so many different formats can be explained by different approaches to information management in this information-driven economy. The combination of competition and innovation results in a proliferation of open source and their proprietary doubles (or triplets, quadruples, etc) that are constantly evolving in response to market ‘demand’.
Impact of the Broadcast Industry
An important area to highlight driving change in this area is the role of the broadcast industry.
Format selections in this sector have a profound impact on the creation of digital video files that will later become digital archive objects.
In the world of video, Kummer et al explain in an article in the IASA journal, ‘a codec’s suitability for use in production often dictates the chosen archive format, especially for public broadcasting companies who, by their very nature, focus on the level of productivity of the archive.’ [7] Broadcast production companies create content that needs to be able to be retrieved, often in targeted segments, with ease and accuracy. They approach the creation of digital video objects differently to how an archivist would, who would be concerned with maintaining file integrity rather ensuring the source material’s productivity.
Furthermore, production contexts in the broadcast world have a very short life span: ‘a sustainable archiving decision will have to made again in ten years’ time, since the life cycle of a production system tends to be between 3 and 5 years, and the production formats prevalent at that time may well be different to those in use now.’ [8]
Take, for example, H.264/ AVC ‘by far the most ubiquitous video coding standard to date. It will remain so probably until 2015 when volume production and infrastructure changes enable a major shift to H.265/ HEVC […] H.264/ AVC has played a key role in enabling internet video, mobile services, OTT services, IPTV and HDTV. H.264/ AVC is a mandatory format for Blu-ray players and is used by most internet streaming sites including Vimeo, youtube and iTunes. It is also used in Adobe Flash Player and Microsoft Silverlight and it has also been adopted for HDTV cable, satellite, and terrestrial broadcasting,’ writes David Bull in his book Communicating Pictures.
HEVC, which is ‘poised to make a major impact on the video industry […] offers to the potential for up to 50% compression efficiency improvement over AVC.’ Furthermore, HEVC has a ‘specific focus on bit rate reduction for increased video resolutions and on support for parallel processing as well as loss resilience and ease if integration with appropriate transport mechanisms.’ [9]
Increased compression
The development of codecs for use in the broadcast industry deploy increasingly sophisticated compression that reduce bit rate but retain image quality. As AV Preserve explain in their codec primer paper, ‘we can think of compression as a second encoding process, taking coded information and transferring or constraining it to a different, generally more efficient code.’ [10]
The explosion of mobile, video data in the current media moment is one of the main reasons why sophisticated compression codecs are being developed. This should not pose any particular problems for the audiovisual archivist per se—if a file is ‘born’ with high degrees of compression the authenticity of the file should not ideally, be compromised in subsequent migrations.
Nevertheless, the influence of the broadcast industry tells us a lot about the types of files that will be entering the archive in the next 10-20 years. On a perceptual level, we might note an endearing irony: the rise of super HD and ultra HD goes hand in hand with increased compression applied to the captured signal. While compression cannot, necessarily, be understood as a simple ‘taking away’ of data, its increased use in ubiquitous media environments underlines how the perception of high definition is engineered in very specific ways, and this engineering does not automatically correlate with capturing more, or better quality, data.
Like error correction that we have discussed elsewhere on the blog, it is often the anticipation of malfunction that is factored into the design of digital media objects. These, in turn, create the impression of smooth, continuous playback—despite the chaos operating under the surface. The greater clarity of the visual image, the more the signal has been squeezed and manipulated so that it can be transmitted with speed and accuracy. [11]
MXF
Staying with the broadcast world, we will finish this article by focussing on the MXF wrapper that was ‘specifically designed to aid interoperability and interchange between different vendor systems, especially within the media and entertainment production communities. [MXF] allows different variations of files to be created for specific production environments and can act as a wrapper for metadata & other types of associated data including complex timecode, closed captions and multiple audio tracks.’ [12]
The Presto Centre’s latest TechWatch report (December 2014) asserts ‘it is very rare to meet a workflow provider that isn’t committed to using MXF,’ making it ‘the exchange format of choice.’ [13]
We can see such adoption in action with the Digital Production Partnership’s AS-11 standard, which came into operation October 2014 to streamline digital file-based workflows in the UK broadcast industry.
While the FADGI reports highlights the instability of archival practices for video, the Presto Centre argue that practices are ‘currently in a state of evolution rather than revolution, and that changes are arriving step-by-step rather than with new technologies.’
They also highlight the key role of the broadcast industry as future archival ‘content producers,’ and the necessity of developing technical processes that can be complimentary for both sectors: ‘we need to look towards a world where archiving is more closely coupled to the content production process, rather than being a post-process, and this is something that is not yet being considered.’ [14]
The world of archiving and reformatting digital video is undoubtedly complex. As the quote used at the beginning of the article states, any decision can only ever be a compromise that takes into account organizational capacities and available resources.
What is positive is the amount of research openly available that can empower people with the basics, or help them to delve into the technical depths of codecs and wrappers if so desired. We hope this article will give you access to many of the interesting resources available and some key issues.
As ever, if you have a video digitization project you need to discuss, contact us—we are happy to help!
References:
[1] IASA Technical Committee (2014) Handling and Storage of Audio and Video Carriers, 6.
[2] Carl Fleischhauer, ‘Comparing Formats for Video Digitization.’http://blogs.loc.gov/digitalpreservation/2014/12/comparing-formats-for-video-digitization/.
[3] Federal Agencies Digital Guidelines Initiative (FADGI), Digital File Formats for Videotape Reformatting Part 5. Narrative and Summary Tables. http://www.digitizationguidelines.gov/guidelines/FADGI_VideoReFormatCompare_pt5_20141202.pdf, 4.
[4] FADGI, Digital File Formats for Videotape, 4.
[5] AV Preserve (2010) A Primer on Codecs for Moving Image and Sound Archives& 10 Recommendations for Codec Selection and Management, www.avpreserve.com/wp-content/…/04/AVPS_Codec_Primer.pdf, 1.
[6] FADGI (2014) Creating and Archiving Born Digital Video Part III. High Level Recommended Practices, http://www.digitizationguidelines.gov/guidelines/FADGI_BDV_p3_20141202.pdf, 24.
[7] Jean-Christophe Kummer, Peter Kuhnle and Sebastian Gabler (2015) ‘Broadcast Archives: Between Productivity and Preservation’, IASA Journal, vol 44, 35.
[8] Kummer et al, ‘Broadcast Archives: Between Productivity and Preservation,’ 38.
[9] David Bull (2014) Communicating Pictures, Academic Press, 435-437.
[10] Av Preserve, A Primer on Codecs for Moving Image and Sound Archives, 2.
[11] For more reflections on compression, check out this fascinating talk from software theorist Alexander Galloway. The more practically bent can download and play with VISTRA, a video compression demonstrator developed at the University of Bristol ‘which provides an interactive overview of the some of the key principles of image and video compression.
[12] ‘FADGI, Digital File Formats for Videotape, 11.
[13] Presto Centre, AV Digitisation and Digital Preservation TechWatch Report #3, https://www.prestocentre.org/, 9.
[14] Presto Centre, AV Digitisation and Digital Preservation TechWatch Report #3, 10-11.
Screenshot of software encoding a file to MP3 used at the Great Bear
After we have migrated your analogue or digital tape to a digital file, we offer a range of delivery formats.
For video, using the International Association of Sound & Audiovisual Archives Guidelines for the Preservation of Video Recordings, as our guide, we deliver FFV1 lossless files or 10-bit uncompressed video files in .mkv or QuickTime compatible .mov containers. We add viewing files as H264 encoded .mp4 files or DVD. We’ll also produce any other digital video files, according to your needs, such as AVI in any codec; any MacOS, Windows or GNU/Linux filesystem (HFS+, NTFS or EXT3.
For audio we offer Broadcast WAV (B-WAV) files on hard drive or optical media (CD) at 16 bit/44.1 kHz (commonly used for CDs) or 24 bit/96 kHz (which is the minimum recommended archival standard) and anything up to 24 bit / 192 kHz. We can also deliver access copies on CD or MP3 (that you could upload to the internet, or listen to on an ipod, for example).
Why are there so many digital file types and what distinguishes them from each other?
The main difference that is important to grasp is between an uncompressed digital file and a compressed one.
On the JISC Digital Media website, they describe uncompressed audio files as follows:
‘Uncompressed audio files are the most accurate digital representation of a soundwave, but can also be the most resource-intensive method of recording and storing digital audio, both in terms of storage and management. Their accuracy makes them suitable for archiving and delivering audio at high resolution, and working with audio at a professional level, and they are the “master” audio format of choice.’
Why uncompressed?
As a Greatbear client you may wonder why you need a large, uncompressed digital file if you only want to listen to your old analogue and digital tapes again. The simple answer is: we live in an age where information is dynamic rather static. An uncompressed digital recording captured at a high bit and kHz rate is the most stable media format you can store your data on. Technology is always changing and evolving, and not all types of digital files that are common today are safe from obsolescence.
It is important to consider questions of accessibility not only for the present moment, but also for the future. There may come a time when your digitised audio or video file needs to be migrated again, so that it can be played back on whatever device has become ‘the latest thing’ in a market driven by perpetual innovation. It is essential that you have access to the best quality digital file possible, should you need to transport your data in ten, fifteen or twenty years from now.
Compression and compromise?
Uncompressed digital files are sound and vision captured in their purest, ‘most accurate’ form. Parts of the original recording are not lost when the file is converted or saved. When a digital file is saved to a compressed, lossy format, some of its information is lost. Lossy compression eliminates ‘unnecessary’ bits of information, tailoring the file so that it is smaller. You can’t get the original file back after it has been compressed so you can’t use this sort of compression for anything that needs to be reproduced exactly. However it is possible to compress files to a lossless format, which does enable you to recreate the original file exactly.
In our day to day lives however we encounter far more compressed digital information than uncompressed.
There would be no HD TV, no satellite TV channels and no ipods/ MP3 players without compressed digital files. The main point of compression is to make these services affordable. It would be incredibly expensive, and it would take up so much data space, if the digital files that were streamed to televisions were uncompressed.
‘Every so often I’ll get the proper CD version of an album I’ve fallen in love with as a download, and I’ll get a rude shock when confronted by the sense of dimension and spatiality in the music’s layers, the sculpted force of the drums, the sheer vividness of the sound. The difference between CD and MP3 is similar to that between “not from concentrate” orange juice and juice that’s been reconstituted from concentrate. (In this analogy vinyl would be ‘freshly squeezed, perhaps). Converting music to MP3 is a bit like the concentration process, and its done for much the same reason: it’s much cheaper to transport concentrate because without the water it takes up a lot loss volume and it weighs a lot less. But we can all taste the difference.’
As a society we are slowly coming to terms with the double challenge of hyper consumption and conservation thrown up by the mainstreaming of digital technology. Part of that challenge is to understand what happens to the digital data we use when we click ‘save as,’ or knowing what decisions need to be made about data we want to keep because it is important to us as individuals, or to wider society.
At Greatbear we can deliver digital files in compressed and uncompressed formats, and are happy to offer a free consultation should you need it to decide what to do with your tape based digital and analogue media.
