tracking errors

17
Dec
2014

DVCAM transfers, error correction coding & misaligned machines

This article is inspired by a collection of DVCAM tapes sent in by London-based cultural heritage organisation Sweet Patootee. Below we will explore several issues that arise from the transfer of DVCAM tapes, one of the many Digital Video formats that emerged in the mid-1990s. A second article will follow soon which focuses on the content of the Sweet Patootee archive, which is a fascinating collection of video-taped oral histories of 1 World War veterans from the Caribbean.

The main issue we want to explore below is the role error correction coding performs both in the composition of the digital video signal and during the preservation playback. We want to highlight this issue because it is often assumed that DVCAM, which first appeared on the market in 1996, is a fairly robust format.

The work we have done to transfer tapes to digital files indicates that error correction coding is working overdrive to ensure we can see and hear these recordings. The implication is that DVCAM collections, and wider DV-based archives, should really be a preservation priority for institutions, organisations and individuals.

Before we examine this in detail, let’s learn a bit about the technical aspects of error correction coding.

Error error error

DVFormat7 Error correction coding is a staple part of audio and audio-visual digital media. It is of great important in the digital world of today where the higher volume of transmitted signals require greater degrees of compression, and therefore sophisticated error correction schemes, as this article argues.

Error correction works through a process of prediction and calculation known as interpolation or concealment. It takes an estimation of the original recorded signal in order to re-construct parts of the data that have been corrupted. Corruption can occur due either to wear and tear, or insufficiencies in the original recorded signal.

Yet as Hugh Robjohns explains in the article ‘All About Digital Audio’ from 1998:

 ‘With any error protection system, if too many erroneous bits occur in the same sample, there is a risk of the error detection system failing, and in practice, most media failures (such as dropouts on tape or dirt on a CD), will result in a large chunk of data being lost, not just the odd data bit here and there. So a technique called interleaving is used to scatter data around the medium in such a way that if a large section is lost or damaged, when the data is reordered many smaller, manageable data losses are formed, which the detection and correction systems can hopefully deal with.’

There are many different types of error correction, and ‘like CD-ROMs, DV uses Reed-Solomon (RS) error detection and correction coding. RS can correct localised errors, but seldom can reconstruct data damaged by a dropout of significant size (burst error),’ explains this wonderfully detailed article about DV video formats archived on web archive.

The difference correction makes

Digital technology’s error correction is one of the key things that differentiate it from their analogue counterparts. As the IASA‘s Guidelines on the Production and Preservation of Digital Audio Objects (2009) explains:

‘Unlike copying analogue sound recordings, which results in inevitable loss of quality due to generational loss, different copying processes for digital recordings can have results ranging from degraded copies due to re-sampling or standards conversion, to identical “clones” which can be considered even better (due to error correction) than the original.’ (65)

To think that digital copies can, at times, exceed the quality of the original digital recording is both an astonishing and paradoxical proposition. After all we are talking about a recording that improves at the perceptual level, despite being compositionally damaged. It is important to remember that error correction coding cannot work miracles, and there are limits to what it can do.

Dietrich Schüller and Albrecht Häfner argue in the International Association of Sound and Audiovisual Archives’s (IASA) Handling and Storage of Audio and Video Carriers (2014) that ‘a perfect, almost error free recording leaves more correction capacity to compensate for handling and ageing effects and, therefore, enhances the life expectancy.’ If a recording is made however ‘with a high error rate, then there is little capacity left to compensate for further errors’ (28-29).

The bizarre thing about error-correction coding then is the appearance of clarity it can create. And if there are no other recordings to compare with the transferred file, it is really hard to know what the recorded signal is supposed to look and sound like were its errors not being corrected.

DVCAM PRO

When we watch the successfully migrated, error corrected file post-transfer, it matters little whether the original was damaged. If a clear signal is transmitted with high levels of error correction, the errors will not be transferred, only the clear image and sound.

Contrast this with a damaged analogue tape it would be clearly discernible on playback. The plus point of analogue tape is they do degrade gracefully: it is possible to play back an analogue tape recording with real physical deterioration and still get surprisingly good results.

Digital challenges

The big challenge working with any digital recordings on magnetic tape is to know when a tape is in poor condition prior to playback. Often tape will look fine and, because of error correction, will sound fine too until it stops working entirely.

How then did we know that the Sweet Patootee tapes were experiencing difficulties?

Professional DV machines such as our DVC PRO have a warning function that flashes when the error-correction coding is working at heightened levels. With our first attempt to play back the tapes we noticed that regular sections on most of the tapes could not be fixed by error correction.

The ingest software we use is designed to automatically retry sections of the tape with higher levels of data corruption until a signal can be retrieved. Imagine a process where a tape automatically goes through a playing-rewinding loop until the signal can be read. We were able to play back the tapes eventually, but the high level of error correction was concerning.

DVFormat6

As this diagram makes clear, around 25% of the recorded signal in DVCAM is composed of subcode data, error detection and error correction.

DVCAM & Mis-alignment

It is not just the over-active error correction on DVCAMs that should send the alarm bells ringing.

Alan Griffiths from Bristol Broadcast Engineering, a trained SONY engineer with over 40 years experience working in the television industry, told us that early DVCAM machines pose particular preservation challenges. The main problem here is that the ‘mechanisms are completely different’ for earlier DVCAM machines which means that there is ‘no guarantee’ they will play back effectively on later models.

Recordings made on early DVCAM machines exhibit back tensions problems and tracking issues. This increases the likelihood of DV dropout on playback because a loss of information was recorded onto the original tape. The IASA confirm that ‘misalignment of recording equipment leads to recording imperfections, which can take manifold form. While many of them are not or hardly correctable, some of them can objectively be detected and compensated for.’

One possible solution to this problem, as with DAT tapes, is to ‘misalign’ the replay digital video tape recorder to match the misaligned recordings. However ‘adjustment of magnetic digital replay equipment to match misaligned recordings requires high levels of engineering expertise and equipment’ (2009; 72), and must therefore not be ‘tried at home,’ so to speak.

Our experience with the Sweet Patootee tapes indicates that DVCAM tapes are a more fragile format than is commonly thought, particularly if your DVCAM collection was recorded on early machines. If you have a large collection of DVCAM tapes we strongly recommend that you begin to assess the contents and make plans to transfer them to digital files. As always, do get in touch if you need any advice to develop your plans for migration and preservation.

 

Posted by debra in digitisation expertise, video tape, 0 comments
16
Jun
2014

Digitisation: methodologies, processing and archival practices

  Time-based correction machines We work with a range of customers at Greatbear, digitising anything from personal collections to the content of institutional archives. Because of this, what customers need from a digitisation service can be very different.

A key issue relates to the question of how much we process the digital file, both as part of the transfer and in post-production. In other words, to what extent do we make alterations to the form of the recording when it becomes a digitised artifact. While this may seem like an innocuous problem, the question of whether or not to apply processing, and therefore radically transform the original recording, is a fraught, and for some people, ethical, consideration.

There are times when applying processing technologies is desirable and appropriate. With the transfer of video tape, for example, we always use time-based correctors or frame synchronisers to reduce or eliminate errors during play back. Some better quality video tape machines, such as the U-matic BVU-950P, already have time-based correctors built in which makes external processing unnecessary. As the AV Artifact Atlas explains however, time-based correction errors are very common with video tape:

‘When a different VTR is used to playback the same signal, there can be slight mechanical and electronic differences that prevent the tape from being read in the same way it was written. Perhaps the motors driving the tape in a playback VTR move slightly slower than they did in the camera that recorded the tape, or maybe the head of the playback VTR rotates a fraction quicker than the video head in the machine that recorded the tape. These tiny changes in timing can dramatically affect stability in a video image.’

We also utilise built in processes that are part of machine’s circuitry, such as drop out compensation and noise reduction. We use these, however, not in order to make the tape ‘look better.’ We do it rather as a standard calibration set up, which is necessary for the successful playback of the tape in a manner appropriate to its original operating environment.

After all, video tape machines were designed to be interchangeable. It is likely such stabilising processing would have been regularly used to play back tapes in machines that were different to those they were recorded on. Time-based correction and frame synchronisation are therefore integral to the machine/ playback circuitry, and using such processing tools is central to how we successfully migrate tape-based collections to digital files.

Digital processing tools  Time Based Correction - Close Up

Our visual environment has changed dramatically since the days when domestic video tape was first introduced, let alone since the hay day of VHS. The only certainty is that it will continue to change. Once it was acceptable for images to be a bit grainy and low resolution, now only the crisp clarity of a 4K Ultra HD image will do. There is perhaps the assumption that ‘clearer is better’, that being able to watch moving images in minute detail is a marker of progress.  Yet should this principle be applied to the kinds of digitisation work we do at Great Bear? There are processors that can transform the questionable analogue image into a bright, high definition, colour enriched digital copy. The teranex processor, for example, ‘includes extremely high quality de-interlacing, up conversion, down conversion, SD and HD cross/standards conversion, automatic cadence detection and removal even with edited content, noise reduction, adjustable scaling and aspect ratio conversion.’ ‘Upgrading’ analogue images in this way does come with certain ethical risks.

Talking about ethics in conjunction with video or audio tape might seem a bit melodramatic, but it is at the point of intervention/ non-intervention where the needs of our customers diverge the most. This is not to say that people who do want to process their tapes are unethical – far from it! We understand that for some customers it may be preferable for such processing to occur, or to apply other editing techniques such as noise reduction or amplification, so that audio can be heard with greater clarity.

Instead we want to emphasise that our priority is getting the best out of the tape and our playback machines, rather than relying on the latest processing technology that is also at risk from obsolescence. After all, a heavily processed file will always require further processing at an unknown point in future so that it can maintain visually relevant to whatever format is commercially dominant at the time. Such transformations of the digital file, which are necessarily destructive and permanent, contribute to the further circulation of what Hito Steyerl calls ‘poor images‘, ‘a rag or a rip; an AVI or a JPEG…The poor image has been uploaded, downloaded, shared, reformatted, and reedited. It transforms quality into accessibility, exhibition value into cult value, films into clips, contemplation into distraction.’

Maintaining the integrity, and as far as possible authenticity of the original recordings, is a core part of our methodology. In this way our approach corresponds with Jisc’s mantra of ‘reproduction not optimisation’ where they write:

‘Improving, altering or modifying media for optimisation may seem logical when presenting works to a public or maintaining perceived consistency. It should be remembered that following an often natural inclination to enhance what we perceive to be a poor level of quality is a subjective process prescribed by personal preference, technological trends and cultural influences. In many cases the intentions of a creator are likely to be unknown and this can cause difficulties in interpreting levels of quality. In these instances common sense alongside trepidation should prevail. On the one end of the spectrum unintelligible recordings may be of little use to anyone, whereas at the opposite end recordings from previous eras were not produced with modern standards of clarity in mind.’

It is important to bear in mind, however, that even if a file is subject to destructive editing there may come a time when the metadata created about the artefact can help to illuminate its context and provenance, and therefore help it maintain its authenticity. The debates regarding digital authenticity and archiving will of course shift as time passes and practices evolve.

In the meantime, we will continue to do what we are most skilled at: restoring, repairing and migrating magnetic tape to digital files in a manner that maintains both the integrity of the original operating environment and the recorded signal.

Posted by debra in audio tape, digitisation expertise, video tape, 0 comments
17
Feb
2014

EIAJ ½ inch Video Tape Transfers – Working with Community Groups to Develop Digitisation Projects

We understand that when organisations decide to digitise magnetic tape collections the whole process can take significant amounts of time. From initial condition appraisals, to selecting which items to digitise, many questions, as well as technical and cultural factors, have to be taken into account before a digital transfer can take place.

This is further complicated by that fact that money is not readily available for larger digitisation projects and specific funding has to be sought. Often an evidence base has to be collected to present to potential funders about the value and importance of a collection, and this involves working with organisations who have specific expertise in transferring tape-based collections to digital formats to gain vital advice and support.

We are very happy to work with organisations and institutions during this crucial period of collection assessment and bid development. We understand that even during the pre-application stage informed decisions need to be made about the conditions of tape, and realistic anticipations of what treatments may be required during a particular digitisation project. We are very willing to offer the support and advice that will hopefully contribute to the development of a successful bid.

For example, we recently were contacted by Ken Turner who was involved in Action Space, an experimental, community theatre group established in 1968. Ken has a collection of nearly 40 EIAJ SONY video tapes that were made in the 1980s. Because of the nature of the tapes, which almost always require treatment before they can be played back, transferring the whole collection will be fairly expensive so funding will be necessary to make the project happen. We have offered to do a free assessment of the tapes and provide a ten minute sample of the transfer that can be used as part of an evidence base for a funding bid.

Potential Problems with EIAJ ½ Video Tapes

Extreme close up of EIAJ video recorder, focusing on the 'tracking' function. The EIAJ video tape recorder was developed in the late 1960s and is a fairly important format in the history of recordable media. As the first standardized video tape machine, it could playback tapes made by different companies and therefore made video use far cheaper and more widespread, particularly within a domestic context. The EIAJ standard had a similar democratising impact on non-professional video recording due to its portability, low cost, and versatility.As mentioned above, the EIAJ tapes almost always require treatment before they can be played back, particularly the SONY V30-H and V60-H tapes. Problems with the tape are indicated by squealing and shedding upon playback. This is an example of what the AV Artifact Atlas describe as stiction, ‘when media suffering from hydrolysis or contamination is restricted from moving through the tape path correctly.’ When stiction occurs the tape needs to be removed from the transport and treated immediately, either through baking and cleaning, before the transfer can be completed.

EIAJ tapes that have a polyethylene terephthalate ‘back coating’ or ‘substrate’ may also be affected by temperature or humidity changes in its storage environment. These may have caused the tape pack to expand or contract, therefore resulting in permanent distortion of the tape backing. Such problems are exacerbated by the helical scan method of recording which is common to video tape, which records parallel tracks that run diagonally across the tape from one edge to the other. If the angle that the recorded tracks make to the edge of the tape do not correspond with the scan angle of the head (which always remains fixed), mistracking and information loss can occur, which can lead to tracking errors. Correcting tracking errors is fairly easy as most machines have in-built tracking controls. Some of the earliest SONY CV ½ inch video tape machines didn’t have this function however, so this presents serious problems for the migration of these tapes if their back coating has suffered deformation.

The possibility of collaboration

We are excited about the possibility of working with the Action Space collection, mainly because we would love to opportunity to learn more about their work. Like many other theatre groups who were established in the late 1960s, Action Space wanted to challenge the elitism of art and make it accessible to everyone in the community. In their 1972 annual report, which is archived on the Unfinished Histories: Recording the History of Alternative Theatre website, they describe the purposes of the company as follows:

‘Its workings are necessarily experimental, devious, ambiguous, and always changing in order to find a new situation. In the short term the objectives are to continually question and demonstrate through the actions of all kinds new relationships between artists and public, teachers and taught, drop-outs and society, performers and audiences, and to question current attitudes of the possibility of creativity for everyone. For the longer term the aim is to place the artists in a non-elite set up, to keep “normal” under revision, to break barriers in communication and to recognise that education is a continuing process.’

Although Action Space disbanded in 1981, the project was relaunched in the same year as Action Space Mobile, who are still operating today. The centre of the Action Space Mobile’s philosophy is that they are an arts company ‘that has always worked with people, believing that contact and participation in the arts can change lives positively.’ There is also the London based ActionSpace, who work with artists with learning disabilities.

We hope that offering community heritage projects the possibility of collaboration will help them to benefit from our knowledge and experience. In turn we will have interesting things to watch and listen to, which is part of what makes working in the digitisation world fun and enjoyable.

Posted by debra in video tape, 0 comments