software

Reports from the ‘bleeding edge’ – The PrestoCentre’s AV Digitisation TechWatch Report #02

The PrestoCentre's* AV Digitisation and Digital Preservation TechWatch Report #02, published July 2014, introduces readers to what they describe as the 'bleeding edge' of AV Digitisation and Archive technology.

Written in an engaging style, the report is well worth a read. If you don't have time, however, here are some choice selections from the report which relate to the work we do at Greatbear, and some of the wider topics that have been discussed on the blog.

The first issue to raise, as ever, is continuing technological change. The good news is

'there are no unexpected changes in file sizes or formats on the horizon, but it is fair to say that the inexorable increase in file size will continue unabated […] Higher image resolutions, bits per pixel and higher frame rates are becoming a fact of life, driving the need for file storage capacity, transfer bandwidth and processing speeds, but the necessary technology developments continue to track some form of Moore’s law, and there is no reason to believe that the technical needs will exceed technical capability, although inevitably there will be continuing technology updates needed by archives in order for them to manage new material.'

Having pointed out the inevitability of file expansion, however, others parts of the report clearly express the very real everyday challenges that ever increasing file sizes are posing to the transmission of digital information between across different locations:rate_vs_size.today-0

'transport of content was raised by one experienced archive workflow provider. They maintained that, especially with very high bit-rate content (such as 4k) it still takes too long to transfer files into storage over the network, and in reality there are some high-capacity content owners and producers shipping stacks of disks around the country in Transit vans, on the grounds that, in the right circumstances this can still be the highest bandwidth transfer mechanism, even though the Digital Production Partnership (DPP) are pressing for digital-only file transfer.'

While those hoards of transit vans zipping up and down the motorway between different media providers is probably the exception rather than the rule, we should note that a similar point was raised by Per Platou when he talked about the construction of the Videokuntstarkivet - the Norwegian video art archive. Due to the size of video files in particular, Per found that publishing them online really pushed server capabilities to the absolute maximum. This illustrates that there remains a discrepancy between the rate at which broadcast technologies develop and the economic, technological and ecological resources available to send and receive them.

Another interesting point about the move from physical to file-based media is the increased need for Quality-Control (QC) software tools that will be employed to 'ensure that our digital assets are free from artefacts or errors introduced by encoders or failures of the playback equipment.' Indeed, given that glitches born from slow or interrupted transfers may well be inevitable because of limited server capabilities, software developed by Bristol-based company Vidcheck will be very useful because it 'allows for real-time repair of Luma, Chroma, Gamma and audio loudness issues that may be present in files. This is a great feature given that many of the traditional products on the market will detect problems but will not automatically repair them.'

Other main points worth mentioning from the report is the increasing move to open-source, software only solutions for managing digital collections and the rather optimistic tone directed toward 'archives with specific needs who want to find a bespoke provider who can help design, supply and support a viable workflow option – so long as they avoid the large, proprietary ‘out-of-the-box’ solutions.'

If you are interested in reading further TechWatch reports you can download #01 here, and watch out for #03 that will be written after the International Broadcasting Convention (IBC) which is taking place in September, 2014.

*Update 2020: The PrestoCentre website is, sadly, no longer operational. Some of its content is accessible via the Internet Archive Wayback Machine : Prestocentre.org  and archived at AVA_NET Library.

Posted by debra in audio tape, video tape, 0 comments

Powerful tape conversion tools using SoX, LAME and Bash Scripting

One of our recent and ongoing jobs is a very large, over 2000, NAB 10.5 inch reel to reel archive of quarter inch professional tape recordings.

To fit into the budget but maintain quality we needed to find a way to streamline our workflow in converting the high resolution 24 bit 96 kHz master .wav  files to CD quality (16 bit / 44.1 kHz) and MP3 (320 kbps) audio files.

Typically this would be done manually with 2 track audio software such as Audacity, Peak Audio Tools, Wavelab, etc with a Graphic User Interface (GUI). For such a large amount of individual files this is time consuming, expensive and unnecessary. While many audio editors have batch processing tools, Amadeus Pro being one of our favourites, they are still not as flexible as combining command line tools with a Bash script.

SoX is a powerful command line audio tool that can be used to edit, convert, process, record and play a wide range of digital audio files. It also has a very high quality resampling algorithm which we are using here.

Once the tape reels have been digitised they are split into individual .wav files which are then renamed with artist and track details using a php script that accesses a .csv file. These 24/96 resolution files are then converted to lower, CD resolution using SoX and to 320 kb/s MP3s using LAME. The script also uses sed, the stream editor, to add the text _16_44.wav or _mp3.mp3 to the filename for ease of identification. The script finds all files with the suffix _24_96.wav in all the subdirectories of it’s working directory and processes these.

At the moment this script is running under Mac OS X 10.5.8 using the MacPorts project but will, in time, be moved to one of our Apple G4 servers running the PowerPC version of Debian GNU/Linux 5.05 ‘Lenny’.

(Originally published on Jun 29, 2010)

Posted by greatbear in audio tape, 1 comment