Three Views Of Digital Preservation

I worked on a large inventory project with a group of media production units in the Department of Defense a few years ago. We were at one of the storage facilities and a shipment of albums that had been on a base in Spain came in that day. As we were opening one of the couch-sized boxes to get an estimate of what was inside, someone warned us to be careful and not dig around in the box too quickly — you couldn’t be sure what critter had slipped in during years of storage and months of international transport. We laughed at what we thought was some friendly teasing, but he was serious. Scorpions, insects, spiders, reptiles, etc. could have crept in at any time, and we had to watch out.

From an archival acquisition standpoint, you never know what’s in the box, but we really didn’t know what might be in the box.

********

1. Narrative

Not knowing the contents of a box — and potential issues with those contents — when it comes into the archive is to be expected. That is why we open them and process collections. Actively ignoring those contents is irresponsible, as metaphorically outlined by Ed Summers in a recent blog post (“Ignoring Our Visual Heritage”) on the problem with the outright rejection of understanding and dealing with managing and preserving digtial collections. This echos the issue in the moving image preservation community over the continued squabble between film and digital, an issue that seems to have escalated over the past few months, especially exacerbated by Paramount’s announcement that it has ceased almost all distribution via 35mm film prints.

Chris Horak, Director of the UCLA Film & Television Archive, was interviewed on NPR about this news story and wrote a follow up blog post on the Archive’s site in response to criticism of his remarks by “digital proselytizers on Twitter and NPR’s comment section”. The Film & Television Archive is one of the premiere moving image archives in the world, and if you enjoy cinema you will have no doubt seen their name frequently as the material source and major backer of some of the most significant film preservation projects we are fortunate enough to benefit from. And Chris has been a foundational figure in the field of film preservation, up through his time at Eastman House to his continuing important work at UCLA and more.

Chris makes it clear that his position is not film-or-nothing anti-digital. He lays out the facts of the studio distribution model, and agrees that there are benefits to the use of digital technology for restoration and distribution. However, it is his belief that film is the only worthwhile archival format, and in that realm digital is too unstable (no permanent archival format; file formats keep changing; files disappear) and too costly overall (constant migration; infrastructure costs).

These are familiar talking points that have been sculpted in the film preservation community, and they have been fairly effective in communicating to the public because, well, they are true in their own way. But the anecdotes and figures are, ultimately, misrepresentative of the practice of digital preservation.

A point by point refutation would likely be read as contentious here, and that is not my intention. It seems tiresome to say it, but digital preservation is a paradigm shift. But then again, it isn’t. It is true that there is no digital archival format because digital preservation is not a format. It is a system, a framework to apply to archival practices which includes making informed format selections knowing well that the format will not persist, establishing and following policies, regular monitoring of collections, backup measures, a centralization of resources and shared services, and more. It’s not about putting items on a shelf and letting them sit, trusting things will be okay.

But though that is a common practice, in my opinion that is poor collection management for physical collections as well. Isn’t the reason digital collections need fixity monitoring the same reason film collections benefit from regular condition checks, and the same reason the University of Missouri Library would have benefited from ongoing assessment of their offsite collections? Digital collections are not so forgiving to the shortcuts we have taken with physical collections, and the paradigm shift is more the fact of being forced to reckon with such issues more immediately than we are used to. That being the case, we also have more powerful, more efficient, and more widely available tools for dealing with digital collection than we have had with physical collection. Though we have more work to do, we can actually do it better and more easily.

But back to my point, digital preservation is a system of policies and processes. The reason things like Toy Story 2 are “almost lost” is not because digital is unmanageable, but because the proper policies for collection management were not followed or were not in place. And the failure of the BBC DAMS is not a failure of digital preservation. A DAM is not a digital preservation system. It is a way to track, retrieve, and distribute or use assets, designed for digital materials. These are fundamental activities of an archive whether dealing with digital or physical assets, activities which in many cases are not fulfilled in physical archives because the money has not been spent to process, catalog, and reformat for access.

A DAMS is a specific type of tool used to support collection management, and it may take many forms depending on the requirements of the organization. The BBC was trying to correct that and mitigate the risk of only a few people knowing where assets were stored or know how to transfer the tapes access (as long as decks lasted). The high cost of the system was likely because it was trying to integrate various production and post-production functions that needed to be performed on very tight schedules (search for content, pull out a clip, edit it into a new story, etc.) which are uses well beyond and totally separate from a basic level of collection management. A large scale, international post-production environment requires a multi-million dollar DAMS to do its work. A typical research archive does not.

High costs are often cited as a reason to avoid digital preservation. Chris does note the high price of the shift to digital projection in commercial theatres, though that’s not really a preservation issue — just a distracting number, as is the big number for the BBC DAMS. I’m not sure about the $15,000 cost for LTO migration. It is in the realm of possibility that there were severe proprietary issues with the LTO-3 tapes that made them very costly to transfer. This is an issue that has been greatly mitigated now with the use of the LTFS interface for data tape formats (including LTO and T10KC), which is available from LTO-5 forward. We’ve seen vendor pricing for transferring files to LTO-5 or 6 in the $250-$300 range per tape (prior to any volume discounting). $15,000 would equate to 50-60 tapes — way more storage than required for one 90 minute film. If there were no extenuating circumstances $15,000 only seem reasonable if one included the cost of purchasing new LTO-5 drives (currently starting around $2,000-$3,000 for a single tape drive, and starting in the $10,000-$30,000 range for multi-tape library systems) to perform the transfer or read the new tapes, a cost which should actually be distributed across the general migration to and use of LTO-5, the same as buying a set of rewinds or a projector in order to properly care for and access physical formats. Whatever the case, yes, as with anything, startup costs and the building of infrastructure are high, but they are investments in continued viability.

Stepping back to speak in general, I find that arguments in this area often fall back to anecdotal or singular examples. This one thing failed, that one thing succeeded, this happened to a giant institution, that happened to my cousin. The problem (or the strength) in such arguments is that they are true in fact (yes that did happen) but that does not mean that the story is broadly applicable to the issue at hand, and any point/counter-pointing spins off into a wormhole, losing the thread and the spirit of the question.

In spirit the position that Chris puts forth and that I have put forth in this venue agree. We need to find the best option for preserving the content which resides on formats of limited longevity with severe technological dependencies and flaws, and there is likely a point in the future when film as we know it will no longer be available or a reasonable option for large scale preservation efforts to retain that content. At that point, if we do not have a viable option to move to, there will not be the time to invent something new. And that is why we need to focus on developing the policies and tools for digital preservation now, and avoid repeating damaging misinformation in regards to those efforts.

2. Theoretical

When we advocate for archiving and preservation, the argument will most likely follow one of two tracks: The cost of not preserving (i.e., the loss of our cultural heritage), or the cost of preserving (i.e., $$$$). We can also argue a third, combined track, that the cost of inaction is tied to dollars as well due to the loss of one’s investment in prior preservation efforts and the greater expense of preserving high risk items. However, within archives, we would probably prefer speaking to the first angle. So much more noble and more in line with our humanistic focus. More frequently, the realities of the market and of funding force us to argue within the second, and much more defensively at that.

One market reality is that archives are cost centers within an organization. They require expenditure to maintain but infrequently realize any direct income in return. Except in rare cases, the idea that licensing or selling archival content will sufficiently fund the department long term I believe has been repeatedly disproven. This is not to say that the archive is valueless. There is a massive amount of institutional value, cultural capital, and risk avoidance in supporting archives and records management, but those are soft arguments which are harder to make stick. Valid arguments I have spent years trying to formulate, but soft in comparison to dollars made or saved. IT departments are also cost centers within institutions, but they are seen as critical to infrastructure and to supporting organizational activities, and they also manage assets that everyone comes into contact with everyday.

A second market reality is the fact that, especially in media preservation, the market dictates the formats we deal with and the equipment that is available to us. When a format fails or stops being profitable, its manufacture ends with little concern from the corporation for continued or future use. Obsolescence sets in, and the archivist must manage a way through and out of that slow decline. The effects of obsolescence in the digital realm mirror this, but not in some extreme way as the hysteria over digital obsolescence would suggest. Some formats fail quickly, some are proprietary and change frequently (as various stocks and formulations do), and some are strongly adopted and persist for years.

Archiving costs money. Preservation costs money. Access costs money. The material of creation – the medium – costs money. And not just one time costs, but expenditure over time to ensure that content remains available in a sustainable and responsible way.

One issue with audiovisual preservation is that our media of choice is a product of commerce. It was originally created to be marketed and sold, and the marketing and selling of media drives its continued or discontinued manufacture. At some point, art adopted commerce, adopted the medium as a form of expression. This did not change the commercial nature of the product, but added a use for which the argument of preservation is at odds with the commercial venture. As noted above, the artistic argument is preferred, and it has historically been a part of the manufacturer’s marketing efforts, but in the end the artistic valuation cannot subsume the commercial concern within a medium that is so entirely dependent on industrial manufacturing.

The mind desires sweetness and light. It craves unity. Archives hold decay. Preservation embraces multiplicity.

3. Poetical

The format is dead. The format is dead and we killed it. Through seeking improvement in quality, in distribution, in manufacture, and in profit, we have persistently slaughtered our formats. The purity we idealize of the moment of origin exists only in legend, as documented well after the fact and continually reshaped by adherents.

The movement of the form itself is one of democritization, from the priests to the people. The format has followed the form, becoming more accessible if less grand. There is still desire and room for the ritualistic and the baroque, but in general the format thrives as it is better able to be understood, felt, and used by a wider, less expert audience with limited intercession from the priestly class. We cannot deny the role of the market here, but the market responds to desire as much as it tries to drive it.

— Joshua Ranger