Digital black hole

It's been a while since I last wrote about the problem of digital preservation. The paradox is that while we have never had access to so much information there's a great danger that much of it will disappear without a trace. Will future researchers be able to access records of our everyday life? Who preserves all the communication going on in social networks? You may say that much of it isn't worth preserving but often it's precisely the everyday communication that can be most interesting. Who preserves all our photos and videos, especially if the applications that host them, like Flickr or Picasa, go up in smoke one day? Websites are preserved to some extent but the vast majority are not. Seemingly the average website has an average life expectancy of 75 days!

I was reminded of this by an article in The Guardian, Is copyright getting in the way of us preserving our history?, where the author sees copyright legislation as an added barrier to digital preservation. One problem area is that lack of archives for TV and radio productions. Getting permission to make these archives accessible on the net is not easy and without the permission of all parties nothing gets preserved.

"the vast majority of documentary films from the 20th century will be forever buried in a lawyer's thicket inaccessible (legally) because of a set of permissions built into these films at their creation".

Added to this is the fact that digital files deteriorate and must be reformatted regularly. Food for thought if you want to be able to show your family photos and films to your grandchildren. It's impossible to preserve everything but we need to consider carefully what types of net content should be preserved and ensure that it's done so in a format that will be relatively future-proof. I doubt if we'll find a format that is as durable as the papyrus scrolls of ancient Egypt.

How free is free?

We all assume that everything on the net is free and that somehow advertising pays for all the services we use. As a result we upload tons of content to servers somewhere out there and believe they're safe there. But what happens when the company providing that service has financial problems and decides to charge for the service or, worse still, decides simply to pull out the plug?

There's a critical article on this theme in Times Higher Education by Tara Brabazon, Beware writers bearing promises of a free internet. In particular Chris Anderson's book "Free" comes under fire as it shows the "freemium" movement to be ultimately highly commercial rather than the philanthropic movement it is sometimes presented as.

"His (Anderson's) “free” is corporatised. The cost of free is permanence, reliability and stability. The old cliché is correct. We get what we pay for: when the price is free, then the “service” can be removed without questions or reprisal."

Brabazon used a web service for storing her audio files that suddenly disappeared because the owners decided that the service wasn't being used enough. Since it was "free" they had no obligation to communicate with the users. In addition, the cost of using many free services is the irritation of having sometimes highly inappropriate ads next to your content; especially sensitive if you're using it for teaching.

We trust companies like Google and Ning but if times get tough who knows what may happen. Our information is at their mercy. Terms can be changed at the drop of a hat and it's important we are aware of this and not place unlimited trust in companies that, after all, are there to make money. The free services are, of course, mostly there as bait to get you into the premium services. I admit the irony of writing this on a free blog tool!

The article does however point us in the direction of a genuine non-profit archive for digital material, the Internet Archive. This is a massive library of films, photos, audio, texts and an archive of 150 billion web pages from 1996 to the present day. The archive “is free and open for everyone to use ..... to encourage widespread use of texts in new contexts by people who might not have used them before." This is possibly the real meaning of "free".

Pictures at an exhibition

I enjoy art galleries and especially work from the past 100 years or so. I was walking around one the other day and I looked at some video art. Of course I began thinking about preservation. Paintings and sculpture can of course be preserved for hundreds of years for future generations to admire and study but what about film and modern video? If future archaeologists discover a video tape or a DVD how are they going to play them?
Today's artists and writers have a vast range of media by which to express themselves but how will it be preserved for future reference? If Shakespeare had written exclusively on the net there wouldn't be anything left of his work now. The collected letters of Wordsworth or Dickens are still available but who will ensure that the collected blogs or e-mails of today's writers are preserved? Many artists are doing very interesting work using net-based tools but such work will be transient, available only as long as the server is connected or the format becomes obsolete.
I read a great science fiction story as a kid about space explorers who land on Earth to find it uninhabited and in ruins, presumably after a nuclear war. They find the remains of human civilisation and most importantly a film. They take the film back to their own planet and scientists discover how to play it. After years of analysis and study they feel that they have a good idea of what life was once like on Earth. Only one thing baffled them. At the end of the film were four words they couldn't decipher .... "A Walt Disney Production".

Digital preservation

Remember the original TV version of Mission Impossible? I always loved the start when Jim Phelps got his orders from a reel-to-reel tape recorder rounding off with the classic line "This tape will self destruct in five seconds" after which the tape magically went up in smoke. Self-destructing tapes are still not commonplace even today but we do have a problem with information that inexplicably disappears.

I've had several cases of files that I've stored on a USB memory stick that suddenly became corrupted and about as useful as a second-hand Mission Impossible tape recorder. My digital camera suffered a similar fate this summer when one day's photos became corrupted data overnight. Back to a favourite theme then; how long can you trust a memory stick to actually store what you put on it? How long will a CD or DVD last?

I read an fascinating article in Dagens Nyheter (in Swedish) about the growing concern about digital preservation. The problem is how we are going to preserve all the digital films, photos and documents on all the world's servers. According to one test around 40% of all DVD-R discs will be unreadable within 15 years and similar figures apply for many other storage forms. All archived material has to be reformatted regularly otherwise the next generation of computers will not be able to read them and just this task will be a full-time never-ending job for a lot of people. In contrast paper can be preserved for hundreds of years.

An interesting example was the BBC's project to produce a digital domesday book to celebrate the 900th anniversary of the first one. One million people contributed to this epic which was produced on a laser disc. Sadly 15 years later the material was unreadable because the technology which could read the files had become obsolete. Furthermore it costs $12,000 a year to preserve a digital movie and a mere $1,000 to keep a traditional film version in a suitable archive.

It's reassuring to read that there are now projects and organisations dedicated to trying to find solutions to this enormous problem. It would also be nice if the industry could cooperate by trying to ensure more compatibility between applications and especially backward compatibility. Otherwise we face losing vast amounts of information, documentation, culture and entertainment for ever.

The Swedish Centre for Long-term Digital Preservation is just one of many international groups working with this. There's also an EU project called CASPAR. Good luck to them!

Why?

I've contributed to a few blogs but never considered starting my own until now. The biggest obstacle has been the fact that the net is bursting with blogs that virtually no-one reads and why should I add to the pile? I belong to a generation that feels that you have to work hard to get published and if you can publish absolutely anything without even a bit of a struggle then what's the point? The point of this and probably the majority of blogs is not to become famous but simply to write down ideas and impressions and be able to access them anywhere. If you have a few friends or colleagues who want to contribute then that's a bonus.

I'm interested in seeing what happens when I start this blog and how it develops. I don't have a clear idea of what I want to do with this but I like the idea of just casting off and letting the wind blow me along. Will anyone stumble upon this and comment? How do you get noticed in the world of blogging? Will I turn up on other blogs? If I continue writing like this, probably not.

Has anyone done any research into dead web sites, defunct blogs etc? There must be millions of sites where nothing has happened for years but they still use up space on someone's server with digital tumbleweed blowing across the home page. Google sometimes offers one as a possible answer to a search but when you get there and it says "Last updated 20 July 2002" you get a sinking feeling. How many petabytes of space could be saved by dumping all that digital trash? Or should it be saved for posterity and if so who decides? If you want some web-based nostalgia take a look at the Internet Archive which has a vast archive of old web sites but I'm not sure how much of the blogosphere they preserve.

I promise to press the self-destruct button on this one before the tumbleweed hits town.