Archiving onto CDs
USING CDs AS AN ARCHIVE MEDIUM: Nigel Deacon
Little is known about the reliability of CDs over the long term. I am collecting information which may help others decide whether storing data on CD is a good idea for them. My perspective can be judged from an email I sent to a friend recently:
.....I am in a bit of a quandary about archiving. It's quite alarming how so much information is being put onto CDs, especially if the primary materials (tapes, prints, etc) are being dumped. Even the BBC archive men are transferring their collection to cd, without realising there may be a problem. There is also a lot of misleading information about the topic on the web, often put there by professionals who ought to know better.
There are advantages to using CDs for sound storage - portability, use of mp3 players, possibility of lossless duplication, file cleanup, and so on. Archiving, however - the preservation of material so that it can be accessed later - perhaps very much later -is another matter. Photographers have a similar problem with digital images.
The BBC Sound Archive is steaming ahead with transferring its recordings from quarter inch tape onto CD-R. They are doing this because it's easier and more convenient to access CDs, and they take up much less storage space. Eventually they may go to on-line access for a lot of the material.
I hope this is not a mistake. My own experience suggests that although CD-Rs are excellent in many ways, they may be unsuitable for long term data storage. Until I have a few more answers, I'm putting my CD-R archiving on hold.
MY EXPERIENCES
I have been archiving material on tape for thirty years, and on cd for about three years, and I have 130 discs of mp3 files. I am starting to get failures (partially or completely unreadable discs which could previously be read by any computer). When the disc is put into the computer, the machine either crashes or says the equivalent of "disc not readable". In addition I have a number of discs which work but which are becoming discoloured. This doesn't improve my confidence. I have always bought what I believed to be good quality discs, but even TDK, Maxell, etc. are affected. I am wondering whether my failures are freak discs or the tip of an iceberg. The mode of failure (sudden, rather than gradual, like a tape) is also worrying. There is no warning.
Detected failure rate (for CD-R discs which were correctly burned and which worked fine to begin with) over a three year period for me is 6 out of 130, or about 4%, equivalent to about 1.5% per year. There are some discs I haven't looked at for months, so the figures may be worse. This makes the 100-year longevity claimed by manufacturers (based on accelerated ageing tests) look doubtful. I will be updating my figures periodically at the foot of this page.
The failure rate of 1.5% per year compares with 5 bad tapes out of 7,000 for cassettes over the period 1974-2005. This refers to tapes which were correctly recorded but which are not listenable now. The failures come to .07% or about 0024% per year.
Minidisc may be an alternative for sound recordings, but my experience is limited to using 100 discs over 4 years (no failures so far). One possibility which occurs to me is to use CD-Rs for exchanging material (e.g. for digital cleanup), but to transfer valuable material onto minidisc for longer term storage.
-----------------------------------------------------------------------------
Barry Hill (ORCA):
The late Barry Hill probably had more experience of recording technology than any other person alive. Here are some remarks which he wrote a couple of years ago:
1.Commercially produced CDs are fragile; CD-Rs are more fragile. They cannot withstand light, small variations in temperature, or much attrition by normal wear and tear.
2. Most CD-Rs in use today are made as cheaply as possible to attract consumer purchase.
3. Yes, a CD holds more data than a minidisc.
4. O.R.C.A. has been conducting tests on sound recording media for forty years and has a vast amount of operating experience. O.R.C.A. keeps all of its master recordings (cassette, reel, DAT, and so on) in a controlled environment which extends the life considerably.
5. There is no such thing as a perfect sound storage medium, but minidisc is more reliable than CD-R.
-------------------------------------------
American Study, 2004 - N.I.S.T
The longevity and robustness of optical discs - CDs and DVDs - has again come into question as the rush to digitize all sorts of material and transfer it to DVD and CD gathers momentum.
In 2004, scientists at the National Institute of Standards and Technology (NIST) looked at CDs and DVDs to see how long digital information recorded on to them would survive. They concluded that most CDs and DVDs will last 30 years under ideal conditions, but many things can make them unreadable. Direct exposure to sunlight can do it; so can heat. Discs last longest when stored in plastic cases in a cool, dark, dry environment. Because gravity can gradually bend the disc, storing it upright like a book is best. The study also found that fingerprints and smudges frequently do more damage than scratches, and recommends handling discs by the outer edge or the center hole. (from www.science.gogo.com)
--------------------------------------------------------------
PC-ACTIVE
A recent issue has an alarming article: Headlined "Unreadable CD-Rs within two years", the writer, Jeroen Horlings, speaks of tests bringing unsettling results. The original article is in Dutch, translated by Ben Boonen. The following is a summary:
Valuable data on a CD-R is not always preserved for long. It seems that data on a CD-R made by particular manufacturers can become unreadable within 2 years.
CD-Rs of 30 different brands, containing data, were stored in their original packing in a locked cupboard for twenty months. After this time the discs were examined professionally. It was found that some were completely unreadable and others were partially unreadable. This was equally true of CD-Rs from well-known and less well-known manufacturers. (unfortunately I have not been able to find any reference to the percent failure rate - ND)
The article is examined in greater detail in the November 2003 edition of APC "Australian Personal Computer" on page 18. APC also states "Recordable DVD's are even more susceptible to deterioration". (http://darkwing.uregon.edu)
-------------------------------
New Scientist: 1 Mar 2003
Extract from article by Barry Fox, National Sound Archive, UK:
.............. "There's already evidence that CDs - the favourite recording medium of the moment - aren't as permanent as archivists had hoped. Their aims are honourable, but today's conservators might simply be laying down problems for their descendants".
------------------------------------
PC Advisor, Jan 2006
Kurt Gerecke, a physicist and storage expert at IBM, in an interview:
"Unlike pressed original CDs, burned CDs have a relatively short lifespan of between two and five years, depending on the quality of the disc. There are a few things you can do to extend the life of a burned CD, like keeping it in a cool, dark space, but not a whole lot more".
The discs degrade. Optical discs commonly used for burning have a recording surface consisting of a layer of dye. This isn't sufficiently stable to remain readable for more than a few years.
"Many of the cheap burnable CDs available at discount stores have a lifespan of around two years," Gerecke said. "Some of the better-quality discs offer a longer lifespan, up to a maximum of five years." (website - www.pcadvisor.co.uk)
From miscellanous websites:
...................In 1994 I read an article in the British music journal The Wire that claimed that compact discs have a life expectancy of ten years. I have seen references to an article in Scientific American making the same claim.... (from www.straightdope.com)
.....................only write on the centre of the disc, where there's no data stored; inks may degrade the information on the disc.
...............Manufacturers claim that CD-R and DVD-R discs have a shelf life of 5 to 10 years before recording, but no expiration dates are indicated on CD-R, DVD-R, or DVD+R packaging, nor are there published reports of tests to verify these claims. It would therefore make sense to purchase new discs as they are needed rather than to order large quantities and stockpile them. (www.clir.org/pubs/reports/pub121/sec4)
.................Few, if any, life expectancy reports
for these discs have been published by
independent laboratories. An accelerated aging study at NIST estimated
the life expectancy of one type of DVD-R disc to be 30 years if stored
at 25°C (77°F) and 50% relative humidity.
[ND: .... the accelerated
ageing test ....it's done at elevated temperature. It doesn't
replicate the main hazards a disc faces in the
collection of an amateur collector - wear and tear, scratches, fingerprints,
and being thermally cycled by the CD player. (ever felt the disc when you
take it out?)
I also have a fairly obvious question .......tests
which take a few weeks or months to carry out....how can they give
useful information
about CD longevity?]
...........some of the other things that I have read indicate that if you are really serious about keeping your data around for years, then you should reduplicate the cds every 2 years or so. For long term accessability you will also need to change the storage medium because current technology fades from use.
...........the whole point of archiving is to not have to retranscribe everything very often. Imagine if book paper had a shelf life measured in 10's of years or less and you were a librarian at the Library of Congress.
.......A PHOTOGRAPHER WRITES....When I wrote of my concerns over being able to read digital media five, ten, or twenty years down the road, there were folks who claimed I was afraid of technology. Yet the fact remains that reading today's disks in twenty years is a dicey proposition unless you're willing to continually move your image files onto new technology as it emerges. But I'm pretty certain I'll still be able to view a slide in 20 years just by pulling it out of the drawer.
So when I'm shooting for the magazine where deadlines and workflow are most important, I shoot digital. When I'm shooting for myself or for my personal image archive, I shoot film.
I'm not the only one who's come to this conclusion. Unreadable CDs and the cost of moving digital images every eighteen months to new hardware have now burned many photographers. (www.vividlight.com)
.....another photographer........Jim R, writes..........Be careful with archiving to CD-R. Its shelf life is not good...buying quality cds... that's hard to judge, especially from the stuff you can buy at a local computer store..... Who knows where they were made, since most of the brand names out there aren't even using the same manufacturer from one year to the next. Most of the stuff they sell is rubbish. CD-Rs can have life-expectancies measured in weeks if you buy cheap discs. Getting a few years out of slightly less cheap stuff is hopeful. I wouldn't feel confident in buying off the shelf CD-Rs for archiving anything longer than 5-7 years. If you want quality, go with something like the Mitsui Gold CD-Rs which, if stored properly, should be good for that 70-80 year lifespan, but cost around 90 cents apiece in bulk.
...another CD user writes................I have experienced data layers peeling away from the disc, as well as faded dyes that resulted in unreadable CDs (and this was on carefully handled Verbatim and Kodak branded media). As a result, I did some research a few years ago and concluded that Mitsui media was the best available, but it's hard to find.
.................My 18-month old 120GB, 7,200 rpm Maxtor HD packed it in
two weeks ago for no particular reason. I have partial backups on CD but
stopped backing up several months ago when I found out that the CD's
don't last. I DO NOT trust CD or DVD media for long term storage. Poor
media if
you ask me. I wish they had never invented the damn things, I put
lots of important data on them over the years just to go back later
and find it ruined and gone forever.
(www.christianbrothers.com --------------------------------------------------
........... the managing director of Vogon International.....
- a company specialising in data recovery - is familiar with CDs
becoming unreadable thanks to the experiences of his customers, one of whom
commissioned Vogon to retrieve pictures of his second honeymoon from a
failed six-month-old CD-R. "The dye layer was fading," Stevenson says,
"but we were able to recover most of the disk. But ..... claims of a
100-year archival life..... are unhelpful and misleading".
In the wrong conditions, such as sunlight or humidity,
a CD-R will lose its information. There will be no warning; one
day it will be readable; the next day it won't.
"CD-Rs should never be left lying in sunlight as there's an element of
light sensitivity, certainly in the poor quality media," says Stevenson.
"I wouldn't rely on CD-Rs for long-term storage unless you're
prepared to deal with them as recommended."
................ the National Archives at Kew.....
"Generally speaking, we don't recommend CD-Rs for long-term storage,"
says Jeffrey Darlington, a project manager at the Archives'
Digital Preservation Department. "We don't regard CD-Rs as an
archival medium. Most of the CD-Rs on the market are not of archival
quality." Instead of CD-Rs, therefore, the National Archives tend to
use magnetic tape rated for a 30-year life. Also, they are careful to
copy, check and re-copy to avoid losing information and this is also a
useful strategy for CD-Rs. But even doing this, there are no guarantees.
There's no way to assess CD-R
longevity at home. All you can do is check periodically. As for whether
manufacturers are guilty of using finger-in-the-air methods, Kevin
Jefcoate, the marketing and product management director at Verbatim,
says: "It's a bit more than guesswork because there's a lot of
scientific evidence to back it up." (He then goes on to talk about
"accelerated ageing" tests for working out the longevity of the dye...
I really can't see the point of these).
It's difficult to see why
manufacturers haven't left a few hundred filled data discs exposed
to the sun, date side up and label side up
in one of their factories, and reported on the progressive failure rate
as they age. It wouldn't cost much
and would be a lot more useful to the
average CD user. Perhaps they just haven't got around to it.
I wonder if anyone reading this has left discs in the
sun to see how long the information lasts.....
..........Jeroen Horlings
from PC Active magazine comments : "We see a lot of manufacturers and they think
that quantity is more important than quality. "The problem
will remain." There are no easy answers, but if you must put
sound or photograph files on CD for future use, I suggest:
1. Don't throw away your originals (tapes, cassettes, reels, black and white original prints, etc).
Nigel Deacon / Diversity website
----------------------------------
UPDATE: my disc failure rate for discs which were
readable after burning, but which are not readable now (Apr 07)
Discs used in period 2002-2007: 300; Discs
with lost data (wholly or partially unreadable): 15
the bad discs: unbranded,
unbranded, maxell, tevion, tevion, imation, imation,
imation, imation, maxell, maxell, imation, maxell, TDK, stationery box.
Best discs: TDK (100+ discs used; one failure)
Perhaps the last word should go to "Stevyn". I found
his note in a chat room, and he's talking
about the way he does his backups ...
....."I just write mine down as ones and zeros on paper.
It takes me a few months to do a full system backup,
but it would take the government years to accomplish
the same task. I figure I'll be saved by the statute
of limitations by the time they figure out what I've
been doing."
N.D.'s summary
2. Inspect regularly, and re-burn valuable items
(or if you can't
bear to do this, make multiple copies on different brands of CD and keep
them in different locations; for example, give them to your friends).
3.
Regard CDs as a temporary
"transfer" medium only. You might consider storing files on high
capacity hard-drives,
suitably backed-up. But whatever you do, don't buy cheap unbranded
CDs - they're trouble.
Discoloured discs which are still working: 2 (
Maxell, unbranded)
NOTE....I've done a 2012 UPDATE to this article.......
