I have backups on a backup hard drive and also synced to B2, but I am thinking about backing up to some format to put in the cupboard.
The issue I see is that if I don’t have a catastrophic failure and instead just accidentally delete some files one day while organising and don’t realise, at some point the oldest backup state is removed and the files are gone.
The other thing is if I get hit by a bus and no one can work out how to decrypt a backup or whatever.
So I’m thinking of a plain old unencrypted copy of photos etc that anyone could find and use. Bonus points if I can just do a new CD or whatever each year with additions.
I have about 700GB of photos and videos which is the main content I’m concerned about. Do people use DVDs for this or is there something bigger? I am adding 60GB or more each year, would be nice to do one annual addition or something like that.
Hmm I am keen for something that could be left in the cupboard for 50 years and still works when brought out.
What does it take me to do home tape storage? Do the tapes needs to be stored with climate control or are they pretty stable? Is it feasible for the average person to load the contents?
I’m thinking of pulling a suitcase out of the cupboard of all the baby photos, but digital files or photo and video.
Tape isn’t readable by normal people even if they found it tomorrow with a drive already configured to be used.
In 50 years good luck finding a working drive compatible with LTO4 when LTO32 is out (it’s backwards compatible only with previous gen).
Unless you write on the box “here there are the keys for 100k bitcoins” they’ll just trash the tape
Yeah from some other comments I think my initial plan (that I’ll research some more) will be:
This way I should get a chance to update storage medium as technology changes as well.
That is not gonna work - read up on byte rot.
Yeah since then I’ve been convinced I need two drives mirrored under zfs, which should handle that scenario.
Hard drives loose their data fast if not powered (within a few years),so do SSD based media. Furthermore the former are very suspectable to mechanical destruction, electromagnetic interference,etc. And even if for some reason your drives last that long there will be nothing to connect them to - you know how we connected hard drives 25 years ago? Via SCSI/IDE. Good luck finding a converter to these now. If you go back further you need ISA controllers for the drives.
This is a really bad idea. Really really bad, especially with the goal you want to achieve. Your data will be gone within 5 to 10 years.
The idea is that I’d swap out drives every 5 years or so. If USB A is no longer in use I’d swap out at that point for something newer. Plus the drives would be powered on every year for the update, it’s just the point that I stop doing it (too old/hit by bus/etc) that the clock would start ticking.
I do like the M-Disc idea though. Probably a similar price, and more in line with the shelf-stable solution I was looking for.
There are still problems with the hard drive solutions:
Powering up the drives for a short period does not help with error correction when sectors get compromised
As said before it is relatively risky as mechanical parts of HDs do not like to be moved only occasionally. While this problem has become less severe over the last years it still exists.
The updating will include copying from one drive to another - this process is highly suspectable to errors that might be correct with the right file systems - but it’s not a guarantee.
And the main problem: You want to achieve a long shelf life - which means you must consider periods of time when you might not be able to maintain the data. What happens when you are not able to do so? And your next of kin are not quite ready to go through your things? To give you an example: You copy your data on the HDs today, maintain the disk’s for four years and want to change disk’s in 5, which means in 2030. Sadly a weeks before you are able to do so, John,your neighbourhood’s stupid school bus driver hits you and you suffer a major traumatic brain injury. Even worse,you don’t die right away but suffer for another 5 years in a nursing home before a infection gets you. Your family meanwhile is not quite ready to get through your things as you are still alive, aren’t you? (For real,this is the case a lot) After your funeral it takes them another year to finally get through all your things. Now your drives haven’t been used for 7 years. Even worse,one of them slips through your next of kind hand and hits the ground hard. How big do you think the chances are the data is still available? I think we both know the answer. While M-Disks are also suspectable to damage there are hardened multi-disk cases that make them pretty much indestructible - nothing any HD case can ever achieve.
So, 50 years isn’t a reasonable goal unless you have a pretty big budget for this. Essentially no media is likely to survive that long and be readable unless they’re stored in a vault, under perfect climate controlled conditions. And even if the media is fine, finding an ancient drive to read a format that no longer exists is not a guaranteed proposition.
You frankly should be expecting to have to replace everything every couple of years, and maybe more often if your routine tests of the media show it’s started rotting.
Long term archival storage really isn’t just a dump it to some media and lock it up and never look at ever again.
Alternately, you could just make someone else pay for all of this, and shove all of this to something like Glacier and make the media Amazon’s problem. (Assuming Amazon is around that long and that nothing catches fire.)
Hmm damn. I don’t really think cloud is the right answer for what I’m trying to do.
I disagree that formats like JPEG won’t be readable in 50 years. I feel like there would be big demand for being able to read the format even if it’s been superceded, on account of all the JPEGs that still living people have.
Maybe I get a big drive. Each year I copy over files from the last year. Every X years I swap the hard drive for a new one, copy all data.
How can I tell if individual files get corrupted? Like the hard drive failed in that section, then I copy the corrupted file to the new drive, and I’d never know. Can I test in bulk? 50k+ photos and videos so far.
Checksums. A good filesystem will do this for you, but you can do it yourself if you want.
If you sync a drive with rsync or something periodically, it’ll replace files with different checksums, fixing any corruption as you go. Then smart tests should tell you if there’s any corruption the drive is aware of. I’m sure automated backup tools have options for this.
I specifically don’t want to be touching previous files on the drive, it should be addition only. So I may need to write a script to do the checks, or compare against a mirror drive. I can do this with the right filesystem, but I’m worried that if I use a filesystem not readable by Windows then it may not be layman-proof enough.
Then I’d go with FAT on a USB, which should be plenty portable into the future. You’ll want to replace it every 5-10 years, and check on it every other year or so.
That’s about as easy to use as I can think of. Decades down the road, physical media like DVDs and tapes may be difficult to find readers for, but USB is versatile enough that someone is bound to have access. Micro SD cards may also be a good option, as long as you keep a couple USB readers around.
I have a terrible track record with USB sticks, including completely losing a stack of photos because of a USB stick.
I’m now thinking the benefits of a nice error-correcting file system probably outweigh the benefits of using a widely supported one. So I might use a pair of mirrored hard drives with SATA->USB cable, then include instructions along the lines of “plug into my linux laptop to access, or take to a computer repair show if you can’t work it out”.
I’m not talking about USB sticks, I’m talking about USB drives, like a HDD or SSD. If you want to go with flash memory, I recommend SD cards because they’re small and cheap, so keeping a few copies isn’t particularly burdensome.
I wouldn’t trust any of these options to last a long time on a shelf though. Check them every year or two and replace every 5-10 years, maybe a little longer if you buy higher quality.
That’s basically what I’m planning too. But my use case is disaster recovery, basically as a cheaper alternative to paying for hosted backup for important, but recoverable information (e.g. ripped media). Everything truly important (pictures and documents) goes to hosted backup as well.
I’m largely relying on documents explaining how to access the backups. If I pass, I expect my survivors to either figure it out themselves or hire someone who can figure it out from my documentation.
Everyone is saying to avoid flash memory. It doesn’t store well.
Another suggestion given is M-Disc, which might be a better option because then anyone should be able to throw it in a CD drive and load it without having to worry about the format of the drive and things like that. And even if CDs are not that common anymore, I think people will still know what they are and be able to find a way to load it even in 50 years. Like if someone found a cassette tape today (I know they were common much less than 50 years ago, but it’s hard to find an example since records came back into fashion). Plus M-Discs are designed for long term storage, so I could worry less about bitrot and files getting corrupt etc. They are write once so I’m not going to write over existing content.