Critical Data Backup

Discussion in 'OT Technology' started by Grelmar, Mar 31, 2006.

  1. Grelmar

    Grelmar New Member

    Joined:
    Jan 25, 2006
    Messages:
    88
    Likes Received:
    0
    Location:
    Calgary, Alberta, Great White North
    Taking this as a tangent off this thread:
    http://forums.offtopic.com/showthread.php?t=2440442

    Backing up that data you want to keep essentially forever, is an issue I've been grappling with, off and on, for 20+ years. As more and more information gets digitized, the issue just becomes more pressing. It's not just documents anymore, it's photos, home movies, financial data, and on and on.

    So how do we make sure that info is going to be available to us 10, 20, 30 years down the road?

    I've "evolved" a strategy over time. I'm not going to say it's a perfect strategy, but it's one that's working well for me right now. Feel free to challenge me on anything I'm about to say:

    Possible Strategies:

    Long term mediums:
    In this we'll include things like CDs, DVDs, Tape Backups, and Zip Disks (yep, people still use 'em.)
    Tape drives, in various iterations, have about the longest history for long term storage, and pretty durn good track record too. A good tape drive backup system can run pretty much perpetually in the background, and as soon as one tape gets filled, you simply insert a new tape. This is a very mature technology, and properly stored tapes can last almost indefinitely. Downside: They're slow. Slow if you want to do a "mass backup" all at once, slow if you need to recover data from them. They're also very format specific. There are a pile of different tape drive systems out there, and they'r not great at talking to each other. If you're backing up things to tape, for the longer term, you have to make sure you hang onto the hardware that created the tapes in the first place.

    Zip Drives are about the second longest running candidate, again with a proven track record. I have an external iOmega Zip Drive with RS232 serial connection I've floated forward from machine to machine for over 10 years now. The drive itself is a tank, and so are the disks. Really hardy. Much faster than a tape drive, and can be used in pretty much the same way. Downside: Platform dependance. Mac Zip Disks are hard to read on a PC Zip reader, and vice versa (though not impossible). Lack of popularity. I really just don't see them much any more. Expense. Zip Disks ain't cheap.

    Optical Disks (CD/DVD) are the relative newcomers of the field. Debate is going back and forth about their long term durability. They're a lot faster than the other two mediums, both to create and to recover from. Not exactly suited to "background" backup process, more of a burn on a regular basis, store it and forget it type thing. Relatively cheap way to go. Blank CDs and DVDs are cheap like borscht. Downside: Fragility. I've lost track of the number of ways you can instantly ruin an optical disk. Scratching them, heat warpage, chipping the edges, are just the beginning. Long term, you have to worry about laminar degridation, and there are a couple of known fungi that eat away ath the things. I have a friend in the Phillipines who claims that in their climate, a CD left unattended, sitting in the open on a desk, will become useless in as little as 6 months, or on a shelf in it's case, they don't expect them to last more than a couple years. Outside the tropics, this is less of an issue, but you have to pay attention to how you store them if you want them to last.

    Floppy Disks: Had to mention them. Useless, dead technology. Let's just move on.

    Permanent Mediums: Basically, print the stuff out on paper and file it it a fire proof box.
    Good for things like tax records, and maybe photos. You could also use microfilm transfer (I've seen it done), or other types of film. Cheapest way to go, on a small scale. Prohibitively expensive on a large scale. Who want's to build a bank vault in their house for long term storage of massive amounts of documents and pictures? Also, different inks and paper have different decay rates. The inks and chemicals in paper can also interact in strange and unpredictable ways, especially when you add in atmospheric chemicals to the mix. Film is no better. Bulky, and prone to decay. Film also "dries out" over time and becomes increasingly fragile. Neither one of these is particularly useful, IMHO.
    Encased Drive Units: Basically, hard drives and flash drives. A rewritable device with a fixed amount of storage.
    Flash Drives: Good for short term back up and transfers. Pretty new tech, relatively speaking. Kinda on the expensive side, too, if you're looking a $/Byte. I've also yet to see any credible data on the long term viability of a Flash Drive. The tech is too new, too unproven, to be genuinely considered for long term critical backup.

    Hard Drives: Highest speed large volume storage out there. Very reliable, in the short term. I've yet to have a hard drive burn out in under 6 months :x:. Can be made portable with an external enclosure, which can also allow you to connect it and disconnect it at need if you're using it as a backup system. Biggest reliability issues come from long term heavy use/wear and tear. Also, because of the nature of how they're used, can be prone to data corruption, mechanical failure, and pro-active degredation (viruses, accidentaly typing "delete" or "format".)
    My Current Strategy:
    Hard drives. Two large (250GB Drives) in the machine for software and files. One large (250GB) USB drive for long term storage of critical data. The basis of the strategy is that no critical data is stored on only one drive. It is stored on both an internal drive, and the external USB drive. The odds of both drives failing simultaneously I consider to be remote in the extreme. This allows me to back up data quickly and efficently in large volumes. I can also set it to perform the back-ups on a scheduled basis. I also don't have to worry about the long term viability of the technology. As hard drive volumes and speed increase, I simply move things forward onto newer drives. I can do this at a negligible cost (1 drive per year at around $100 each, or 28Cents/day if you like). Also, by automating the backup process, negligible time commitment, with the bulk of the time coming annually when I swap drives around.

    Seems to be working, though I've only been doing this for about two years now.

    Anyone else have any thoughts? Ridicule me for a fool, if you like. I'm open to learn.
     
  2. mdaniel

    mdaniel S is for Shiksa

    Joined:
    May 6, 2000
    Messages:
    52,501
    Likes Received:
    315
    Location:
    Northwest Mejicooooooo
    I wouldn't even mention Zip unless it was grouped with floppies under "Useless, dead technology. Let's just move on." They're still vulnerable to magnetic fields and lets not forget the click of death. They were cool in their day (1998) but its over.
     
  3. Yep

    Yep Knick knack paddy whack, give the old dog a bone

    Joined:
    Jan 22, 2001
    Messages:
    4,603
    Likes Received:
    0
    Location:
    South Jersey
    What happens if there is a fire and your house burns down? Do you have an offsite backup?
     
  4. crontab

    crontab (uid = 0)

    Joined:
    Nov 14, 2000
    Messages:
    23,454
    Likes Received:
    12
    this for home use? use whatever your budget allows. cd, dvd, hdd, whatever.

    for business, even home business, tape. end of discussion. hard drives are not a reliable medium for archiving data. Send your tapes offsite too. Someone elses house. Better yet a safe deposit box at your bank.

    tapes aren't slow either. Avg throughput for semi-newer technologies nowadays is ~30 MB/s or 100 GB/hr isn't too bad for one single drive. There are higher end drives that move 120 MB/s, 432 GB/hr, but these drives cost as much as a house. But regardless, there are many ways around the older slow backup performance.

    Your current solution for home is perfectly fine.
     
  5. crontab

    crontab (uid = 0)

    Joined:
    Nov 14, 2000
    Messages:
    23,454
    Likes Received:
    12
    Under that notion all storage medium except optical will be under dead technology. HDD's and tapes are also affected by magnetic fields.
     
  6. mdaniel

    mdaniel S is for Shiksa

    Joined:
    May 6, 2000
    Messages:
    52,501
    Likes Received:
    315
    Location:
    Northwest Mejicooooooo
    So when was the last time you saw a Zip drive? Yeah there's a few floating around but they're basically deader than the floppy.
     
  7. EvilSS

    EvilSS New Member

    Joined:
    Jun 11, 2003
    Messages:
    5,104
    Likes Received:
    0
    Location:
    STL
    The biggest problem is that unless you migrate to new archive media every 10 to 20 years or so, most likely your media will outlive your ability to read it. I know people who have 8" floppies and even the old platter disks, but no way to read them. Hell, even 5.25" drives are hard to find, usually only used. What would most of you do if I handed you a MFM hard drive to pull files off of?

    So really, you not only have to consider how long your media will survive, but also how long the ability to read it will be easy to obtain. So what you are doing is the right thing. By constantly upgrading your archive media you are preventing both issues from occurring.

    Personally I archive to DVD-RAM, since it's actually very well designed for the job. But as time goes on I'll have to migrate away from it and on to a new format (HD DVD-RAM, for example). I use two sets of 2 discs, one I keep at work and rotate home every month or so. I sync my archive folder from my HD with each disc. I figure with up to 5 copies of the data I most likely won't loose it.
     
  8. Mikey D

    Mikey D New Member

    Joined:
    Oct 13, 2002
    Messages:
    6,666
    Likes Received:
    0
    Location:
    Royal Oak, MI
    T10000 :noes:
     
  9. Peyomp

    Peyomp New Member

    Joined:
    Jan 11, 2002
    Messages:
    14,017
    Likes Received:
    0
    I've been looking at methods of backup lately, and to me tape seems like a joke. Tape systems capable of archiving a big RAID database are more expensive than the cluster that runs the database itself. And its still magnetic storage, and its on-site. To me it seems like tapes are old, useless crap for most applications. Why not replicate the data on-site, and off-site? Then replicate it somewhere else if you're concerned. Keep as many live copies as you need, in geographically separate locations. And if you want something safe off-line, is a USB hard drive sitting in a film bag in a vault any worse than a tape? Do inactive hard drive platters degrade over time at a faster rate than tapes? The price of tape systems that can hold alot of data is totally obscene.

    Kryder's Law is killing the tape drive. It will be completely dead in 5 years. Save your money.
     
    Last edited: Apr 1, 2006
  10. crontab

    crontab (uid = 0)

    Joined:
    Nov 14, 2000
    Messages:
    23,454
    Likes Received:
    12
    Haha. All of your posts in here have no merit. This one takes the cake. Please leave, you have no idea.
     
  11. Grelmar

    Grelmar New Member

    Joined:
    Jan 25, 2006
    Messages:
    88
    Likes Received:
    0
    Location:
    Calgary, Alberta, Great White North
    Oh? Larry and Sergei don't believe in tapes either. They did the math, and figured multiple redundant clusters based on cheap hard drives and commodity PCs was the most economical way to run the largest privately owned distributed computing project on the planet.

    Just sayin'
     
  12. Mikey D

    Mikey D New Member

    Joined:
    Oct 13, 2002
    Messages:
    6,666
    Likes Received:
    0
    Location:
    Royal Oak, MI
    You couldn't be more wrong, while replication is a cool thing and all having a pipe large enough to do that in the required time frames is easier said than done.

    Of course the highest end tape drives and silo's are expensive but when you're backing up 20TB + a night you need something with that kind of speed and capacity to hold data. A hard drive isn't meant to be shipped offsite and onsite all the time, too easily damaged where as a tape can easily have an offsite copy created and shipped offsite. Tape couriers such as Iron Mountain are not gentle with tubs of tapes, i'd hate to see how fast a drive was damaged.
     
  13. Aimless

    Aimless Resident drunkey

    Joined:
    Nov 5, 2001
    Messages:
    2,534
    Likes Received:
    0
    Location:
    Wisconsin
    Reliable backup / enterprise backup = tape (onsite & offsite)

    /argument
     
  14. mysteryman

    mysteryman OT Supporter

    Joined:
    Mar 20, 2006
    Messages:
    1,448
    Likes Received:
    0
    For my personal data:
    -RAID 1
    -Backup to another workstation
    -Backup to external hard drives

    For my business data:
    -RAID where applicable
    -Backup to secondary drive
    -Offsite

    I have all data going back as far as I can remember. Never lost anything to a virus / drive failure / etc...knock on wood...
     
  15. dorkultra

    dorkultra OT's resident crohns dude OT Supporter

    Joined:
    Oct 14, 2005
    Messages:
    22,743
    Likes Received:
    27
    Location:
    yinzer / nilbog, trollhio
    for my dad's dental office, he makes a daily backup onto a flash drive and brings it home every night. then loads the backup onto his pc.
    each month a burn a cd with the latest backup on it and he keeps them in his safety deposit box at the bank
    i also have 2 hard drives that have been cloned off his main computer as backups.
    i figure it's the cheapest and best way for him to do backups of his data.

    as for my personal data, i've uploaded most of it to online storage and i keep a backup of my personal storage drive on another hard drive.

    if my house burns down, i could live with losing it all...i would have much more to worry about at that time
     
  16. Goonigoogoo

    Goonigoogoo Active Member

    Joined:
    Nov 30, 2004
    Messages:
    14,765
    Likes Received:
    0
    Well the company i work for backs everything up to tape drives and so do our clients. My only advice to you if you choose tape drives are go with SCSI drives, the IDE's are horrid. I for one hate tape backups.

    FTP's are a viable solution but that just means you're backing up to another hard drive offsite, all hard drives eventually break down, it's no a matter of if, it's when.

    My personal fave is cd/dvd but with that you are limited to small capacities.

    If you have data that important that you need 30 years from now then 1 backup solution is not for you, you may need 2+ solutions.
     
  17. bandwagon

    bandwagon Copy/Paste

    Joined:
    May 27, 2005
    Messages:
    1,501
    Likes Received:
    0
    Something you havn't considered: USE OPEN FILETYPES. What happense if the format you use is not commonly supported later, or the format gets closed down by the patent holder? Sure, it seems far fetched, but its a good idea to use open filetypes (tar, png, ogg, xml, ... ) when you can.
     
  18. Peyomp

    Peyomp New Member

    Joined:
    Jan 11, 2002
    Messages:
    14,017
    Likes Received:
    0
    And yet you don't actually explain how I'm wrong. Therefore you lose :)

    Tapes are a lousy value. I don't care to use them.
     
  19. Peyomp

    Peyomp New Member

    Joined:
    Jan 11, 2002
    Messages:
    14,017
    Likes Received:
    0
    Why is a tape superior to a hard drive, if both are inactive?
     
  20. Peyomp

    Peyomp New Member

    Joined:
    Jan 11, 2002
    Messages:
    14,017
    Likes Received:
    0
    Drives shouldn't have to be shipped off site all the time. You replicate at least once on-site, and once off-site. Handling a mirror of a DB < 1TB with moderate traffic doesn't need a very big pipe. In this instance there are service personnel that can rotate between USB 2.0 drives for backups. Therefore tape has no role in my enterprise. The value on tapes is horrible. As I pointed out before, data density in drives doubles for the price in time with Moor's law. Tapes don't have this going for them. It actually makes sense once your data gets large enough to ship entire workstations around, which is how astronomical data is handled cheaply and effectively.

    God forbid that you challenge what junior sys admins learned from their daddy's, though. In this case the strategy that I outlined works extremely well, and we save money and have better reliability.
     
  21. 4x4guru

    4x4guru New Member

    Joined:
    Feb 23, 2004
    Messages:
    737
    Likes Received:
    0
    Location:
    In HELL !!!
    Agreed.. this guy's posts are usually so far off base its not even worth posting a response to.

    He's a jr. admin... leave him alone.:mamoru:
     
  22. MrMan

    MrMan New Member

    Joined:
    Jul 13, 2004
    Messages:
    308
    Likes Received:
    0
    We should sticky this thread (if all the bashing/flaming are out).
     
  23. Grelmar

    Grelmar New Member

    Joined:
    Jan 25, 2006
    Messages:
    88
    Likes Received:
    0
    Location:
    Calgary, Alberta, Great White North
    Bashing and flaming aside, this has been a great thread. Lots of good info.

    As with any good discussion, I'm going to have to go over it a couple of times and re-arrange my thinking a bit.
     
  24. 5Gen_Prelude

    5Gen_Prelude There might not be an "I" in the word "Team", but

    Joined:
    Mar 14, 2000
    Messages:
    14,519
    Likes Received:
    1
    Location:
    Vancouver, BC, CANADA
    I'd say HD's are the way to go in the future, but they don't fit the bill right now. Tapes are cheaper than HD's per Gig - until that changes, stick with tapes.
     
  25. Mikey D

    Mikey D New Member

    Joined:
    Oct 13, 2002
    Messages:
    6,666
    Likes Received:
    0
    Location:
    Royal Oak, MI
    :ugh:

    Everyone's entitled to an opinion I guess.

    When you find a 500GB hard drive that costs $100 let me know and then you'll be right on tape value being horrible. P.S. STK tape media for instance can later be rewritten to a higher density as new drives come out to increase their data storage amounts and speed.

    9840A 20GB/40GB ->9840C 40GB/80GB
    9940A 100GB/200GB -> 9940B 200GB/400GB
    T10000 500GB/1TB -> Next generation when they are released will double capacity using the same media.
     
    Last edited: Apr 3, 2006

Share This Page