Backups, Archives and Lightroom – a few thoughts

I recently needed to find a few images I took more than ten years ago. And discovered it was surprisingly hard to locate them. Not only had I long archived them out of the current LR catalog, but that “Archive” Lightroom (LR) catalog was long abandoned itself, on a drive that I only had one copy of remaining of.

Add to that a “turn the office upside down” style search for the unique power cable for that particular drive, then a discovery that old drives start to randomly lose data and suddenly I’m thinking it might be a good time to rethink my archiving strategy. In a hurry!

So this blog is just a way of passing on a few of the thoughts I’ve had while trying to sort this.

  1. Keep the data moving!
    Whatever the storage medium, nothing lasts forever. Some CDs or DVDs are prone to disk rot or scratching, hard drives (HDDs) fail, USB drives lose or break power and data cables (and their HDDs fail), cloud storage can be a route for hackers, or you may not always have internet connection. So whatever the current “best practice” seems to be, make sure you’ve moved the old stuff onto it. Yes, even the really really ancient stuff like that which you first had on floppy drives…
  2. Don’t let your hardware become obsolete
    Hmm, that floppy drive, that seemed to be pretty durable if rather small. Oh wait, too late, might not be possible to retrieve info from that now. That’s the other problem with old media see, sometimes even if the media is still fine, the hardware or software needed to read it has become so obsolete it’s useless. Remember Zip drives too? A really great idea – a rugged floppy that held 100x the data. Try finding a drive that’ll read one now…
  3. Lightroom Cataloges are a PITA when they get too big!
    Having archived the really ancient digital images off into an “Archive” catalog several years ago, I tried to keep everything all in one catalog. A good idea in terms of keywording and being able to find things without having to search for where you stored that archive catalog. But a real pain for size of preview files, and slowness of catalog.
    My current strategy is now to keep just the latest two years of images in a “current LR” catalog. And everything else in an Archive catalog – which has no previews prebuilt. Less easy to just browse through the archive, but the previews will generate when needed and it’s better to search on keyword anyhow… Well it will be once I’ve got all the images fully keyworded. That will take a long while, so really I’m reliant on remembering when any image was likely to have been taken and using my old file structure (by date) to find it. Not a problem with older images, but something to move away from now.
  4. Export and import in LR Catalogs
    Exporting LR catalogs seem to be problematic. And once you split a LR Catalog, it’s keywords can grow independantly. Which means that as each new year gets added to the archive you may have some keyword reconcilation to do.
    My main way of getting around this is to take a backup copy of my curent catalog, and rename it as Archive LR. It’s now a seperate catalog but contains all my keywords to date. Then I add in the old catalog archive to this Archive LR catalog and remove images from each catalog so the current one only has my most recent two years worth, plus those I’m working on currently and recently processed images. And my archive has everything but those.
    And at the end of the year, I’ll have to hope that “export as catalog” works a little bit better from a small catalog – or find a way of flagging “picks” in an alternative way (that info is only held in the LR catalog, not the EXIF metadata), then drop them from the current catalog and reimport into the archive one.
  5. Why not just export the older images to the old archive catalog?
    Two reasons. One the old catalog had to be updated to the latest software version – it was on LR2 – that’s how old it was! Second, trying to export as catalog from the current catalog was proving very difficult, error prone and time consuming. By doing it this way, there’s less data to move between the two catalogs, so fewer errors – I hope!
  6. Keep three copies of every image you want to keep!
    I thought two copies was good enough a few years ago. And a lot cheaper and less hassle. But my recent experience suggests that the industry best practice of three copies is worthwhile. With two copies, if one gets lost or destroyed, you’re down to just one copy. Which may also fail at any time. With three copies, one can be kept offsite (or in the cloud if you prefer/have enough cloud storage space/high enough bandwidth internet connection) so if any physical disaster happens to the building your main storage is in, you’ve a backup elsewhere. And if something happens to one drive, you’ve still got two chances of recovery!
  7. Internal drives are the way forward.
    OK, this isn’t news. I’ve been using extra internal drives for several years now. They’re cheaper, and don’t have the problems of lost/wrong/broke cables, slowness and bulk that external USB drives do. They’re also just about possible to transfer between a PC and laptop with a seperate docking station. 3TB is currently the best amount of space for not too much dosh. I buy three identical drives at a time and use Beyond Compare to manually keep the copies in sync with each other.
  8. DO NOT use proprietary backup software.
    Particularly for JPG images. This will often compress your images further, (remember JPG is a lossy format of compression – every time the JPG image is saved it loses more data), and imho is always a PITA to recover things from as individual files are not visible and itall too often becomes an “all or nothing” reversal to a previous state of computer. In the way that windoze system restore is highly variable in whether it actually works or not. Manual syncing is more effort, but gives a much better chance of actually being able to recover a lost image later on! Oh and backup software itself gets obsolete very quickly – manual syncing software far less so.
  9. This one should be obvious, but… Make sure the drive your LR catalog on has plenty of spare space! Lack of disk space makes the catalog run very slowly and prevents backups.
  10. Allow lots of time. More time than you want to spend… But it’s better than losing images…