Managing Your Photography Data Chaos: A Personal System for Efficient File Structure & Secure Backup
“An oil painting of a photographer surrounded by the chaos of unbacked up data & forgotten images” – DALL-E 2
Two year ago I worked off a single hard drive then one day I plugged that sucker into my computer & nothing popped up.
I ended up losing a bunch of stuff & it sucked – after that, I sat down & worked out a system that I was happy with – this is it.
There are two parts to this:
File Structure & Storage/Backup
The file structure helps me index images & means if something like a Lightroom file corrupts (which is has) I’m working within something which makes sense, not digging through thousands of images in a single folder called something along the lines of ‘hfhfhfh’
Below is an example of how I sort the images
YEAR > MONTH > DATE & NAME OF PLACE SHOT > CAPTURE
(I add the corresponding number e.g “3.March” so that if it’s sorted by name it goes from top to bottom)
In terms of the last three you see within the folder (Capture, Selects, Working) the way I work with these is by importing images & video into ‘capture’ (generally via Lightroom), once I’ve selected images moving those into the ‘selects’ folder & when I start editing in Photoshop I save that working file within ‘working’…I’ll also sometimes add a ‘proof’ folder within there where all final images as .jpegs will go in
This structure helps with keeping track of back ups too because it’s as simple as only needing to duplicate that new folder from a recent trip into the various backups e.g when I get back from a trip & create a folder full of images let’s say “26.Kosci Mount Sentinel” only that folder needs to be duplicated into the various external drives.
Speaking of external drives – let’s have a look at backing up…
Image files aren’t generally too big & SD cards aren’t generally too expensive.
Because of this Once an SD card is full instead of wiping it I just change over to another & keep that full SD card as a backup of the raws with the dates listed. If EVERYTHING fails & I know the general date of a trip I’ve lost images from I’ve got a backup right from the word go.
Photos get imported into one External HHD drive, this is the drive I work off. I’ve also got two others which I duplicate files onto (within the previous mentioned file structure). One of these I store in a safe & the other offsite (aka my parents) the latter of which won’t always get duplicated onto as often. This gives me 1 working drive & 2 other backups
The last backup is cloud storage, more specifically – Backblaze.
I’ve got this setup on my laptop to back everything up including external hard drives I specify – while that drive is plugged in it’s automatically copying those images of pretty flowers & mountains I’ve just taken & storing them into the cloud. The only downside to the backup is initially you need to have that drive plugged in for a large amount of time because it first need to create a full copy of that drive, after that you’re all sweet.
I’ve used Backblaze for a while & just signed up for their affiliate program after writing this article so if you sign up feel free to use this link – I’ll get a small commission
I’ve had 2 drive failures since adding this system
The first time my working drive died & I was able to buy a new external drive & use one of the backup drives as a replacement
The second time I hadn’t remembered to duplicate files from a recent trip onto any drives & Backblaze saved my ass in that case because while I was working away at editing an image it was quietly backing up to the cloud
Q. I use XQD cards, I can’t fill these up & put them aside – they’re too expensive!
A. Easy – you can transfer them onto a cheaper SD card
Q. What happens with Backblaze when a drive fails, don’t I have to re-upload everything again?
A. Nope! Backblaze has a cool thing called ‘deduplication‘ “Once you select the new drive with identical data, no data will be reuploaded to your backup just remapped, as our deduplication process prevents data reuploading”. When doing this 3.5TB of data took about half a day on a 25mbps upload internet connection
Q. What happens if I want to edit old images? Do I have to remember where these working files are within the older folders & make sure I’m duplicating/ keeping track of those too?
A. Sometimes I’ll just add a global ‘#working’ folder (you can see in in my folder screenshot) where all these edits go. When I go to do a backup I just make sure to move files within that working folder also.
Q. What if I work off an external SSD?
A. Great! That’ll be super fast for editing but most external SSD’s don’t have a huge amount of space or if they do it’s super expensive. You can still work off one but that probably just removes a backup. Maybe buy another external HHD for backing up?
Q. What if my total storage is over 5TB?
A. Probably a Raid system setup on a NAS is the way to go then because currently, HHDs seem to max out at around 5tb…I haven’t crossed that bridge yet. I will note also that as far as I’m aware Backblaze doesn’t offer support for drives accessed via a network as it could be very easily abused but do seem to have native support for some NAS devices
Q. What drives do you use?
A. For the 3 drives I work with they’re all Seagate Onetouch 5TB drives – essentially the cheapest 5tb drive I could get
A note on Lightroom Catalogs
I’ve intentionally been vague & not mentioning catalogs within Lightroom because I bounce from using Lightroom to Adobe Bridge or just looking at folders themselves but I do also run a Lightroom catalog backup & duplicate that into the external drives too