Why#
Moving data out of commercial cloud services is not a technical problem. It is a series of decisions.
What do you keep? What do you discard? What belongs in a permanent archive versus a working folder? Which files are yours to decide about, and which belong to someone else who will make that decision themselves when the time comes?
These questions do not have technical answers. The infrastructure exists to hold whatever you decide — but the decisions themselves require sitting with a folder full of someone’s documents and being honest about what it contains.
This week was that work.
Family#
The external drive that has been sitting in a drawer contains the complete digital history of everyone in this household. Eight hundred gigabytes. Photos from every phone anyone has owned since 2008. Documents going back further. Backups of backups, some of which turn out to be duplicates of other backups. Files that were clearly important once and whose purpose is now unclear.
The approach was to sort before importing. The principle is simple: importing disorder creates disorder. A document archive that contains everything — including irrelevant things, duplicates, and files that belong to people who haven’t been consulted — is not a useful archive. It is a more organised version of the problem it was meant to solve.
Some of the data on the drive belongs to other family members. Their folders were not touched. They will decide what they want imported when they join the stack. That is not a technical decision to make on their behalf.
The photos are different. A single import of the full library — close to five hundred gigabytes for one person’s Google Photos account alone — goes into a shared family photo system where everyone will eventually have access. That migration started this week and is still running.
Tech#
The photo migration (to Immich) revealed a constraint that planning had missed. The home server has a fast internal drive, but it is not large enough for half a terabyte of photos plus everything else the system already holds. The solution — moving the photo library to a larger external drive connected via USB — was straightforward, but the timing mattered: it had to happen before the second batch of imports, not after.
The first batch also demonstrated that the server’s memory limit is not generous enough for the combination of import processing and machine learning running simultaneously. The server froze completely during the first large run. A restart, a swap file, and adjusted resource limits for the duration of the migration were the response. The import resumed correctly — the tool checks what already exists on the server before uploading anything, so nothing was lost.
The document archive required a different kind of work. Not a tool problem but a classification problem. Three systems need to understand which documents belong where: a permanent archive for official records (Paperless-ngx), a working storage system for files that are still in active use (Seafile), and a knowledge base for procedures and reference material (Outline). The boundary between the first two is the most important distinction, and it is not always obvious.
A contract that is still being negotiated belongs in the working system. The same contract, signed and filed, belongs in the permanent archive. A tax return in progress belongs in the working system. The submitted return belongs in the permanent archive. The working system changes. The archive does not.
Getting that boundary right before importing anything required looking at several thousand files and making a judgment about each one. Six hundred duplicates were removed. Several hundred files went to a category that means: the person this belongs to should decide. Roughly a thousand documents are ready for the permanent archive. The rest will wait.
Project#
Two workstreams running in parallel produced the same observation from different directions: the hardest part of data migration is not the infrastructure. It is the judgment.
The infrastructure can hold whatever you give it. The photo system deduplicates on content hash — if the same image appears in three different exports, it is stored once. The document archive processes optical character recognition on everything it receives, making every file searchable regardless of format. These are solved problems.
What the infrastructure cannot do is decide which files are worth keeping, which person a document belongs to, or whether a folder of work files from a job that ended five years ago needs to come with you into a system your family will rely on.
Those decisions were made this week, one folder at a time. The work is not finished. The triage category that means “I am not sure yet” contains sixty-three items. The category that means “this belongs to someone else” is waiting for Part III, when the rest of the family comes online. The category that means “decide this yourself” has nearly seventeen hundred items that will require a different kind of attention.
The archive is being built to last. That is worth being careful about.