In the old days, this was somewhat different. My data were on punch cards backed up on magnetic tape. I didn’t know exactly where the tape was, but I could look through the big glass window and see the tape library, and if I requested a set of data files, someone would mount the appropriate tape reel onto a big tape drive. When writing ‘software’ (including SPSS instruction sets, etc.) I could throw data and software into “scratch” which was a space on a hard drive (not the same as today’s hard drives) or a drum (a drum-shaped hard drive about the size of a Smart Car). The “scratch” space would be deleted on a regular basis, so I would put stuff there specifically knowing that I would not have to clean it up later. It would just go away.

Instruction sets/software and data could be stored on cards, magnetic tape, unruly disks or drums, or even paper punch tape (the mid 20th century version of the USB stick). But no matter what, not only did I (and everyone else) pretty much know where the data were, we also made specific decisions as to where to put the data to accommodate the not very automated process of accessing it or backing it up.

When I got my first real desktop PC (not counting my TRS-80) I ordered it (from my brother, who had access to such things) with two 20 meg hard drives, in an explicit effort to control the “where” part of this process. I configured the computer like this:

Drive C:
Main DOS system and software files
Scratch space (temporary folders, files to delete at next opportunity)

Drive D:
Files to use and now and then back up

This way, I could back up an entire drive and thus back up my data without all those pesky program files. The system and software, in turn, was “backed up” by virtue of me having floppy disks to reinstall everything. In those days, re-installation was not a lot more time consuming than restoring from backup, and allowed for re-configuring software and getting rid of crap software that one regretted installing to begin with.

You may or may not think that this approach was wise, but that is not my point. My point is that I thought about how to manage my digital life with explicit reference to where my stuff was, in particular, my data files.

Most people do this to some extent, but as shown in the examples above, not really. A lot of people probably think that the CD’s they copied into their computer are “in iTunes” as though iTunes was at thing that stuff could be “in.” Which it is not. Those music files are regular computer files no different at the system level than your c.v. stored in a file called “resume_version22_1982.doc” or whatever. But, they, the songs, are located collectively in a set of folders that are in turn organized in other folders in some arrangement that makes it impossible for a normal human to figure out, using folder and files with non human-meaningful names. Same with your email, most likely.

So no, thinking “My music is in iTunes and my email is in Thunderbird” does not allow you to find, copy, delete, read, edit, or otherwise access any of your songs or emails other than by using said software, and then, you can only do with those files what that software allows you to do. Which might or might not include exporting it for use in another format, with exporting capabilities determined by things other than what is technologically possible. Which, in turn, is like having a car that occasionally refuses to turn left for marketing reasons.

The degree to which this disconnect between you and your files is true depends on the operating system you use, and the software you use. Linux/Unix is all about directory systems, and is the most straight forward. In theory, ever single file that is yours is in one directory called “home” or a subdirectory thereof. This is in contrast to Windows, in which your files are … somewhere else, and the system seems to be designed to make it as difficult as possible for you to find them.

This does not mean that you can easily find everything that you might consider a “file” or similar entity in Linux. There is a good chance that your email software uses some bizarro file that you can’t easily see inside of. (I use alpine which puts the emails inside a text file, but hardy anybody does that.) There are “hidden” files in Linux just like in Windows (in Linux, everything that starts with a “dot” (“.”) is automatically “hidden” …. meaning you can’t see it unless you “unhide” that which is hidden). There are other strangeosities as well.

This where to put stuff issue came up when I had a conversation with my brother the other day about backing up. We probably have different backup philosophies. My brother is an actual computer expert who runs other people’s computer and so on, so to him backing up means doing what you need to do so that you can restore the state of the system … software, settings, and data … accurately and quickly when something breaks. To me, backing up is securing the data files (including pictures, documents, whatever) and I do not really care about the software. Of course, there is an overlap. The programs and scripts I’ve written are data, not software, by this thinking.

Also, I have at least to kinds of data: Stuff that does not change and stuff that could change The stuff that does not change includes photographs downloaded off a camera. Yes, I can modify those but I’d generally leave the original intact. Also, all the documents, such as presentations and handouts, for a given class I’ve taught are archived, for various reasons. the original is left untouched and a subsequent year’s material modified from the originals (in theory).

So, there are two kinds of data: Archive and dynamic. In backing up, I can add to the archive without having to check if anything previously backed up has changed, because it hasn’t. But to do this, I have to have a space where I put archive material… a separate directory or device where such things go. That might seem dumb, like extra work, like I should have the computer just back everything up or otherwise automate the system. But consider this: Not counting .iso files (which are a whole ‘nuther issue) I have about 194 gigabytes of “archive” material and 10 gigabytes of “dynamic” material. Is it really smart for me to run a ca 200 gigabyte backup every day, or even every month? No, of course not. Better to have the dynamic stuff backed up all the time, and to maintain the archive in a much slower, more ponderous, but still effective manner. If I’m using an online backup, I don’t need to verify that 200 gigabytes of data is the same as it was yesterday, or last week, or whatever if I know it is. Also, any change in archive is not routine … it is something broken. Once something is in the archive, the copying is always one way, out of the archive from a given source. My archive is worot (write once, read only thereafter).

So this all leads to two pieces of advice: 1) Make choices that allow you maximum direct access to your files (like using Linux instead of Windows, or appropriate choices of application software) and 2) Divide your files and stuff into categories that have to do with the nature of the backup and your access to it, and then by topic. For me, my desktop is my scratch space, Dropbox is my dynamic file space, and I’m not going to tell you where I keep my archive. This means that I have photographs in two places: The archives (downloaded off a camera, left alone) and the various cropped or otherwise revised, and in some cases simply selected and resampled or compressed photos. I have course material in two places: Archived away but accessible, and dynamically changing until I’m done with it (then it goes in the archive).

How do you store your files? How do yo do backups?

ADDED: It has been pointed out (below, in comments) that iTunes now organizes your files in a sensible human readable way. Good for iTunes I was working with older information.

The same commenter suggests that email must also be organized that way. Don’t believe it. Especially if you use Outlook!!

In any case, it is still true that your files are where the secondary software puts them, not where you want them, necessarily. Which is not an entirely bad thing.