I have a lot of friends who read this blog who are quite smart technically. When they read this, they are bound to say “Whoa Bill, you really did some really stupid stuff!”. To this I will preemptively respond “You are correct”.

Anyway, here is my tale of data woe and stupidity.

Recently, work was kind enough to buy me a new SSD drive, so I took out the slower SATA drive and replaced it with the much faster SSD drive. Because the SSD drive is somewhat smaller, I decided to move my iTunes media library from my laptop to a 2TB external hard disk attached to my home iMac [1]. Because I am OCD about my system being as cruft-free as possible, I installed a fresh version of Snow Leopard on my new SSD rather than doing some sort of system restore (e.g. from Time Machine) of my prior system.

Because I was reconfiguring everything and because I was pretty sure that all my data was present either on my laptop or my iMac, I blew away the Time Machine backups that were also on the same external hard drive. This was mistake #1.

A couple of days later I realized that I had forgotten to copy over a couple of movies that I had bought from iTunes recently from my old laptop hard drive, which was now collecting dust on an office shelf since I had replaced it with the new SSD. One evening – a few days into Christmas vacation - at approximately 12:30 in the morning, right before I was about to go to sleep, it for some reason became very important that I get the movies off of the old hard drive. So I put the laptop to sleep, took out the SSD, put in the old hard drive again, and woke up the laptop.

Did you spot mistake #2? This was a doozy. I’ll repeat part of what I said above:

So I put the laptop to sleep…

That’s right, I hadn’t shut down the computer when I swapped in the old hard drive. Can you guess what happened? I don’t really recall because I was sleepy and I had had a glass of wine (or two) that night. I said it was vacation, so nightly wine consumption is inferred. I’ll call this sort of work in this sort of mental state mistake #3. All I recall was that OS X was quite confused, and so was I for a minute, until I realized that I had effectively performed a brain transplant on my Mac without shutting it down. I said “Oops” and shut down the computer and tried to start it up.

It wouldn’t start. I can’t recall the error but I’m pretty sure I saw a terminal instead of a GUI, which any semi-technical Mac user will understand implies that you’ve managed to fuck things up real good.

At this point I remember suddenly feeling significantly more awake and sober.

I shut down the computer again and put the new SSD back in. I got lucky and the only fallout was that it had to do some data integrity checking but was otherwise fine. At this point I decided to cut my losses and put the old hard disk back on the shelf and went to sleep.

Interestingly at this point, I hadn’t realized that I had just irreparably harmed my old hard drive. For reasons I don’t fully understand, I came to the half-assed conclusion that I had probably just not connected it 100% right and this is why the system wouldn’t boot with it, even after the proper shutdown/restart cycle.

A few days later my son asked me “Daddy, where is <episode such and such> of ‘The Clone Wars‘”? I looked and low and behold, it was gone. So were 10 other Clone Wars episodes. So were just about every movie and music album I had bought from iTunes in the previous several months [2]. Normally at this point I would look in either 1. the Time Machine backup, or 2. my laptop hard drive but I had blown away 1. and 2. was sitting back on my shelf in an unknown but bad state.

So I did some research and ordered a SATA-to-USB enclosure [3] for the old hard drive so that I could take another crack at its data without having to either 1. use it to drive my laptop, or 2. to have to crack open my MacBook Pro again [4].

When the SATA-to-USB enclosure arrived, I took the old hard drive off the shelf and stuck it into the enclosure and plugged it into my iMac. It didn’t auto-mount as you would expect from a healthy external drive, so I opened OS X’s Disk Utility. Basically Disk Utility could tell it was a hard drive formatted as HFS+, but that’s about all it knew. It bombed out on any disk operation. I momentarily anthropomorphized Disk Utility and imagined that it was looking down on me and my sorry hard disk with a mixture of scorn and pity. “I’ll do what I can, but for God’s sake Jim, I’m a doctor, not a …”

I digress.

I did some Googling on using Disk Utility to fix messed up disks, and discovered the “Repair” button. I immediately clicked this button. It performed some serious grindage, but ultimately said something like (paraphrased) “Your disk is b0rked. You should try to salvage anything you can and then format it.” Mistake #4 (which continues in the next paragraph) was that I started operating on the b0rked disk without understanding what I was doing, or why.

It was at this point that it finally dawned on me that when I had swapped in the old hard drive after only putting the laptop to sleep (vs. shutting down), it had received some pretty severe brain damage. My immediate (and current) theory was that when OS X woke up, it got terribly confused by finding a different hard drive that had a very similar on-disk OS configuration and did it’s best to fix things, but ended up corrupting things, because it was running in a state its developers would never imagine someone would be stupid enough to enter.

Anyhow, at this point I started to consider my options. I did a quick calculation and guessed I was missing about $200 worth of re-purchasable iTunes purchases, which is frankly pretty minor in terms of data loss – I could have lost something truly valuable like pictures of my kids being born. Also, I was pretty sure that the hard drive was actually fine from a hardware perspective so the loss was pretty minimal. At this point the data recovery operation became more of a personal challenge than a necessity.

At this point I also started to realize how careless I had been and how lucky that I hadn’t lost actually valuable data nor damaged my beloved new $700 SSD. So I tweeted a two–parter observing my silliness [5] and asked the Twitterverse for advice on advanced data recovery utilities. My Jazz buddy Jason Wagner immediately called out DiskWarrior, which upon examination, had a strong testimonial from Mac Übermensch John Gruber, which was enough to get me to fork over $99 for the software.

You might wonder if I performed some sort of cost benefit analysis before choosing to buy DiskWarrior. The answer is “no”; my rationale at this point was simply that I had become fixated on defeating the b0rkage and salvaging some data. Dammit. One sort of weird thing I noticed while purchasing DiskWarrior that would make sense about ten minutes later was its very strong terms and conditions along the lines of “I ACKNOWLEDGE THAT ALL SALES OF DISKWARRIOR ARE FINAL AND IN NO CIRCUMSTANCES WILL I BE GRANTED A REFUND”.

So I bought and installed DiskWarrior (painless), glanced at the manual, and ran it. Just like Disk Utility, it told me my disk was FUBAR and suggested contacting Alsoft technical support for more options [6]. This is obviously why all DiskWarrior sales terms and conditions are so strict on finality of sale – you only buy the thing if you’re desperate with a data loss situation, you probably only ever use it once and it’s pretty deeply unsatisfying if it doesn’t work! But I didn’t actually get upset at all – it was more a feeling of resignation. I knew I had screwed things up real good and I had heard from enough smart people at this point that DiskWarrior was good software to figure that the data on this disk just wasn’t going to be salvageable without probably pulling in data recovery experts, which wasn’t worth it for $200 worth of re-purchasable iTunes content.

I threw one last hail mary by sending a note to Alsoft technical support, but in the two days of high-latency email exchanges [7] I discovered a path to recovery for all of my missing iTunes content.

Over the Christmas break I had a quite long support email chain with a very helpful iTunes support person named Marlee on a peculiar iBooks problem. Marlee had been so helpful and friendly on the iBooks issue that I thought I would piggy-back my data loss issue on the iBooks email thread. I was under the impression that you have to repurchase any iTunes music/movies that you lose, so I thought Marlee would be doing me a favor if she did anything, on the basis that I was very helpful and friendly on my side of the iBooks issue.

ANYWAY, I replied to the iBooks email thread with a very friendly and self-deprecating admission of data management stupidity, and embarrassed request for a special favor to help me get back at least the Clone Wars episodes that my son was missing greatly [9]. For reasons I don’t understand Marlee didn’t respond but rather a very friendly Apple employee named Raghavendra responded telling me that if I would just tell him my order number for the purchase, he would post it once again to my account [10]. I was a bit embarrassed given that I had sent a somewhat personal note to one iTunes support person and received a response from an entirely different iTunes support person but this embarrassment was trumped by my glee and surprise that I was apparently completely wrong in my assumption that I had to repurchase all of the lost iTunes content.

Short end of an overly-long story: I formatted the old laptop hard drive (no problems), sent additional emails to iTunes support to get my other lost content (no problems), and ordered an additional 2 TB external backup drive for a to-be-formulated-but-surely-more-rigorous data backup strategy.

I guess these are the takeaways for me:

  • Even if you’re fairly technical, err on the side of extreme caution whenever data is involved, especially if you’re messing with your data backups.
  • If you realize you have a data situation, do nothing until you think it through, despite the strong urge to “do something”.

Footnotes:

  1. Yes, my iTunes media library theoretically should have always been on a home computer, but that’s another story.
  2. I still don’t understand why only the last several months of iTunes content was missing. It should have been with the rest of the media on my external hard disk, but I guess that’s the sort of reasons that causes us to ensure we have multiple copies of things as part of our backup strategy. It probably has something to do with the overly complicated move of my media library from one location to another. iTunes is definitely not optimized for this operation.
  3. I ordered this SATA-to-USB adapter/enclosure. It worked well.
  4. A MacBook Pro is only slightly easier to open than a bank vault.
  5. I try hard not to do silly things but am quite wiling to laugh at myself when I do.
  6. To be completely fair to Alsoft’s DiskWarrior product, there is a chance that had I not mindlessly run the Disk Utility “Repair” function on the b0rked disk, DiskWarrior might have been able to fix it. Based on later reading of DiskWarrior manuals, it became clear that the more you try to fix your b0rked disk before you let DiskWarrior take a shot at it, the more likely you’ll do additional damage. Bottom line: When you’re trying to recover data, think very carefully about your options before acting as your odds of successfully recovering your data might drop with each successive flail.
  7. The latency in the email exchange was a function of them being six time zones ahead of me (I think) and also my being very busy with an OSLC presentation [8].
  8. More on the topic of presentations in a future entry.
  9. Not because he actually really wanted to watch them badly, more just because he didn’t like the thought that they were missing.
  10. For reasons I don’t understand, iTunes support’s only failing is lack of a “Search” function through customer order histories, instead relying on the customer being able to track down order numbers themselves. I can only assume that Apple doesn’t think such a function is important in the grand scheme of things, otherwise they could have implemented it like eight years ago.