Apple releases fix for iPhoto '11 - in the most confusing way possible

Update: On Sunday, Apple fixed the confusing headline in their support article described below. It now correctly says “Update iPhoto '11 to 9.0.1 before upgrading library”. Much better!

---

 Apple has just released an update for iPhoto '11, which they say “in extremely rare cases, could result in data loss when upgrading a library from an earlier version of iPhoto.”

I can't deny that the language really rankles. Frankly, I think they should be worrying more about their customers' photos that were lost and less about how it sounds. There should be a press release that alerts people to the problem and to the solution to make sure that no one else loses this very precious data.

And it doesn't help that they've done it in the most confusing way possible. The headline to the support article says “Upgrade iPhoto library prior to iPhoto '11 Update”:



But that's exactly what you should not do! Indeed, the whole rest of the article is about updating iPhoto '11 before upgrading your iPhoto Library, since upgrading your iPhoto Library with the originally shipped iPhoto '11 is what is erasing people's files.

This is made worse by the fact that they released an update called "9.0.1" for iPhoto "11". It turns out that iPhoto '11 is really version 9.0 (while the last version, iPhoto '09, was really version 8). It's confusing Marketing speak and right now it's pretty annoying. It makes you think that you should go back to iPhoto '09 and update it before installing iPhoto '11. That's not the case. To find out what version you have, choose iPhoto > About iPhoto.

The other problem is the confusion between “updating” the software (iPhoto itself) and “upgrading” your library of photos (the iPhoto Library package, of which there may be more than one, it may have some other name, and is usually found in your Pictures folder).

When you install iPhoto '11, it doesn't automatically or immediately touch your library. So, it's safe to install iPhoto '11 from the disks, then update it with Software Update or by manually downloading and installing the update. Your library will not yet be affected. You should definitely do this first, regardless of the headline of Apple's support article.

Once you open iPhoto '11, it will ask you if you want to upgrade your iPhoto Library. Here's where a lot of people (oh yeah, I forgot, extremely rare cases) had trouble. Make sure you have at least one and preferably two or more full backups of your library or libraries before proceeding. And make sure you've installed the update.

Apple says that updating your iPhoto Library can take “up to an hour or more” and that even if the program looks like it's hung, you should not force quit. Be patient. And contact AppleCare if you get stuck.

iTunes invalidates EPUB files for iBookstore???

Second update:
OK, thanks to more information from Andrew, I've figured out what the problem was. Although it's true that iTunes annoyingly adds that iTunesMetadata file to your original EPUB file when you copy the file to iTunes, that's not what's causing the invalidation. But if you use iTunes to add a cover to your EPUB, as described in this video by Terry White, and then upload that EPUB to iTunes Producer, that EPUB will not validate and thus will not be accepted into the iBookstore, because the cover image is not properly contained in the OPF file.


Instead, the cover should be included in the EPUB file either via Pages' "Use first page as book cover image" option, or via the instructions in my book.

---

First update: I've just been able to upload an EPUB to the iBookstore with the iTunesMetadata.plist file intact, so I'm not sure exactly what's going on. It's not in the iBookstore yet, but the iTunes Producer says it validates, despite the iTunes file. I'll let you know what happens. It looks like I may have jumped to conclusions that the iTunes Metadata file was the culprit in this story. (It does continue to bug me that iTunes changes my files without asking!)

One of the things that this issue brings up is how hard it is to test things in the iBookstore. First, you have to sign up (which requires a US Tax ID and US Bank Account valid iTunes Store account). Then, you have to have an ISBN for your book, which in the US can cost as much as $125. And finally, you have to actually have the real content of a real book—not a test, according to Apple's rules—to upload. For many of us who are testing the waters and don't yet have the final product in hand, these are difficult impediments to overcome. Apple, couldn't you make it easier for us to test our EPUBs?

---

I got a note on my blog the other day from Andrew Brooks who is publishing an enhanced ebook to the iBookstore. He has used Apple's Pages software to create the EPUB and even included enhanced elements. But when he went to upload the EPUB file to the iBookstore, he got this error:

“An Apple requirement is that all the files in the ePub be listed in the OPF file. So, the error was that there was an unmanifested iTunes artwork file.”

Little did he know that when he opened the EPUB file in iTunes, iTunes had the audacity to add a little file to his original EPUB document. No, not only to the EPUB that he copied to iTunes, but also to his original EPUB file. The file is called "iTunesMetadata.plist". You won't see it unless you look inside an affected EPUB.

To add insult to injury, when Andrew tried to upload his EPUB to the iBookstore, his ebook was rejected because of the aforementioned error! In short, iTunes adds an erroneous file to his EPUB and then Apple says he can't upload the file because of it.

It turns out that Andrew used iTunes to add a cover to his book, as described in this video by Terry White (and how people have been doing for records for a long time). Unfortunately, although the EPUB and cover will work properly in a local iPad, Apple won't accept it because the EPUB no longer validates. You have to add a cover to your EPUB either with Pages' "Use first page as book cover image" option, or via the instructions in my book.

As I told Andrew, that's pretty ridiculous. We're all testing EPUBs on our iPads and iPhones and we need iTunes not to invalidate our EPUBs while we do it.


There are two solutions. First, keep a clean copy of your EPUB—that has not been copied to iTunes—for uploading to Apple. Or second, crack open your EPUB after copying it to iTunes and eliminate the iTunesMetadata.plist file.


OK, and third, send a note to Apple and tell them that when you copy a file to iTunes, that does not give them the right to alter your original file. And shoot, if they're going to mess with your files, they shouldn't complain about it afterwards.


Note: I have not been able to test this myself, because I don't have the content for an ebook, and so Apple will reject anything that I try to upload based on the fact that it's a test. I'd love to hear if others have had this problem.  I uploaded an old story that I wrote in college to test this issue. I'm going to need some more content for future tests!

Wait to upgrade to iPhoto 11: Apple says fix is imminent

I just spoke to someone at Apple who called me all the way here in Barcelona to see if my data was OK and if I needed any help. I was pretty impressed. They know how important our photos are to all of us.

They confirmed that there was indeed a window during the updating process in which a force quit could “create data loss”. I asked if I would have had the same problem without force quitting and the answer was probably not, though they didn't really specify how long I would have had to wait. There are people on the Apple Discussion Forums who have let their updates go for days with no apparent signs of life. I asked if RAM could have been an issue, and they said no.

They asked me if I had any projects created with older versions of iPhoto, and I said I had two from iPhoto 5. And they asked me if I had any video playback extensions for QuickTime (I don't). They also asked how big my library was (233gb).

I asked if changing permissions of the iPhoto Library was really helpful (as someone suggested on one of my blog posts), and was told that it was only part of the solution. Deleting old projects does not seem to have any effect whatsoever.

Mostly, and this was the most important thing, they told me that a fix from Apple was imminent and that we would soon have official confirmation of same.

It's probably best to wait for the fix, but if you can't, be sure to properly BACK UP your iPhoto Library before upgrading to iPhoto 11!

DON'T RELY on Time Machine for iPhoto backup before upgrading to iPhoto 11

I got more than 6000 visits to my website yesterday from folks worried about iPhoto ERASING their iPhoto Library, like it did to mine. People are listening. Which is good. But sounding such an alarm is an enormous responsibility that I don't take lightly. I am a very long-time Apple fan and iPhoto is one of my favorite programs.

But every time I think I might be being too harsh on Apple (maybe it was just me, maybe I really could have done something besides force quit when it hung and said it wasn't responding, maybe somehow my 233gb of photos really were shoehorned into 6.5gb etc. etc.), I go to the Apple Discussion Forums and read something like this:

"Well I am one of those who didn't have backup, so thousands of pictures and videos of my kids growing up is gone."

or this

I've lost all the photos of my new son, his Christening, my family over from Australia and my dad's 70th Birthday

and this last one is pretty awful:

iPhoto libraries are not backed up by Time Machine if iPhoto is running during the back up process. I leave my iPhoto application open constantly. I bought the best MacBook Pro money could by with 8GB of RAM so I could have multiple applications open for my design work.  This is an absolute horror show for me. 

and then I'm convinced I haven't yelled enough.

First and foremost, make a BACKUP before you upgrade your software.

And DON'T RELY on Time Machine. Although Apple says it works "in the background" without you having to do anything, it doesn't. I'm afraid I've been ranting about that for a while too. If iPhoto is open (and I leave iPhoto open all the time), Time Machine will skip your iPhoto Library, even as it backs up everything else.

You will think you have a backup, and you will be wrong.

Here are the steps:

1. Close iPhoto. (Again, Time Machine will not back up an open iPhoto Library.)
2. Run Time Machine, or other backup software.
3. Make a separate, additional, independent backup of your iPhoto Library onto an external hard drive, or other media, and store it off site (in some other building). Even if you can't figure out where to take it off site, don't let that stop you from making the second copy. Just drag the iPhoto Library to an external disk.
4. Look at your backups and make sure they have the same size as your original iPhoto Library. To do so, select the iPhoto Library and choose File > Get Info.

Only then should you attempt an upgrade to iPhoto 11.

What if you've already attempted an upgrade, and had your iPhoto Library disappear?

What worked for me was starting up my computer in Safe Mode (with the Shift key down), and then rebuilding Disk Permissions (use Disk Utility, it's in your Utilities folder in the App folder), and then I started up iPhoto with Command and Option pressed down. In the dialog box that appeared, I chose the options 1, 2, and 6. It then asked if I wanted to upgrade my library, and I said yes. And it worked. It took a couple of hours (my library is 233gb) but it worked. It's really slow for me, but so far, and I'm not convinced about the interface, but it looks like my photos are intact.

If that doesn't work, I've also heard of people

eliminating duplicate fonts from the Font Book
Remove DIVX codecs from Library/QuickTime or ~/Library/QuickTime
and even removing files from within the iPhoto Library package
using disk recovery software

Of course, you can also call Apple or post your particular circumstances on the Apple Discussion Forums. There is a wealth of information there.

Good luck. And keep spreading the word:

DO NOT upgrade to iPhoto 11 without a BACKUP!!!

Installing iPhoto 11

I tried to upgrade to iPhoto 11 yesterday, only to find that my 233Gb of photos had been reduced to 6.5Gb, and that my 50,000 photos from the last ten years were gone. Completely. And I was not alone. Reading through the Apple Discussions Forum is heart-wrenching. Post after post from folks who have lost thousands and years of photos. I have done my level best to spread the word and make sure that people have a solid backup (as I thankfully did) before upgrading. If you follow me on Twitter, you're probably sick of me posting about it. But I love my own photos too much to not do everything I can so that no one else loses theirs.

And I guess I love iPhoto too much to give up on it. I thought a lot about whether I really wanted to try again. I even gave Picasa a cursory look as Farhad Manjoo recommends. In the end, I felt like all my public grousing at least demanded that I try again and see if I could find a solution. OK, and I admit it, I still wanted to see it for myself.

Thanks to the folks in the Apple Discussion Forums, I found a solution that worked for me. I restarted in Safe Mode (by holding down the Shift key as the computer starts up). Then I went to Disk Utility and repaired Disk Permissions. After copying my backup of my iPhoto Library back to my computer—which took more than two hours—I launched iPhoto with the Command and Option keys held down. In the alert that appeared, I chose options 1, 2, and 6. iPhoto then asked if I wanted to upgrade my library, to which I said yes.

And then I just waited. And waited. I even left to run an errand. Several hours later, it was done. And all the photos are there. I'll let you know if it was worth it.

But don't forget to make a BACKUP before you try it!

WARNING!!! iPhoto 11 Erases ENTIRE iPhoto Libraries

I was very excited to hear about iPhoto 11. I'm a longtime iPhoto user, with more than 50,000 photos in my iPhoto Library that lately has grown to 233Gb. The Full Screen feature looked great, and I'm dying to see the new books feature. I wrote an entire book on iPhoto book themes and can't wait to see what's new.

Until today. Upon upgrading to iPhoto 11, I got the revolving gray wheel for a very long time, and the Finder said that iPhoto had "stopped responding" so I force quit and started again. I was finally able to upgrade and rebuild my library, but there was nothing there. Every event was labeled properly, and contained placeholders for my images with my labels and even my ratings, but the images themselves were not there. They were all gone.



When I went to see how big my iPhoto Library was, it said 6.3Gb. There's no earthly way my 233Gb of photos could suddenly fit into 6.3Gb. Further confirmation is that suddenly there are 230 extra GB available on my hard drive.

I would be sick (and furious) except that I backed up my library twice before attempting the upgrade. Please pass this information on to everyone you know so that no-one else loses their photos.

Apple's Discussion Forums are full of other people experiencing the same issue. Make sure you have at least one backup before attempting to upgrade. Also, if you depend on Time Machine, make sure Time Machine has been able to complete a full backup with iPhoto completely closed. Note that Time Machine will not properly back up your photos if iPhoto is open.

It's absolutely unconscionable that Apple should release software that would erase an entire iPhoto Library. We must spread the word to minimize the damage. Thanks.

As soon as I figure out how to upgrade successfully, I'll post that information here.

First fiction ebook in iBookstore with video!

I had hoped to write this article using one of my own books as an example. In August, as I wrote here, Apple changed the rules for submitting ebooks with embedded video. If you are interested in publishing a book with video, and downloaded the updated iBookstore Publisher User Guide 1.3.1, you might have been worried about the requirements it contains.

For example, it says you must have a black frame at the beginning and end of every video.

It also says that final files must be produced with Compressor 3.5.2, which, for the record, is only included with Final Cut Studio and Logic Studio Retail, which even when discounted by Amazon, cost lots of money. I count myself among those who thought, "Oh, maybe I don't have the proper tools to put video in my ebooks."

Then, I got a nice note from Jeffrey Dinsmore who had bought my EPUB book and had happily used it to publish an exciting new project that combines short stories with short films. He was thrilled that their new ebook had been accepted by the Apple iBookstore. He says it's the “first fiction ebook in the iBookstore with embedded video”.

It's pretty interesting as a book, that is, enhanced ebook. It's curious to have both media right there in front of you, and to read one and then view the other and think about how and why they were presented differently. You can download a sample and see for yourself, though I don't think any of the videos are contained in the sample. Luckily, you can see them on the Awkward web site.

But that's not why I'm writing this post. I'm writing it because Jeffrey proved, by getting his book into the iBookstore, that those extreme requirements in the iBookstore Publisher Guidelines aren't really requirements. They're recommendations. When I asked him if he had followed the guidelines about black frames, using Compressor, and using a specific resolution of the source video, he replied, “I did absolutely none of the above ... just slapped the .m4v files in and crossed my fingers!”

So, there you go. If you want to publish an ebook with embedded video or audio, all you have to do is follow the instructions as outlined in my EPUB book.

P.S. I wasn't able to get any of my own enhanced ebooks into the Apple iBookstore, not because they weren't technically accurate, but because Apple considers them to be “examples”, despite the copious step-by-step information they contain. I'll keep trying. Meanwhile, if you get your enhanced ebook into the iBookstore, please let me know!

Working at Home

Here's the sort of meme I'd be interested in: what does your workspace look like and how do you make it as efficient as possible? If you decide to join in, please send me a link. I'd love to see where and how you work.

[Cross posted from A Year in Barcelona]

I suppose I should be writing about our road trip through France over the weekend, but instead, as I assembled my office as I do every morning, I thought I might show you my workspace.

You see, here in Barcelona, we have a very small apartment, perhaps about 70 square meters (700 square feet), and I think that includes the hallway and the walls. It's very compact. I don't have a dedicated office space because there simply isn't room for one. Instead, I use the dining room table.

As soon as everyone leaves in the morning, I get rid of all the breakfast crumbs and make sure the table is completely clean. I hate a sticky desk.



Next, I grab the mini Belkin USB hub from the bookshelves, which is already hooked up to my wireless mouse and wired keyboard, and plug it in to the outlet on the floor.



Next, I grab my external monitor from the same bookshelf. I would have liked to have gotten one of those beautiful Apple jobs, but instead, I got a cheap, compact, and light LG (Mine is the M227, very similar to the one linked, but perhaps only available in Europe.) It even doubles as a TV, though I don't use it for that. The monitor goes on the table, and I plug it in as well.



I set my Macbook Pro on the corner of the table with the ports facing the monitor. It's just sleeping, I almost never turn it off. I have to plug it into the wall outlet first, or else the monitor doesn't work. Then I just plug in the monitor cable and the USB hub cable. Voilà, the monitor goes on.



Finally, I grab my fancy mouse pad (read: pad of paper of appropriate size and texture), my wireless mouse, and my stack of papers, from the bookshelf where they live during the evening and set them on the table. I'm ready to go faster than it took to read this article.



In the evening, just before everyone gets home, I reverse the process and put everything in reverse order back onto the bookshelf so they're easier to get the next morning.

I also have a printer and a scanner on the bookshelf, but only put them on my desk when I need them.

What does your workspace look like? How do you make it efficient?

Publishing Ebooks on Barnes & Noble

Barnes & Noble has recently made it possible for self-publishers to upload and sell ebooks for Nooks and the Nook app on the iPad and iPhone—through their Pubit! system. I was particularly interested in this development, because Barnes & Noble supports EPUB format—the open source, non-proprietary format for ebooks.

To sign up for Pubit!, you have to have a US Tax id (a social security number or Employer ID number), a credit-card from a US bank, and a US bank account to which your earnings will be sent. These are almost, but not quite, insurmountable obstacles for non-US residents.

There was one thing in the Terms & Conditions that caught my eye. First, while you set the List Price for your books (upon which your royalties are based), you agree to give Barnes & Noble “sole and complete discretion” over the price they charge your customers. I can’t quite explain why that rankles, though I know I hate giving anyone “sole and complete discretion” over anything of mine!

Next, they’ll pay you 65% of the List Price if that price is between $2.99 and $9.99, but only 40% if the List Price is less than $2.98 or more than $10. It seems that, like Amazon, which has a similar tiered system (70% for the midrange and 35% for very cheap or very expensive), they want to pressure you into keeping the price within a certain range: not so cheap that it’s not worth their while, not so expensive that they end up storing your files for an eternity with no sales in sight.

The excellent news is that you can upload EPUB files directly, which means you have complete control over the layout (and can take advantage of the tips in my EPUB book). However, if you don't have an EPUB file, you can also upload Microsoft Word docs (DOC and DOCX), HTML, RTF, and plain-text documents, all of which will be automatically converted to EPUB.

You can decide whether or not to apply DRM, though once you upload a book, you can't change your mind about DRM. What I'd like to know is whether non-DRM books uploaded through Pubit and bought through Barnes & Noble can really be read on any EPUB compatible ereader (like say iBooks on the iPad). I'll work on testing that and report back.

Book files can't be larger than 20Mb.

It’s nice that you don't need an ISBN, the international book number that booksellers use to keep track of a book within their channels. B&N will assign an “internal” 13-digit number (presumably not an ISBN) to your book.

Curiously, Barnes & Noble says you can only upload books in one of the following languages: English, Spanish, French, German, Italian, Portuguese, Latin, and Dutch. This seems a rather arbitrary limitation. Why not Catalan, or any of the Scandinavian languages, which can all be properly represented with the character set that the Nook already supports?