Proof of Concept

I am a pretty mediocre farmer. I tend to have bursts of energy and then hope that Nature will do the rest. And more often than not, she comes through, like this spring, with 7 lambs... way more than we would know what to do with, even if we weren't going to live in Barcelona for a year (in less than 3 weeks!). Yesterday, I sold all the lambs and one of the ewes to a friend, and last night, I listened as the remaining ewes plaintively called for their babies. Real farmers don't feel guilty, I fear.

But I was also worried that they'd be bursting with milk. Mastitis is no joke, so when I went out to check them this morning, I tried milking them. Icelandic sheep are considered a "triple-purpose" breed because they are often raised for meat, for their beautiful lustrous wool, and for their rich milk. I had half-heartedly tried milking one of the more docile ones last year, but time and other projects got the better of me. This time, since it seemed more of a necessity, I tried harder.

And it was easy! Well, at least with Honey, who is amazingly calm and easy-going. It didn't hurt that she was in seventh heaven with the bit of grain that I gave her. I had been enviously watching DH go off to milk the cow all summer, and it was fun being the one doing the milking. Since they were already out in the field, I did have to get DH to come help hold them steady, especially Fleur who is the most skittish of the group.

And though I didn't quite get a whole quart, it felt like a proof of concept. It would be better in the barn, with a stanchion, doing all four ewes, with a regular schedule, and I'm sure a couple of improvements more, but it really does work. Now I'm going to go sample some good Manchego, and when we come home, maybe I'll make some of my own.

Enhanced Books in iBookstore using video element

Update, 25 May 2011: I have recently published a new miniguide explaining exactly how to add audio and video files to your EPUB formatted ebooks so that they work properly and look great in iPad, iPhone, and iPod touch, as well as NOOK Color. You can find more information in this article.

Simon and Schuster released the Enhanced edition of Nixonland, by Rick Perlstein, today. This enhanced edition includes both video and audio snippets. Those of us interested in creating our own ebooks with embedded multimedia have been trying for some time to figure out how to do such a thing AND pass EpubCheck 1.0.5, ostensibly one of the conditions for making it into Apple's iBookstore.

Curiously, though, a look inside Nixonland reveals that this document is not XHTML 1.1 because it uses HTML5's <video> element to embed video, and thus cannot possibly pass EpubCheck 1.0.5.

Either Apple made a concession to Simon and Schuster (most probable) or it suddenly is not requiring EpubCheck validation.

I will be posting more specifics within the day.

Non-English InDesign CS4 exports italic/bold as Normal

Here's another InDesign bug. Curiously it happens only in non-English versions of CS4, confirmed for now in Spanish and Italian. I tell you about it here so that you don't spend hours wondering what you're doing wrong. It's so not you!

If you have defined and applied a Character Style with Italic in InDesign and then export to EPUB, InDesign creates this CSS:

font-style: normal;

If I export that exact same file from my American version of InDesign CS4, the code comes out correctly:

font-style: italic;

The same thing happens with bold.

Of course you can edit the CSS by hand (as described in my book!), but what a hassle? And what weird thing is going on with InDesign CS4 that the localized version should act differently than the original one?

A similar, or at least related error occurs when you export an entire book from the Spanish version of InDesign CS4. The @font-face rules at the top of the CSS file all reference font-style: normal; and font-weight: normal; which results in the last @font-face rule applying its referenced font (in this case, italic) to the entire book.

@font-face {
font-family: "Adobe Garamond Pro";
font-style: normal;
font-weight: normal;
src:url(Fonts/AGaramondPro-Bold.otf);
}
@font-face {
font-family: "Adobe Garamond Pro";
font-style: normal;
font-weight: normal;
src:url(Fonts/AGaramondPro-Regular.otf);
}
@font-face {
font-family: "Adobe Garamond Pro";
font-style: normal;
font-weight: normal;
src:url(Fonts/AGaramondPro-Italic.otf);
}

Since few ereaders are even supporting embedded fonts, the best way to solve this problem is to delete all of the @font-face rules completely.

Do you have a non-English version of InDesign CS4? Please let me know if you can reproduce this bug. Thanks!

And Adobe, if you want us to take InDesign seriously as an EPUB generating tool, you have to take it seriously too. Even in your localized versions.

[And an aside. I used to localize software so that's one of the reasons I'm so intrigued with this problem.]

And thanks to Jorge Portland who brought this problem to my attention, and Letizia Sechi who confirmed its existence in the Italian version of CS4.

Creating an index for EPUB with InDesign and GREP

[Just happened upon Joshua Tallent's very similar technique for creating indexes in his Kindle Formatting book. For the record, I wrote this before I ever saw that section in his book. Great minds think alike! :)]

I really like indexes. I wish fiction books had them, but I believe non-fiction books MUST have them, especially mine. I'm intrigued by the possibilities for indexes with ebooks, but wasn't able to wait for the fancy stuff to be ready for my new book about EPUB. So, I created my own.

I use InDesign to prepare my EPUBs. Curiously, InDesign won't export either the TOC or the index to its EPUB file. They simply don't appear, with or without links. So, I had to get creative.

The first problem was how to get the contents of my already created index out of InDesign. If you copy the index out to a text document, you don't get any of the formatting. You can export to Dreamweaver, and it will create an XHTML document, including everything but your TOC and index.

The solution is to copy the index and paste it into a new InDesign document. Then choose File > Export for... > Dreamweaver. You'll have successfully fooled InDesign into thinking the index was just normal text and it will create a nice XHTML file for you, with properly formatted index entries.


I'll leave it up to you to add the proper CSS so that your index looks snazzy.

The real trouble is how to make the links work. There's no way to know what the pagination will be like in an EPUB document. An EPUB can be viewed on all different size screens, and your readers can change the font size and thus change the pagination, as many times as they like. Page 23 in an EPUB might be half of page 16 in the print edition one day, and all of page 30 the next!

So, how do you link the index to the referenced pages without totally recreating the index entries one by one? (Not even an index lover like me is going to do that.)

My solution was to mark the actual physical print pages in the EPUB. This has two benefits. First, if ereaders ever get around to supporting the pageList function, I'll be able to use it to relate the print edition to the EPUB edition (which could be useful in classrooms, for example, to get everyone to the same content). For the record, Apple recommends the use of pageList, though I can't see that iBooks does anything with the information, at least not yet.

My book only has 192 pages, so I decided the fastest way to mark the beginning of each physical page in the EPUB was by hand, inserting <span id="pxx"></span>, where xx is the appropriate page number. It certainly took a lot less time than creating a target for each and every index entry (which still wouldn't have solved the problem of compiling multiple references into a single entry with multiple page references).

Next, I used GREP to convert my XHTML index entries into links to the now-marked pages.

The first step was to change the very few index entries that had numbers in the text itself. For example, I changed HTML5 temporarily into "HTMLfive" so that the 5 wouldn't be affected by my global changes later.

Since my EPUB has five separate XHTML files that might contain index entries, I had to use five different GREP expressions.

Here's the first one:

Search For:
( |–)(7|8|9|1[0-8])(,|</p>|–)
(Which means, search for a string that starts with a space or an en dash, followed by either a 7, 8, 9, or a 1 followed by one of 0-8, and then followed by either a comma, closing p element or an en dash. Which means, in plainer english, find me any index entry number from 7-18 (the pages in my introduction). I limited it to numbers after a space or en-dash and before a comma, closing </p>, or en dash so as to not grab two-digit numbers within three-digit ones.

Replace with:
\1<a href="ePub-STTP-1.xhtml#p\2">\2</a>\3
This Replace phrase references the three things we found in the Search phrase. The first was either a space or an en dash. The second is the page number, and the third is either a comma, closing p element or an en dash. And so we'll replace what we've found with either the space of the en dash, followed by the code for a link that references the XHTML document that contains pages 7-18 (the introduction), followed a hash symbol and the letter p, followed by the page number we found, followed by the end of the a element code, followed by the page number we found as the clickable text, followed by </a>, followed by the comma, closing p element, or en dash.

You have to run it twice because the first time it will get all the entries before an en dash and the second time it will get all the ones after it. But in my tests, it couldn't find the en dash if the search text was contained in the previous replacement text (which makes sense).

And poof, all the index entries for the introduction are done.

Now, we'll do the index entries for the first chapter, which in this example, goes from page 19 to 44.

We'll Search for:
( |–)(19|2[0-9]|3[0-9]|4[0-4])(,|</p>|–)
The only thing different here is the page numbers we're looking for. Here we want 19, or a 2 followed by 0-9 (that is all the 20's), or a 3 followed by 0-9, (the 30's), or a 4 followed by 0-4 (from 40–44).

And we'll replace it with:
\1<a href="ePub-STTP-2.xhtml#p\2">\2</a>\3
Which is the same as the first replacement text, except that this time it's the XHTML file that contains pages 19-44.

I won't bore you with the three remaining pairs of search and replaces, although I'm happy to share them if you need them.

Of course, you may need to adjust your search patterns to match your own index's formatting. And if you have cross references (see this topic, or see also that topic), you should change those into links as well.

Then I changed those few index entries that had numbers in the text (like HTMLFive) back into numbers (HTML5).

I was done with the links themselves, and all that was left was to finish off the XHTML code (the head section, closing body and html tags, etc.), and then add it to the content.opf and toc.ncx files (as described in my book).


Once compiled into the EPUB it looks like this:


Find the entry you're interested, click the page number, and away you go!

This is not a perfect system. Since any ereader might divide what used to be a print page into various ereader pages, showing the beginning of the printed page may or may not include the referenced topic. The reader may have to turn an electronic page to find the topic. I added a note to the index to explain this.

Still, it's pretty good. It's lovely to be able to follow contextual index entries and find what you're looking for without having to search, say, every instance of "content.opf".

By the way, I have posted the complete index for my new EPUB book. (The links are not live on the website, since they'd have no place to go.)

EPUB Straight to the Point - EPUB Format Available Now!

I am so happy to announce that the EPUB version of my book about EPUB, titled EPUB Straight to the Point: Creating ebooks for the Apple iPad and other ereaders is available right now. I've created a new section of my website that gives a synopsis, an excerpt to show you what it looks like, shows the final table of contents, and lets you browse the complete index (which is live in the EPUB version), as well as buy the EPUB version of the book right away (facilitated by Kagi).

I will only be selling the electronic version from my website. If you'd like to wait for the print edition—which should be in bookstores by the end of the month, you can preorder the book from Amazon, Barnes & Noble, or my publisher, Peachpit Press.

This is the first time I've ever sold my own books from my own website, and I'm both excited and nervous. Please let me know if you have any trouble with the Kagi site, or with my end of it, and if there's anything I can do to make the process easier.

Note that the EPUB version of my book has no DRM. You can read it in any ereader that supports EPUB.

Hyperlink with Ampersand Causes InDesign Export to EPUB to Fail

I've been working on a book about creating EPUB files using InDesign and other tools for about six months. Today I sent the final files to the printer—it should be in bookstores by the end of the month. Then I chose Export to EPUB and it failed. Just failed. No details, no clues. It did give me this lovely dialog box:

I had been converting RGB images to CMYK, so I thought maybe that was the culprit. I changed them all back. Nothing. Not only nothing, but each time I tried an export, the script would stay on the screen, requiring me to quit all the way out of InDesign.

I thought maybe a large PDF file at the end of the book was the trouble, but that didn't make sense since it had exported fine with it just two days ago. Thanks to suggestions from my Twitter friends, I restarted and repaired permissions.

I googled "InDesign EPUB Export Failed" but only got a few people with the same problem and no solutions.

I emailed a couple of InDesign gurus I know, including Teus de Jong, who has written a number of useful scripts for InDesign, including one that adds the otherwise missing required date and fixes the annoying bug that breaks hyperlinks and cross-references in exported EPUB files.

Teus wrote me back offering to look at my files with a debugger (since the dialog box was so vague), but it's 200Mb and seemed a lot to ask. So instead I asked if I could run the debugger myself.

No problem, he said. Open the OEBExportMenuItemLoader.jsx file, which is in the startup scripts folder, within the HTML for Digital Editions (or XHTML for Digital Editions) folder, inside your main InDesign program folder. (I just opened it with BBEdit.)

On line 31, he said, change var DEBUG = false; to var DEBUG = true;

Next time you try to export the EPUB, a DigitalEditionsExportLog.txt file will appear (on his PC it goes to My Documents, on my Mac it appeared in the Documents folder).

So, I followed his instructions, reexported the file, and lo and behold the DigitalEditionsExportLog.txt file appeared as promised. There was lots and lots of stuff about creating this style and that style, but at the very end, it said:

Error: XML error in line 474
- Not well-formed (invalid token)
Error Number: 60
File: undefined
Line: 794

Line 474? XML error? I puttered around InDesign's help files trying to see how to look at the XML representation of my file. Where would that be? I wrote Teus one more time hoping I wasn't bugging him too much and he told me that that probably referred to a temporary XHTML file created by the script.

I went back to the DigitalEditionsExportLog.txt and the last XHTML file referenced before the error was this:

Entry: OEBExporter.addChunk
temp file should be: /private/var/folders/xf/xfcWz+ZZGZKuQUR7NebUiU+++TI/TemporaryItems/onedoc_4/OEBPS/ePub-STTP-5.xhtml

I opened Terminal and copied in the whole path to the file, without the file name, followed by a return. (There must be an easier way to get to this location, but I confess I don't know what it is.)

Then I typed open ePub-STTP-5.xhtml.

The temporary ePub-STTP-5.xhtml file opened in BBEdit.

I scrolled down to line 474 and found this:

<div class="numbered-steps" xml:lang="en-us">
2 Download both KindleGen and the Kindle Previewer from <span class="link"><a href="http://www.amazon.com/gp/feature.html?ie=UTF8&docId=1000234621">Amazon’s site</a></span>.</div>

It turns out that the hyperlink that I created to Amazon's site contains an ampersand, and when InDesign tried to convert it to XML within the hyperlink, it didn't properly code the ampersand, and the resulting XML was badly-formed (a grand faux-pas in XML land). InDesign didn't know how to recover, and so stopped exporting the file. (If you're not sure what's causing the error, you could use a syntax checker to point out the error. In BBEdit, you'll find it at Markup > Check > Document Syntax)

I went back to my InDesign file, removed the hyperlink (I just put the URL in the text instead), and exported again, and it worked like a charm. I'll put the link in the EPUB by hand (properly coded!)

Moral of the story: Don't add hyperlinks with URLs that contain ampersands to an InDesign document if you intend to export the file to EPUB.

And second, if InDesign fails to export to EPUB, you have a number of tools at your disposal.

And thank you, thank you, Teus de Jong!

Their, They're, and There and the Direct Connection from Brain to Fingers

Just for the record, I totally know the difference between their, they're, and there, between it's and its, between night and knight! My problem is that my fingers don't know the difference. I'm a really fast typist, thanks to Mr. Berkman, back at Hartford High School, and I have verified that there is a direct link between my brain and my fingers. I can type someone else's text (as I did often in college), sing a song on the radio, and think about something totally different, all at the same time. Sometimes I find myself reading books to my kids, and they're sitting there in rapt attention and I have no idea what I just read. It's a strange thing.

Although I rarely write long-hand anymore, when I do, it's a bit of a disaster. I often write the word "five" as "5ive", I often add the last letter of the last word to the beginning of the next word, and when typing a more unusual word that begins the same as one I type often, my fingers often type the more common word instead. It's as if they have a mind of their own. And they can't keep up with my thoughts though they do the best they can.

Thank goodness for editing and editors!

So, if you ever find one of those typos in my writing, please do tell me about them. I hate it when they slip by. But I swear, it's not because I don't know. It's my fingers' fault.

RGB, CMYK, and Export to EPUB

I'm almost done writing my book EPUB Straight to the Point: Creating ebooks for the Apple iPad and other ereaders. I've been writing it in Adobe InDesign CS5, and typically when I finish a book, I convert all my illustrations from RGB to CMYK before sending the files on to the printer. Without going into the whys and wherefores of RGB vs CMYK, screen vs print, etc., etc., you should know that the CMYK versions of originally-RGB images tend to be slightly darker and more muted. You simply cannot create the same colors on paper as you can on screen.

For this book, I'm creating both a print and an EPUB version of my book (following the instructions in the book itself!) And it occurred to me that the EPUB version of my book is destined for RGB screens like the iPad, and so I want to use the RGB images in the EPUB—not the CMYK.

Here, for example, is the original RGB version of one of the screenshots in my book:


Notice particularly how vivid the red arrows are, and how bright the green W is at the beginning of the page.

Now here is the CMYK version destined for print:


In the converted CMYK version, the red arrows are much darker and the bright green W is much more muted.

The solution is to export to EPUB before converting your images to CMYK.

If you make changes to the content later and need to reexport the EPUB file, as long as you haven't changed the images, you can just copy the images folder from the older EPUB to the newer one. At least that's my plan. I'll let you know how it goes.