Thursday, September 30, 2010

Why I Crack Open InDesign EPUBs


InDesign CS5 is currently my favorite tool for generating EPUB files from existing documents. It's probably no coincidence that I'm a long time InDesign user in general, quite comfortable with its features and idiosyncracies. In fact, I don't use a standalone word processor at all. I hadn't bought Word for years until I needed it for examples for my latest book. I use InDesign even for the most basic letters. And InDesign's EPUB generation is pretty good: it generates all the persnickety XML files and exports a fair bit of the design from your original document.

But it could do so much more.

Here's why I currently have to crack open InDesign files (solutions to all these problems are in my book). Let's hope the next version of InDesign does all this for me!

1. First and foremost, because the EPUB doesn't validate. To do so, I must add the dc:date element.

2. Because InDesign doesn't properly designate an image or given page as a cover. This is huge!

3. To add metadata. Without standard, accurate metadata, you might as well not publish an ebook because no-one will be able to find it.

4. In order to apply text wrap to images and sidebars. Since InDesign supports the creation of text wrap, around images by themselves, groups of images and captions, and sidebars of text, it should also be able to export it into the EPUB (especially considering that only small adjustments in resulting XHTML and CSS are required).

5. To wrap text around drop caps.

6. To format first line of text (what in InDesign is a Nested Style) so that it maintains formatting in EPUB.

7. To specify which paragraphs should be kept together, for example, images with captions, headers with first paragraph following, etc. Again, InDesign supports keep together but doesn't export to EPUB.

8. To specify page breaks. Although iBooks does not yet support page breaks (which is incredible in itself), other ereaders do. This is a major concern of ebook publishers.

9. To specify widows and headers. One more instance where InDesign already has the functionality but does not support it in export to EPUB. Hugely important for ebook publishers (even though not yet supported in iBooks, it is supported by other ereaders).

10. Because Apple says, on page 15 of their Publishing Guidelines, that "All book elements (for example, the cover, table of contents, first chapter, index, and so on
must be identified in the <guide> block." Note the use of the word "must". Note that <guide> is optional according to the current version of EPUB, and Apple has not yet rejected books without <guide> blocks, to my knowledge.

11. To add pagination guidelines, so that readers can find passages from a particular page of a print edition in their ebook edition (e.g., in a college class).

12. To eliminate erroneous default values that InDesign applies to non-specified properties (which mess up CSS inheritance).

13. To correct/use font names supported by the iPad.

14. To add Zapf Dingbats (which must be added as UTF-8).

15. To add text in foreign languages, like Hebrew, Arabic, Thai, Chinese, Japanese, etc.

16. To fix the spacing bug that occurs when exporting headers (see page 163 in the print edition of my EPUB book).

17. To force iBooks to support left-aligned text (by inserting a span tag within every p element). Granted, this is a hack, but text will continue to look really awful when fully justified until we have decent hyphenation dictionaries in ereaders.

18. To change spacing and margin units from ems to percentages in order to adapt to smaller screens. This is a complicated issue, and will probably get more complicated, but also hugely important.

19. To deal with hyphenation.

20. To fix or insert links. Currently, EPUB export breaks if you try to export a link that contains an ampersand, and most links to Amazon to buy books contain ampersands! <strong>Update (3/Oct/2011): CS5.5 exports links from multiple chapters correctly! </strong>

21. To insert video or audio. I would recommend following Apple's lead on this, using HTML5 tags, but allowing for different formats.

22. To maintain the inheritance among styles in the InDesign document in the generated styles in the CSS. I know this is complicated, but if I can do it with a little bit of GREP, couldn't ID do it automatically?

Most of these issues are described (and solved with workarounds) in my EPUB book. But wouldn't it be nice if InDesign just did this for us? What do you wish InDesign could do?

23 comments:

  1. Will probably buy your EPUB book. :-) Is there a basic ebook format that one should initially use, one that has the widest application and easiest conversion? Is that EPUB?

    I have a Kindle book for sale. Used simple HTML for formatting, worked great. I KNOW HTML... would like to get techno-ducks in a row here for maximum epublishing efficiency.

    ReplyDelete
  2. So far, I'm stuck because I have never used Terminal and really have no idea how to do that. And I really have trouble messing with code (though I've learned to handle what DW forces me to do with CSS and HTML).

    So, your book has been of little use so far. Not your fault, but sad.
    So, yes, I want InDesign to make ePubs that do all you say with a validating final product.

    ReplyDelete
  3. I live only a couple of miles from where Adobe develops InDesign in Seattle and run into Adobe staff all the time. For the last couple of years, I've been getting a sense that the emphasis at Adobe is changing, at least at the marketing and management level.

    Look at Adobe's product line from a marketing perspective. When a new version of InDesign comes out, most of the layout and design pros like yourself will upgrade. It's not a large group to start with and there's not much room for growth when almost everyone upgrades, however cautiously.

    But there is enormous room for growth in the business and government world. Convince some office worker that he can't create documents in Word any more, that he must use InDesign, and you've not only got a huge pool of potential customers, you've got one that's easy to grow.

    And how do you do that? You do what Microsoft did with their products: add feature after feature with little concern for the finished product. And to puff sales even more, put the emphasis on suites not individual products. You just need to convince that office worker that he'll occasionally need Photoshop for some purpose, even if it's just fooling around with his vacation photos.

    The result is the Microsoftication of Adobe's products. You add lots of ill-thought-out features, because that's what Office Johnny likes and what he needs to get his boss to sign off on an upgrade. And because he doesn't know much about the nuts and bolts of publishing, he's more attracted by a poorly done ID-to-Flash conversation than he is repulsed by a poor quality ePub export.

    There is another factor. Businesses are very big on PDF, using it for all sorts of purposes as they attempt to go paperless. Someone at Adobe told me a few years ago that Acrobat for Windows was the company's biggest money maker. But the business world has yet to get into ePub. It's not something they get breathless about when Adobe comes knocking on their doors asking about new features.

    In short, like it or not, design pros don't get the attention they once got at Adobe. The money is in the business market and that's where development emphasis is. You and I may want InDesign to create powerful, standards compliant ePub. But we're a minority of their sales. Significant change will only come if the world of business and government becomes infatuated with ePub.

    ReplyDelete
  4. Liz, I've found great use for your book, and I am still working at a project to convert an existing magazine I design to epub. I agree, there are many things that InDesign should do and if we keep writing to Adobe, we will get the point across.

    There are many things that your book covers and for that I am grateful. There is one section I am having trouble with and would like to see if you could talk about in one of your blog posts. It's about the topic of creating a Navigational TOC in InDesign. You write:

    "You could also create sub-entries in the table of contents menu for subsections, or even illustrations or quotations. It’s up to you. For this example, we’ll keep it simple and just use the chapter titles."

    Would you be willing to expand on it and direct us on how to create sub-entries, illustrations and quotations? I think it would go a long way to solve a navigation problem I'm having in wanting to use illustrations AS the headers that link to the TOC.

    Thanks so much for your articles. Your blog is one of the few I regularly visit. (Balance between creative and networking)

    ReplyDelete
  5. @John: Yes, I believe EPUB is the easiest and has the widest reach. It is an independent standard, beholden to no single company.

    @David: How could I help make Terminal more accessible while we're waiting for InDesign to get up to speed? Honest, it's not that bad :)

    @Mike: Perhaps, but I'm the eternal optimist!

    @Lisa: Thanks for your kind words and great suggestion. I'll see if I can work up a good example that better illustrates more complicated TOCs.

    ReplyDelete
  6. I love InDesign too, but I am bemused by the direction CS5 has taken -- towards greater ease of animation. It's as if Adobe was trying to "rescue" Flash from HTML5 by tying its fortunes to InDesign.

    I would have preferred much, much better EPUB generation and enhanced XML handling instead of animation, and I'll bet I'm not alone.

    ReplyDelete
  7. to summarize, you say that indesign is totally inappropriate to make epubs but by fiddling and struggling with the code, it is possible to get something acceptable, why not use Sygil? or Jutoh? and what about the Adobe Digital Solution promised for late summer 2010 here: http://labs.adobe.com/technologies/digitalpublishing/

    ReplyDelete
  8. @French Spanish: A very fair question. Two reasons: first, many publishers have existing files in InDesign that they want to convert to EPUB. Second, and perhaps more importantly, InDesign lets you create new books for both print and EPUB simultaneously.

    ReplyDelete
  9. Another issue I would like to see addressed is the placement of anchors. InDesign places them in a line above their target reference. It makes more sense to place them after the opening tag of the target reference, as then it does not obscure the underlying sequence of tags, which makes for easier setup of CSS contextual selectors. For instance I greatly prefer:

    <h2>Head Content</h2>
    <p><a id="myanchor" />Following paragraph</p>

    …to…

    <h2>Head Content</h2>
    <a id="myanchor" />
    <p>Following paragraph</p>

    …because a style such as…

    h2 + p {text-indent:0;}

    …will not be broken by the <a> tag in the middle.

    I started out doing a lot of stuff like:

    h2 + p, h2 + a + p {text-indent:0;}

    …but now move all the anchor tags to where I prefer them with grep.

    ReplyDelete
  10. Since the EPUB standard is about to be updated and the likelihood of significant change quite high, perhaps there is some reluctance to make any substantial investments among hardware and software vendors in this market. How long do you think that will take and what is the likely outcome? Will XHTML be completely supplanted by HTML 5? Will the canvass tag, SVG, and web fonts be included?

    ReplyDelete
  11. Liz

    17. To force iBooks to support left-aligned text (by inserting a span tag within every p element).

    Better to use CS4 where this is not a problem.

    ReplyDelete
  12. I'd be interested to know if you feel that the export feature in CS5 is significantly improved from CS4. I currently use CS4—as does my entire office, and I'm reluctant to ask for an upgrade unless I know it will make my life (i.e. export to epub and subsequent fixing up) any simpler. Right now there's quite a bit of sprucing that must be done. But since the book only exists in its finished state in Indesign, I must export from that program. (Apologies if you've discussed this elsewhere already!)

    ReplyDelete
  13. Hi Elizabeth, I am enjoying and absorbing 'Epub Straight To The Point'. The ebook I am in the process of bringing to market has about 2 dozen videos. I have currently embedded them in pdf and to do that I used cs4 indesign.

    My question is about links. As epub doesn't support video (like having them embedded as in a pdf) I am wondering if having a link to them would be one possible solution? I could have a website with each individual video having its own page and then each link would be that particular page showing the video.

    Make sense or not? Thanks, Bob

    ReplyDelete
  14. Hi Liz,
    am following the video/audio developments with interest. Re:
    2. Because InDesign doesn't properly designate an image or given page as a cover. This is huge!

    in my indb file I just make the first file an indd file called 'cover.indd' which contains only a high res jpeg of the cover image and when I publish it's read as both the cover image on the 'shelf' in iBooks and as the plate next to the toc...for conversion to mobi with Calibre you have to re-import the jpeg again to get it to resize, but that seems to work perfectly for me...or am I missing something?
    J

    ReplyDelete
  15. @David: Regarding Terminal—Apple has made it very easy to get started with all of their applications including Terminal. It is similar to typing in commands to DOS if you are familiar with PCs. However, I'm loving the fact that Apple makes everything drag and drop.

    Start Terminal: Go to Macintosh HD > Applications > Utilities > Double-click Terminal. In the menu go to Help > Terminal Help. You can learn everything you need to get started and the commands needed.

    I was initially having trouble with typing in the path, but you can get the exact path to open files by dragging the file and dropping it into Terminal. The path is automatically typed in for you. Read about shortcuts in the help. If you have other trouble, go to Apple and try to get answers in their forums. That is what they are there for.

    Lisa

    ReplyDelete
  16. Liz,

    I'm finally getting somewhere with a question I've had since starting this project: How to use illustrations AS the headers that link to the TOC!

    This is early on in my testing, but you CAN designate a navgable TOC header to an illustration (the image/headline page for a magazine article) instead of a text headline.

    Using the magic image size, I convert my page layout with headline and image to become my illustration as a .jpg. I generally have to make the image longer or crop to make the proportions. I create a style called header-illustration and follow the directions from your book of inserting the image in the flow, just before the beginning of the article.

    Once my layout of the epub is done in InDesign, I follow the directions from your book for creating the navigable TOC. You can designate more than one level 1 TOC style to be used, and sub headers as level 2, and so on. I would suggest a test that includes a regular text headline AND an illustration treated as a headline so you have both in the test example. I finish the process of preparing the document and export to EPUB.

    In the .xhtml doc you will notice that the image tag is wrapped in an h1 tag, the class is header-illustration, and the id is toc-anchor. Inside the h1 tag, the image tag is also wrapped in a span tag.

    Open the toc.ncx file. You'll notice that there is a navLabel tag for each TOC entry, which is a navMap tag. Look into the navMap line where your image should be as a TOC entry. After the opening navLabel tag there should be a strange looking text tag. This refers to your header-illustration and looks like this: a tag with the word "text" and forward slash AFTER. This is followed directly by the closing navLabel tag.

    Compare this navLabel to your text based header navLabel, which is an opening text tag, followed by whatever the text of your header is from your InDesign document (matches the h1 text in your .xhtml), followed by the closing text tag, and the closing navLabel tag.

    Currently, if you open your test epub in ADE, there will be a blank, but click-able TOC entry in the navigable TOC area. To change it from being blank to text, go to the navLabel area that refers to the illustration and change the empty text tag to a regular opening text tag, followed by the text you would like to show up as your chapter or headline in the navigable TOC, followed by the closing text tag and closing navLable tag.

    Validate your epub and open in ADE. The clickable blank spot in the TOC should now reflect what you typed in for the navLabel text. I am currently getting my example to work in both ADE and Calibre. Any help with testing it out in other ereaders would be greatly appreciated.

    (Liz, sorry about spelling everything out, I don't seem to be able to put proper code in to show it, so I described it instead.)

    ReplyDelete
  17. Liz, great book and blog. I hope Adobe is listening to you. I also hope you manage your way onto the CS6 beta, if one exists or will exist. You can add this one to your list:

    I'm working on a directory, with extensive use of tables for the entries, because I wanted to have the phone numbers right justify and use percentages for the table and columns so it could be expandable to the window. When Adobe breaks the xhtml files up (to keep under 300K), it's not breaking my table properly, in fact it doesn't complete the entire row (it breaks it after the first column).

    ReplyDelete
  18. @Mark, have you seen "Opening New Windows in iBooks"? (http://www.pigsgourdsandwikis.com/2010/12/opening-new-windows-in-ibooks.html)

    It's ideal for big tables :)

    Glad you're enjoying the book and blog!

    ReplyDelete
  19. Yes, I saw your post and will review it again. My tables are designed to not look like tables, but just to display directory data better. There are also some other strange things happening (and as a programmer, I can tell this needs to be tightened up on Adobe's side). I have my export set to create a new file for every first level heading. Well, when the table breaks, Adobe is creating a separate small file for every level of element up to where the table starts again. So, for example, I get a file with just html tags, then the next file has html and body tags, then first div and so on. I get about 8 of these before the table data starts again. Well, we all know what this does to the ePub. I get a bunch of empty pages between table splits. I'm going to have to go in and remove these, fix where the table splits, and edit the file that lists all of the documents.

    I also wish ePub supported leaders, but perhaps that is for another day. I went to an HTML5 seminar last week, and sometimes I feel this ePub standard is not moving as fast as it could. I guess we need to struggle with cheaper hardware and what it can support to get this technology out to the masses. But, my feeling is something else is going to come along and replact this at some point if it does not evolve. I don't care how many people are using it. If some major manufacture just created an eBook format based on HTML5, it would blow ePub out of the water and be easier to market as the next gen technology.

    ReplyDelete
  20. Hi Liz,

    First and foremost, I'm a big fan of yours, since the very first HTML QuickStart Guide you’ve authored a few years back. I bought two of that book, the first one went missing when one of my friends borrowed it way years ago.

    So I bought another book, If I recall, a revised version, which eventually gone missing too when, this time, one of my office friends borrowed it from me (they all sure love that book).

    I’m not pulling your leg when I say that everything I know now in HTML started with your book. Your QuickStart Guides are immersing. You explain everything in a plain and simple manner which ordinary, non-technical readers like me will understand.

    Well now, to get to what I want to say, French Spanish Online commented above about Sigil (an ePub editor). I actually use InDesign in creating ePub files, then when it’s coding time, I use Sigil. Although Sigil is not liberal in editing ePubs (it actually changes the directory structure of the whole ePub, then changes the XHTML contents accordingly) it’s great in making sure that some of your problems get solved, namely list numbers 1 to 3. It also has a built-in validator (though buggy at times but over-all it’s okay). Just my two cents regarding ePub creation and editing.

    All in all, I hope InDesign listens to you, because if they do, then they listen to us (your readers) as well. More power Liz.

    Mers

    ReplyDelete
  21. Hi Mers:
    Thank you so much for that lovely note. You made my day. Do you remember that first book... so skinny! Many years ago! Thanks for keeping me going all this time.

    And isn't this new stuff fun? I'm so excited about all the possibilities.

    As for Sigil, I need to look at the new version. I don't like it when programs change my code, especially since audio and video are not yet standard and so Sigil strips them out (so I've heard). But I know that it can be helpful in some situations. Thanks for reminding me!

    kind regards,
    Liz

    ReplyDelete
  22. I'd love to see this post updated for CS5.5 as the update seems to address some of these issues. For example: it would appear that the new version supports drop caps, video, more image control and possibly page breaks. I'm still researching all these features and have found little help on the web with these specifics.

    It would also be good to know if your latest supplement ebook about video can be updated to include CS55 info. For example, what is the easiest way to leave space for the videos in the indesign document so you can see the text that follows it? Is there a limit to the number of videos you can embed and still have poster frames? I'm experiencing both of these issues in CS55. I currently have about 15 videos embedded in a single .indd and none of the poster frames I've chosen are displaying.

    Your books and website have been an invaluable resource. Thanks!!!

    ReplyDelete
  23. Hi Liz! Your book has been a great resource for me. I was able to create my first epub book last month with your help, thank you! My second epub is almost complete but ran into an issue with the cover file. I followed all of the instructions from your book and the exact same steps as I did for the previous book that worked fine. When I add the cover xhtml file and all of the necessary components in the content.opf the first page of the book is blank.

    Do you have any idea what is going on? Thanks again!

    ReplyDelete

More of my books