Word Needs To Die

I was working on a eBook conversion workflow for a small publishing house last week, as a favour to the owners. I thought I could get away with a couple hours of work: maybe write a few scripts, chain a couple of existing libraries together, and then email them my code. I was dead wrong. I gave up after two days of work.

The problem was with Word. Word’s doc and docx formats are proprietary, clunky to work with, and incredibly hard to convert to ePub and mobi without weird artifacts and edge cases. It doesn’t help that the standard publishing workflow is in Word — many writers, editors, and publishers use Word source files in their daily lives.

The challenges of working with Word are not new. Smashword’s MeatGrinder engine requires authors to tediously format their doc files; other guides warn authors against using Word to ebook conversions. The Outsell-Gilbane report on Publishing Transformation advises publishers to switch to XML-first workflows ‘as soon as possible.’

There are two likely solutions for this:

1) Write a perfect converter from Word to X, where X is any other text-based markup format. This is a technological problem, and is incredibly hard.

2) Get writers to write in non-Word formats. This is a social problem, and is incredibly hard.

The comparison between the two solutions above is, of course, a little unfair. The truth is that the second problem is easier than the first … but only in the sense that nobody has really tried taking a crack at it. There have been many attempts at writing a good Word conversion library, but all attempts have failed for various edge cases. There have not been strong attempts at creating a beautiful writer-focused tool, save perhaps Scrivener. But Scrivener isn’t popular the way Word is – ideally, you’d want something so pervasive writers would be crazy not to use it.

(I could, by the way, be wrong on the first issue – if you know of a good library to use, please hit me up in the comments).

I’m very tempted to take a stab at both problems over the Summer. No promises, but these are huge problems I wish someone would solve. The alternative to a Word-first workflow is a greatly simplified publishing process, one that is accessible to both writers and publishers alike.

Here’s a taste of that alternative world: Matt Neuburg wrote an essay on his book publishing process for O’Reilly Books. It is, admittedly, very technical, and it demands some programming knowledge. But his process is this: he writes chapters in a text-based format; generates HTML for quick previewing (ebook formats are HTML-based, after all) and then, when he’s ready, types a single command to send his source files directly to the O’Reilly server.

Here’s the really cool bit: because he writes all his chapters in a conversion-friendly format, O’Reilly is able to instantly generate a PDF – all properly type-set with fonts and layout as in an actual O’Reilly book. Neuburg then gets a copy of this PDF to preview, walking around his house with the book loaded up on his iPad. If he so wishes, Neuburg may run another one-line command, and all the readers who have subscribed to O’Reilly’s Early Release program for his book gets a copy of the updated book – in PDF, EPUB, or web form (at Safari Books Online). Naturally, his editor is able to plug into this process from the O’Reilly side of things, and every change is backed up in a Subversion repository.

In Neuburg’s own words:

  • I’m working in plain text, lightly formatted; so my writing and editing and revising are easy and nimble.
  • I’m using TextMate, a text editor that makes my use of lightly formatted text easy.
  • I can preview my work as HTML, which makes me a better proofreader.
  • I can “chunk” my book into nice-looking HTML chapter files for public consumption, so the rest of the world can watch me work.
  • Thanks to the O’Reilly commit hook, I automatically get a PDF version of my work. This is fun and encouraging as the book grows, and makes me an even better proofreader.
  • We’re using Subversion, so my editor and I have an easy time communicating changes back and forth to each other.
  • Without any trees being killed, readers can purchase an electronic Early Release edition of my book, and they are kept up-to-date as I continue to write and revise.

My point: moving away from Word enables writers and publishers saner publishing workflows. It doesn’t make sense for the writing/editing process to be done in a format separate from the ones used in the publishing process.

Word is a curse on digital publishing workflows. The sooner we move away from it, the better.

Possibly Related Posts:

Category: Publishing · Writing Tools
  • Rob Siders

    I think this is a great goal. Sadly, it won’t happen. People don’t want to think, and simple markup for text will make them do that. As yuck as Word — on many levels — there’s a usability affordance there that almost everyone can grasp: those icons on the toolbar that make your text look the way you want it to look. Select, click, done.

  • http://elijames.org Eli James

    Sadly, I think you’re right. All possible attempts at solving this would be really, really to do.

  • http://www.marywwalters.com/ Mary W. Walters

    I have been editing books (and other publications) for decades, and you won’t get me away from Word because it’s the best system there is for working with a writer on edits. The tracking system that allows the editor to suggest a change and provide a reason for the change by way of a comment, and then for a writer to accept or reject the edit, is incomparable. In self-publishing, excellent editing is critical. Those who don’t incorporate this step are legion at the moment, and they are producing books of abysmal quality. As awareness of the importance of editing increases, the greater the problems you will have in moving writers away from Word. As a writer (5 published books, 4 with traditional presses) and editor, I suggest you focus on the converter. :)

  • http://elijames.org Eli James

    What if there’s an alternative (better?) way of tracking edits? Would that make you consider switching? 

    For instance – the workflow Neuburg adopted with his editor at O’Reilly offers the same editing ability with the added benefit of being in a source format that’s easy to convert.

    I bet the publishing industry will soon move away from Word-first workflows. Publishers just can’t keep spending money on conversion services and software. Especially when the conversion software in question will never be perfect.

  • Christopher Wright

    I’ve been a technical writer for almost 20 years, and I’ve found Microsoft Word to be one of the most unreliable platforms for ANY project where consistent formatting is absolutely essential. I currently use LibreOffice, which is not perfect, but it has the advantage of being a) free, b) more reliable in preserving formatting choices, and c) uses a file format that is a completely open standard.

    It’s editing tools are acceptable.

  • http://www.facebook.com/ianjwalkley Ian Walkley

    I think all that is really needed is a couple of good template formats. One for ebooks, and one for printed books, with pagination alternated and properly sized for the printer. Word can easily save into pdf format, which programs like Scrivener can’t. And the editing capabilities of Word are essential. I don’t agree it’s that bad, really.

  • sevwinters

    http://openoffice.org That’s what I use. Wayyyy better than word 

  • Jodie Renner

    As a freelance editor, I love Word’s Track Changes function, and no trees get killed! The author sees all my additions and deletions, plus comments in the margin, and can accept or reject my changes quickly and easily. I also send them a clean copy, with all my changes accepted, so they can see how it would look if they accepted all my insertions and deletions. And then it’s quick and easy to delete the comments, so they have a clean, final copy very easily – no copying and pasting to another file.

    I’ve edited PDFs but I really don’t like it. It’s much more time-consuming for the editor, and then the writer has to go in and make all those changes, leaving room for either not noticing some of the suggestions on the little post-it notes or highlighted, or making new typos while implementing the changes, which doesn’t happen in Word’s Track Changes function, as it’s the editor who makes the changes, and the smart writer will accept most of them. Or in my case, they’ll usually just use the clean, Final copy and make any other tweaks on that one. That way the little corrections are intact.

    Sorry for being so long-winded!

  • Ann Littlewood

    MS Word was big and old and gnarly before epublishing was invented. But part of the problem lies with writers–they never bother to learn the tool. They do not know to lay off the Enter key and the Tab key or how to set up simple styles. Painters study paint and brushes, calligraphers study ink and pens, etc. Writers study word stuff but not their writing tool. And, yes, this would be incredibly hard to change.  But I am trying. Ask me about my workshop… Or not.

  • http://talesofthebigbadwolf.com/ sgl

    Word -> PDF -> epub (Via Calibre)?It’s not perfect, but it’s what I’m using to read my work on a nook. 

  • Rob Siders

    They won’t as long as authors still write using Word…

  • http://elijames.org Eli James

    Thanks for the pointer, sgl. I’ll check out that specific conversion path …

  • http://www.marywwalters.com/ Mary W. Walters

     I would learn an entire new program and write my next book in it if it had the same editing capabilities as word but would make the layout and design process simpler at the other end of the line. We are in this together and need to make the whole process as seamless as possible. I just insist that we must include a way that makes editing a natural part of the order. I am not ever going to edit a pdf again — it was the most aggravating experience I have had and I could not be a good editor as a result. In Word, I can be an outstanding editor. As far as writing, I can do that in any program. Or with a pen on the ceiling, if necessary.

    I really resent the comments on here that talk about the ignorance of writers using Word. I am familiar with all of its capacities and capabilities and so are 90% of the serious writers whose work I edit. And so are 90% of the academics whose papers I edit. Don’t dump on the writers — we’re as technically with it as anyone. We spend half our procrastination time learning how to play around with settings on the computer, for God’s sake.

    BTW, I am The Militant Writer and have been writing for several years about why writers don’t need publishers and agents any more. Here’s a sample post: http://maryww.wordpress.com/2011/05/30/the-author-as-publisher/

  • C LaGuire

    I prefer to write in a text editor (I use Apple’s TextEdit) but the interface of all the plain text editors is designed for coders — that is programmers — not novelists.  It would only take a few tweaks to the interface, though, to make any of them an ideal environment for a writer.  Basically let you view the document like you do in an ereader.

    Most of the text editors have done a half job on this: they allow the user to designate the font and font size, and even style just for display — for the user’s comfort — but these are not embedded in the text.

    If they would allow the user to set  the display of paragraphs the same way – with an indent or space between paragraphs which are NOT in the text – it would make writing fiction in a text editor much more comfortable.

    However, the thing I expect to see in the next few years is a word processor that is native HTML.  That is, on the front end, it will work like a word processor, but in the guts of it, it’s plain, efficient html.  Your formatting options will be based on limited tags and styles which are clean from the start.  And you will be able to go straight into the code and deal with that if you want to too.

    In other words, a word processor which is designed from the ground up for ebooks and conversion to multiple formats.

  • http://twitter.com/zoetewey zoetewey

    Personally I haven’t used Word in a few years now. I use Libre Office for any documents that Word users share with me. I use Google Docs at work and when I want to share documents with people.

    Both Libre Office and Google Docs have comments and some level of versioning available (in the form of Track Changes). That’s okay when I want to work with others on something.

    When I’m working on something and I don’t have to share it with anybody (or need them to make edits at any rate), I use Scrivener.

    It would be nice if Scrivener did versioning. Though it does have a lot of features that make writing easier, that’s one that I miss. That and being able to share the full project as easily as I can share a Word doc through Google Docs (or in a worst case scenario, by email).

    On the bright side (if this is one), I can always export things into Word docs and RTF so that Word users can edit. Scrivener’s pretty good with exporting into various formats.

    I’d agree that Word’s formats need to die, but I don’t have strong feelings about the program itself–with one exception.

    On the rare occasions that I’ve had to use it to do layout, it was horrible. Better to use Publisher (if you must use something from Microsoft Office), and even better to use Adobe Indesign.

  • Clare K. R. Miller

     I don’t know about other “programs like” Scrivener, but Scrivener itself has no problem exporting as PDF.

  • Leon Paternoster

    It’d be good if the *idea* of markup (i.e. separating content and presentation) was taught in schools. It won’t happen, but there’s no reason why it shouldn’t when UK students are currently taught Word and Dreamweaver.

    From the comments it sounds like the stuff around Word (such as track changes) is what writers would miss. So a Markdown editor with lots of collaboration features that exports to HTML, ePub and PDF would do the trick.

    Another alternative is a WYSIWYG editor that generates markup. This sounds contradictory, but I find Markdown Pad works pretty well.

  • http://nancyadamsfiction.com/ Nancy Adams

    Hear! Hear! I use Mac’s Pages software and LOVE it. When I self-published my Christmas short I wound up going straight from Pages to pdf for the print file I downloaded to CreateSpace.

  • Thomas

    Olé!!! Word’s a pain in the backside. Fine app for setting up paper docs, but dinosaur for e-docs. Personally (don’t shudder), I write in Markdown  in any text editor I happen to have access to (Android, Win, Google, iPad) and just make sure I keep a “master copy” on the web (Google Docs, in my case). When I’m done I just convert to html. From there, as you point out, one can go everywhere with ease.

    Oddly enough, noone seems to care that if you “spread” your documents to lots of users/readers, html can ALWAYS be read, regardless of the gizmo you read it on, since every gizmo nowadays has a browser. Moreover, it is easy to convert decent html to mobi or epub – or pdf, for that matter. But you’re right, it’s a social thing – or perhaps simply a thing of innocent ignorance and lazy habit ;o)

    FWIW, it’s not worth your time to have a go at a decent Word converter. Concentrating on a pain-free end-to-end authoring tool for e-books – such as Pandamian – would be a far better use of your time. Again FWIW, I’d suggest you think a bit about separating the “read-books-here” part of Pandamian from the authoring part. Offering Pandamian as a sort of Google Docs-like web-based “Scrivener” seems (to me) to be what Pandamian’s potential is really all about. When you look at the frustrations in the forums of KDP, it seems to me that the indie author world is ripe – VERY ripe – for such an authoring tool : “Switch to writing in this online word processor and forget all your troubles with converting to an uploadable Kindle, Kobo or iBook – just use “Save As” ;o)

    Just my 0.2 Euros ;o)

  • Radh

    I think my problem is, I don’t know any alternatives for Word, really. I usually write in a .txt format through a very minimalist text editor like Notepad or Writer, and then format it in Google Docs or wherever I am posting it online.

    Does Scrivener have the option to export as a PDF or a MOBI? I honestly don’t have the money to buy a new word processor. I’ve had the same Word for 10 years, and that itself was a copy from a friend.

  • Clare K. R. Miller

     Yes, Scrivener will export as PDF, mobi, epub, HTML, doc, odt, and pretty much any format you can think of. That’s one of the (many) extremely cool things about it. It does cost money, though (but you can try it free for 30 days). If you’re looking for a free word processor you may want to check out OpenOffice and/or LibreOffice.

  • http://twitter.com/zoetewey zoetewey

    Also, it’s not particularly expensive–not as expensive as Word anyhow.

    It’s actually currently free on Linux (on the other hand the Linux version is currently in beta).

  • http://twitter.com/NancyNorbeck Nancy Norbeck

    As a writer, proofreader, and editor, I disagree most strongly that Word’s Track Changes feature is anything like wonderful. It’s difficult to read, difficult to manipulate, and if there are more than a few edits, it results in a document that will make you blind long before you manage to decipher it. Editing should not be this hard, and the way Track Changes works, it’ll send more people running screaming from the editing process than it’ll bring into the fold. (Not that I’m at all sure the masses give a good goddamn about well-edited documents/books anymore anyway, given the comments they leave on those abysmal self-pubbed volumes on Amazon, especially if anyone tries to point out that there’s a need for editing.)

  • http://twitter.com/MaryWWalters Mary W Walters

    I’ve been editing for a long long time using track changes. I love it, and so do my clients. To each her own.

  • John Liu

    I think the key is to be able to convert text into HTML5 which is portable to most digital format (EPUB,  MOBI, PDF etc).

    I’ve come across this new website http://www.towerbabel.com that looks pretty promising.

    They are not opened to public yet but I got my invite in 2 weeks after signing up.

    I love it so far, beautiful design and it even has collaborative writing features