Say you’ve finished a major arc of your online novel. You want to turn aforementioned arc into a download, and perhaps make that available for purchase from the store section of your site. From here on, however, you’re met with two problems: 1) you’ll have to convert your text to an appropriate ebook format; and, 2) which one?

The ebook format fiasco is sometimes called ‘the tower of eBabel’, and for good reason: there are too many of them. But because we deal in digital fiction, and because ebooks are fast becoming viable models of distribution, we need to consider the sticky question of which ebook format, and why. This post attempts to answer that question. (Note that this is quite difficult to answer without looking into the future, simply because it is unclear if there’s ever going to be a victor in the ebook format wars. But I’ll get back to that in a bit.)

Context

E-book formats are no longer created from scratch. In most cases, the ebook maker - regardless of whether it’s a vendor or an open-source project - will decide to adapt and use an existing format, or to have some underlying programming language to make coding the format easier. Today, that language is often XML, or eXtensible Markup Language. Before we talk about the various ebook formats in proper, it’ll be good to talk a little about XML, and why it’s so popular as an underlying language.

The answer to that lies in XML’s name. ‘Markup’ and ‘Language’ are pretty self-explanatory; it tells us that XML is a programming language that consists primarily of markup tags, much like HTML.^[1] In fact, an XML document looks pretty much like any HTML page, the only difference being that XML is powerful enough to define and shape other languages ^[2]. But unlike HTML, XML is extensible. This means that XML allows you to define and create your own tags. For example, if I were an e-book-format creator, I can easily create and define <title> as a tag describing the title of an e-book. <title> doesn’t actually exist in XML. However, because XML is extensible, I can create what is effectively a whole new platform for my e-book format, and it’ll contain <title>, and whatever other tags I see fit to use. All I have to do is to define them, so that my ebook reader will understand which bits are which, and treat those sections accordingly.

You can tell that XML is useful precisely for this flexibility of form and function. The language is now used for many, many things - sometimes even as the foundation for web services to send requests and responses, behind the scenes, server-to-server. And if you take a look now at even the simplest of RSS feeds, you’ll find a language that is defined - and made possible - through XML.

Most of the major ebook formats today are all built upon some foundation of XML. The ePub format, widely tipped to become wide-spread, is built on a strong XML base. The Amazon Kindle format is built on a modified version of the Mobipocket ebook platform, which is in turn built on XHTML (with a dash of javascript/frame support). So is the format used by the new Sony Reader, though that’s known as the Sony BBeB. The conclusion you can take away from this is that sooner or later, XML will become a major part of your workflow regardless of which ebook format ends up as the eventual winner of eBabel. There’s no running away from it. The good news is, however, that XML is a remarkably convertible format. It’s going to be easier and easier to work with as most major software vendors make the jump to XML-based files; case in point: Microsoft Word’s new docx format is built on XML, and it’s not very hard to convert XML to other formats - say, PDFs, or HTML, or an XML-based ebook format of your choice.

The e-book Formats

So let’s get started. The following are the e-book formats in use today, ones that I believe still have a fighting chance of becoming the format of the known universe.

1. Amazon Kindle’s AZW. The Kindle uses Amazon’s proprietary AZW format, but can read unprotected Mobipocket e-books, HTML, Word documents and plain text (.txt) files. You convert to AZW using Amazon’s online Digital Text Platform, and you format your e-book using rudimentary HTML. AZW supports DRM (unfortunately) and is built around the Mobipocket format - though, confusingly, DRM-protected Mobipocket files cannot be read on the Kindle, because they’re not exactly one and the same. Is it worth it? Publishing your work in the AZW format grants you immediate access to the Amazon online store, where a number of online writers have been making a decent sum selling their work … some of which have been regularly hitting the top 10 bestseller lists for Kindle e-books. So … yes, it’s worth it.

2. Sony Reader’s BBeB, which stands for Broadband eBooks, is perplexing: Sony does not offer any tools to convert to the format, making the Sony Reader a closed medium to all but the biggest of publishers. In fact, the only way to publish for the Reader is via RTF or PDF … but XML to PDF conversions aren’t solid, not at the moment, and RTF limits your formatting options (it’s hardly better than a .txt file, to be honest). And there is at least one unofficial converter to BBeB, but Sony’s lack of support for writer releases is discouraging at best. Is it worth it? No.

3. Mobipocket (also known as mobi). The Mobipocket format was originally created by Mobipocket SA, a French company, in 2000, which was then bought over by Amazon in 2005. It’s been around for quite a bit, and it’s probably the only ebook-ish format at the moment that can claim full multi-platform compatibility. It runs on just about everything: the Kindle, the Palm OS, Symbian, Windows, Mac, and on the iPhone (the Stanza reader allows you to read Mobi books, though it was recently bought over by Amazon and is now in a vague sort of flux). It is, however, not very popular, and there doesn’t seem to be a captive audience or a community built around the format. A quick snoop around the official Mobipocket site confirms this. Why? I’m not sure, not at the moment (and I’m still looking for proper mobi-related numbers) - but a surprising amount of traditional publishers offer their ebooks in a mobi format. Is it worth it? This is hard to say. On one hand, the Mobipocket software suite is completely free, and it’s old enough to make conversion and formatting very easy on the writer. But the truth is that it’s not an exciting format to talk about, and this lack of excitement can probably be attributed to a lack of Mobipocket users … even with free software for just about every platform. And if you’re not likely to get serious ebook readers on Mobipocket (and you can’t sell mobi ebooks on Amazon for Kindle, anyway), then I guess it’s not worth it to spend so much time and energy on a format not many people would use in the first place.

4. ePub originally started off as the OEB (Open eBook) initiative. ePub is currently tipped to be the next big ebook format, if only because it’s backed by a loose consortium of publishers, writers, and programmers, who are tied together in the IDPF, or what is known as a ’stardards and trade organization for the digital publishing industry’. As mentioned earlier in this article, ePub is built on XML, and so the IDPF leaders are currently trying to push it as a distribution standard for e-books. This means a couple of very interesting things. If the ePub people have their way, publishers will no longer have to produce e-books in different formats for different e-book vendors; they publish in just ePub, and demand that everyone else (say, Amazon) convert ePub to their own proprietary format. And it’s really simple to do that, primarily because ePub’s built on a nearly 100% XML base - itself a highly convertible format. Is it worth it? As of late 2008 Sony announced that their reader would now support the ePub format, and publishers (or at least, the ones who have vested interest in a digital book future) have been relatively supportive of ePub over others. If the IDPF people get their way and ePub becomes the industry standard (or even if it becomes just a distribution standard), ePub would well be worth it. I’m fairly optimistic that ePub will win - at the very least, I want it to win - but the road to that future is far from clear-cut: Amazon has yet to announce any plans about ePub compatibility. They’re the one major player who’s yet to come around to ePub, and for what it’s worth - I think that it’s going to take a bit of time, some elbow grease, and a lot of arm wrestling to get them to see things from the publisher’s point of view. But give it time. It should happen … eventually.

5. Adobe’s PDF format is probably the most known amongst the e-book formats I’ve discussed so far^[3]. There’s not much to talk about: PDFs are simple, familiar, and easy to use regardless of medium, plus they’ve been around long enough for everyone to know, more or less, what a pdf file looks like. And because the PDF format is so old, it’s not likely that you’ll ever meet anyone with a computer that can’t read the PDF file format. Is it worth it? Hell, yes.

The Format That Wins

I want to make a case here that the primary ebook format we’re going to work with is probably going to be whichever ebook format wins on the iPhone. The Apple developer conference, WWDC, happened not very long ago, and several very interesting things became clear during that conference, most of it worrying news to the rest of the mobile phone industry, but good news for the rest of us. Here’s what Daring Fireball’s John Gruber has to say:

On the whole, there was a palpable sense that the iPhone is a peer to the Mac in Apple’s eyes. This isn’t about counting how many sessions were devoted to each. Nor is it an indication that the Mac as a platform is slowing. Quite the opposite in fact — Apple is selling more Macs than ever, and, knock on wood, there’s a strong consensus amongst developers that Snow Leopard is going to be the best release of Mac OS X yet. It’s simply that for however fast the Mac is growing, the iPhone is growing far faster.

But the two platforms are symbiotically intertwined. The Monday schedule at WWDC is static. In the morning comes the keynote, which the press attends and where all public announcements are made. After lunch, though, there comes what is effectively a second keynote, this time with material aimed squarely at developers. A technical keynote, as compared to the morning’s marketing keynote, if you will. This technical keynote has for as long as I can remember been titled “Mac OS X State of the Union”. This year the title changed to “Core OS State of the Union”.

Hence the symbiosis: Apple now has two full-fledged developer platforms, Mac OS X and iPhone OS, derived from one core system. Neither felt more important than the other this year at WWDC, which is remarkable considering that one of them hadn’t even shipped two years ago.

But look at their vectors — their relative rates of growth — and ponder how much longer until WWDC begins to feel like an iPhone developer conference with a Mac developer track. My answer: next year. In other words, I think it will have taken just three years for the iPhone to supplant the Mac as Apple’s primary platform. By 2011 it will be obvious.

It’s simply a matter of users. During Phil Schiller’s keynote, he showed a graph of the “OS X” user base over time, with steady growth over the first part of this decade followed by a sharp jump from 25 to 75 million over the past two years. This figure was widely mis-cited, however, as showing growth in “Mac OS X” users. It did not. The graph said “OS X”, not “Mac OS X”, and what Apple meant to show were the combined number of users of Mac OS X and iPhone OS. It was a very misleading and poorly-designed chart.

This doesn’t prove anything on its own, but stick with me for a bit. I’ve been seeing several articles arguing the point that AT&T isn’t providing immediate MMS and tethering support due to fear that their network would crash the very instant a million or so iPhone users decide to connect their devices. And I’ve noticed that the iPhone is itself a remarkably tactile platform, one perfect for reading books, and that we’ve already seen a number of apps showing us just that: that reading, and reading on your iPhone, is one hell of a revelatory experience. We’ve also been hearing rumours of an Apple tablet, with all the touchy goodness associated with their current multi-touch technology, and having that released in the not-too-distant-future would mean bringing the tactile interface to a fully-fledged operating system. And that, lastly, all those people connecting to an online network on such a small device will be a community of captive, fanatical users limited by the processing capabilities of their phones, but not by their phone’s features … making the iPhone all at once better than any ebook reader out there (cough the Kindle cough) but also perfect for reading text on the go.

But all of the above are small, fragmented pieces of information, hardly worth talking about, individually. It’s when you look at them from a broader perspective that things begin to become a lot more exciting, particularly from a digital-fiction point-of-view. Allow me to pull it all together for you: Apple sees the iPhone as a peer to their traditional Mac platform; the iPhone is a superior tactile device perfect for on-screen reading; the iPhone has a fanatical userbase that is connected to the Internet, one that downloads and consumes content through the iPhone itself; and Apple is a master at enabling 3rd-party (software) innovation. Put two and two together and you’d realize that this platform is ready for just the right ebook app^[4] to come along, and whichever one it is - be it Amazon’s Kindle app, or an Eucalyptus-type reader, or even one that we’ve never heard about - whichever one that is, that app will be the turning point that defines our industry. Want to know which format you should end up supporting? Watch the iPhone, and watch it closely.

____________________________________________________
^1. HTML isn’t really a programming language, but XML resembles it in the sense that both have very simple opening and closing tags as a foundation, like, say: <head></head> or <blockquote></blockquote> ↩

^2. Don’t worry too much about how XML works with other languages - that bit’s not relevent to this article ↩

^3. Though I must note here that the PDF is really more of a document format, not an ebook one. ↩

^4. This is dependent on one more factor: the app must have seamless integration with an online store, which in turn must be stocked with a good collection of ebook titles. In this aspect, at least, Amazon seems to have a clear lead, but no more so than if Apple decides to enter the ebook market themselves. If they do, or if some publishers decide to take things into their own hands and cobble together an online store/app combination, then I’m willing to bet that things will get very interesting, very fast. ↩

I’ve spent the last couple of posts at Novelr speculating on the future of web fiction - which as an activity, I must admit, was very fun to do. But it wasn’t a very useful one for the writers who read this blog. The essential questions remain unanswered: what do you do when the publishers finally wake up to the Internet? What can you do to prepare for a digital book future? 

Before I go into specifics, understand that you should take this article with a pinch of salt. These are steps that I believe aren’t too far off, and ones that I think can go a long way in preparing your writing for a more vigorous, more competitive online fiction sphere. On the flip side, however, I may also be completely wrong, and I’m obliged to warn you now that while this is a post that deals with practical steps, it’s also a post that deals with uncertainties. It is a first attempt in telling you what to do to get ahead in a place that doesn’t exist yet. If I’m wrong - and there’s a good chance that I am - then I suppose we can meet up 10 years from now and laugh at my stupidity. 

A Summary

Before we begin it’ll do to recap what exactly it is we’re preparing for. I’ve talked about this in the past, but for those of you who don’t have the time to dig into Novelr’s archives:

1. Publishers are exploring digital alternatives to books, and are currently figuring out how to distribute, market, and deliver them to the consumer. They’re forced to do so by the current recession, which is hitting the people in the publishing industry harder than most.
2. Printed books will not go away, but they’ll be staying on as ‘bespoke, art-directed paper packages’ - the top of a piramid of consumed fiction.
3. Self publishing, and by extension self-funded writing efforts like blooks and web fiction are going to become ‘tryouts’ for publishing houses. Publishers will look closely at the comments surrounding a self-published piece, and if it’s mostly good, and they think they can sell it, they then pick it up and sign-on the author for a traditional book deal. Haper Collins’s has tried to centralize these efforts - they’ve started a website called Authonomy and are hoping unpublished writers come to them with their manuscripts.
4. Writers will flock to the Internet in the sudden realization that there’re more ways to get published than just the agency/slush pile. We will be swamped with online manuscripts. Readers will go to certain filter sites, or perhaps stores, to find good things to read online.
5. Or not. They may want to put these stories in iPhones, Kindles, or one of the many portable device options poppig up today. They will want to read, and they will want to read away from the computer.

I’m not sure of the degree to which these predictions will come true, but for the sake of this article we’ll pretend that it’s a future we’ll have to prepare for. Which leads us to the focus of this piece: what can we do, now, to prepare for it?

Blogs Are Dead

I will be approaching this article with one assumption in mind: that blogs, as a form of presenting fiction, have failed. Which is rather ironic, considering the amount of fiction blogs I’m reading today, both for pleasure and for work (I have an obligation to review for WFG); and also ironic because my usage of the term ‘blook’ may have to be revised, and for good. But I believe we’re looking at a future where blogs aren’t going to be the main form of Internet fiction consumption, and here’s why.

The first thing we have to think about is the nature of the blog. Blogs are time-intensive things, and they require constant and consistent updating to be of any attraction to the reader. I once spoke of this as a good thing: that blogs force writers to perform on-the-fly writing, and I still do believe that the form has some unparelled attractions, attractions that cannot be found in books or even in writing magazines. But let’s ask ourselves a question: if we accept that publishers are moving onto the Internet, and we accept that they’re going to be finding the best ways to present fiction online, then what are the odds that blogs will be their medium of choice? What are the odds that of the majority of novels put on the Internet would be in blog form, and that the readers will be most used to consuming their online fiction via blogs? Not much, I’d expect - publishers aren’t going to invest so much of their time and energy into a medium that requires just that - lots of time and energy. And to back that up - take a look at the experiments we’ve seen conducted by the big wigs - how many of them are in blog form? We Tell Stories and The Golden Notebook and Tor.com are all beautifully designed websites; websites designed with only one purpose in mind: to be read.^[1]

That is not to say that blogs are not designed to be read. But we have to admit that we’re facing a structural problem when we try to tell stories with blogs - there is a wealth of information we have to design around, and most writers don’t bother to design at all. Many of a blog’s original features were not built with storytelling in mind. When I see things like reverse-chronological archives and trackbacks and comments I think of diary writing and community, not books and paper. And while some of these blog features can be adapted to storytelling, most of them remain deadweight; obstacles that get in the way of the actual jumping into the story that we want readers to experience.

On a side note, I wonder if this is one of the reasons why online fiction has taken so long to get off the ground. A reader comes to a blog with a set of expectations in mind, expectations that they have to overcome when they’re dealing with a serialized fiction blog (not so with short stories, or flash fiction - for these, blogs are extremely well suited as a presentation form). Note that online comics are not posted in the blog format, they’re presented in specially designed websites that are built around the expected interaction between reader and comic. There are no deadweights; no obstacles. No unnecessary fluff.

The bottom line here is that readers will eventually get used to a form of digital prose presentation, and that form will probably not be blogs. And that leads us to the next question - what to change into?

In the last post by Gavin we talked about how and why a publishing industry slump will help online fiction. In the comment storm that followed James of JPS/fact presented a counter-argument as to why online fiction is not yet an alternative to the traditional publishing world. James and I were supposed to do a Q&A post on Novelr, but due to time constraints (mine, mostly) we have settled on me writing this post, with him editing it. The arguments and ideas forthwith are, at the core, his.

First, a recap. We know that the traditional publishing industry is upon dark times - an obvious parallel would be the music industry, which was grappling with piracy and the Internet before iTunes came along and blew everything up. In the previous post Gavin wrote that the time is ripe for a similar thing to happen in Book World - and I agree with him. But before we begin discussing how best to blow things up let us talk about the challenges that are unique to us - and online fiction - in particular.

Quality

The first point James brings up is that online fiction suffers from chronic quality drought. The problems we have with quality are two fold: first of all we do not have a legion of editors, proofreaders, people who are familiar with text and who constantly hound at authors (again and again and again) to polish up, jettison chapters, rewrite characters, rethink themes and the sort. Secondly, we have little (as yet) serious works in online fiction. Traditional print fiction does not suffer from these problems - their editorial processes are so tight we accuse them (rightly, it seems) of being patronizing to new authors, and I’ve personally lost count of the amount of Book Awards designed to promote an ever-escalating bar of quality for new novels. They also have an old, long-standing gauntlet of academics and critics through which new novels are thrown into … online, all we have is The Blooker Prize.

How are we faring on these points? Not very well, I’m afraid: we’re still figuring out an editing process for online fiction (in the comments section we’ve got a lot of talk about readers being editors - I do think, however, that there is a limit to the effectiveness of this method) - however, as for quality I am confident we will win out in the end. The quality of blooks now are a lot better than they were one year ago, when I first started Novelr - and as we continue to experiment with the form and the function of the screen we will only get better and better at presenting stories online.

Accessibility

Online fiction isn’t as portable as the dead-tree version. We need batteries, we need a screen; that screen isn’t easy on the eyes; we have yet to build a globally accepted standard for electronic books. I have dealt with this problem before on Novelr: like James, I believe it is impossible to port an offline work to the digital world without significant change. Rather, writing has to be tweaked to suit the way we read things on a screen. And that’s leaving out things like hypertext and images - which, used wisely, help boost the immersive power of a story.

We have another problem in this area, however: did you know that only 27% of Internet users read blogs? And if we look at reading in a broader sense we have to admit that we are losing our kids to video, music and games. How many Gen Ys know the pleasure of turning to the last page of a book? If they do read, it is in bites - on blogs and newspaper websites, never more than a few lines of information. We will have to fight to get them to realize stories are another form of entertainment - just because they don’t like the reading they do in school doesn’t mean that reading isn’t fun.

But back to the technology - despite what most critics say I believe we’re in a far better position than we care to admit. I am writing this on a beautiful glossy LCD screen, and Amazon’s Kindle makes some headway in solving the screen and battery problem, though it is too expensive and too rare at the moment for any real impact. But this is what I am excited about: I am following a little known technology called Seadragon very closely - below is a demo of the technology being put to its paces in front of a live audience. My breath caught as I watched it. Tell me if yours does as well.