Say you’ve finished a major arc of your online novel. You want to turn aforementioned arc into a download, and perhaps make that available for purchase from the store section of your site. From here on, however, you’re met with two problems: 1) you’ll have to convert your text to an appropriate ebook format; and, 2) which one?

The ebook format fiasco is sometimes called ‘the tower of eBabel’, and for good reason: there are too many of them. But because we deal in digital fiction, and because ebooks are fast becoming viable models of distribution, we need to consider the sticky question of which ebook format, and why. This post attempts to answer that question. (Note that this is quite difficult to answer without looking into the future, simply because it is unclear if there’s ever going to be a victor in the ebook format wars. But I’ll get back to that in a bit.)

Context

E-book formats are no longer created from scratch. In most cases, the ebook maker - regardless of whether it’s a vendor or an open-source project - will decide to adapt and use an existing format, or to have some underlying programming language to make coding the format easier. Today, that language is often XML, or eXtensible Markup Language. Before we talk about the various ebook formats in proper, it’ll be good to talk a little about XML, and why it’s so popular as an underlying language.

The answer to that lies in XML’s name. ‘Markup’ and ‘Language’ are pretty self-explanatory; it tells us that XML is a programming language that consists primarily of markup tags, much like HTML.^[1] In fact, an XML document looks pretty much like any HTML page, the only difference being that XML is powerful enough to define and shape other languages ^[2]. But unlike HTML, XML is extensible. This means that XML allows you to define and create your own tags. For example, if I were an e-book-format creator, I can easily create and define <title> as a tag describing the title of an e-book. <title> doesn’t actually exist in XML. However, because XML is extensible, I can create what is effectively a whole new platform for my e-book format, and it’ll contain <title>, and whatever other tags I see fit to use. All I have to do is to define them, so that my ebook reader will understand which bits are which, and treat those sections accordingly.

You can tell that XML is useful precisely for this flexibility of form and function. The language is now used for many, many things - sometimes even as the foundation for web services to send requests and responses, behind the scenes, server-to-server. And if you take a look now at even the simplest of RSS feeds, you’ll find a language that is defined - and made possible - through XML.

Most of the major ebook formats today are all built upon some foundation of XML. The ePub format, widely tipped to become wide-spread, is built on a strong XML base. The Amazon Kindle format is built on a modified version of the Mobipocket ebook platform, which is in turn built on XHTML (with a dash of javascript/frame support). So is the format used by the new Sony Reader, though that’s known as the Sony BBeB. The conclusion you can take away from this is that sooner or later, XML will become a major part of your workflow regardless of which ebook format ends up as the eventual winner of eBabel. There’s no running away from it. The good news is, however, that XML is a remarkably convertible format. It’s going to be easier and easier to work with as most major software vendors make the jump to XML-based files; case in point: Microsoft Word’s new docx format is built on XML, and it’s not very hard to convert XML to other formats - say, PDFs, or HTML, or an XML-based ebook format of your choice.

The e-book Formats

So let’s get started. The following are the e-book formats in use today, ones that I believe still have a fighting chance of becoming the format of the known universe.

1. Amazon Kindle’s AZW. The Kindle uses Amazon’s proprietary AZW format, but can read unprotected Mobipocket e-books, HTML, Word documents and plain text (.txt) files. You convert to AZW using Amazon’s online Digital Text Platform, and you format your e-book using rudimentary HTML. AZW supports DRM (unfortunately) and is built around the Mobipocket format - though, confusingly, DRM-protected Mobipocket files cannot be read on the Kindle, because they’re not exactly one and the same. Is it worth it? Publishing your work in the AZW format grants you immediate access to the Amazon online store, where a number of online writers have been making a decent sum selling their work … some of which have been regularly hitting the top 10 bestseller lists for Kindle e-books. So … yes, it’s worth it.

2. Sony Reader’s BBeB, which stands for Broadband eBooks, is perplexing: Sony does not offer any tools to convert to the format, making the Sony Reader a closed medium to all but the biggest of publishers. In fact, the only way to publish for the Reader is via RTF or PDF … but XML to PDF conversions aren’t solid, not at the moment, and RTF limits your formatting options (it’s hardly better than a .txt file, to be honest). And there is at least one unofficial converter to BBeB, but Sony’s lack of support for writer releases is discouraging at best. Is it worth it? No.

3. Mobipocket (also known as mobi). The Mobipocket format was originally created by Mobipocket SA, a French company, in 2000, which was then bought over by Amazon in 2005. It’s been around for quite a bit, and it’s probably the only ebook-ish format at the moment that can claim full multi-platform compatibility. It runs on just about everything: the Kindle, the Palm OS, Symbian, Windows, Mac, and on the iPhone (the Stanza reader allows you to read Mobi books, though it was recently bought over by Amazon and is now in a vague sort of flux). It is, however, not very popular, and there doesn’t seem to be a captive audience or a community built around the format. A quick snoop around the official Mobipocket site confirms this. Why? I’m not sure, not at the moment (and I’m still looking for proper mobi-related numbers) - but a surprising amount of traditional publishers offer their ebooks in a mobi format. Is it worth it? This is hard to say. On one hand, the Mobipocket software suite is completely free, and it’s old enough to make conversion and formatting very easy on the writer. But the truth is that it’s not an exciting format to talk about, and this lack of excitement can probably be attributed to a lack of Mobipocket users … even with free software for just about every platform. And if you’re not likely to get serious ebook readers on Mobipocket (and you can’t sell mobi ebooks on Amazon for Kindle, anyway), then I guess it’s not worth it to spend so much time and energy on a format not many people would use in the first place.

4. ePub originally started off as the OEB (Open eBook) initiative. ePub is currently tipped to be the next big ebook format, if only because it’s backed by a loose consortium of publishers, writers, and programmers, who are tied together in the IDPF, or what is known as a ’stardards and trade organization for the digital publishing industry’. As mentioned earlier in this article, ePub is built on XML, and so the IDPF leaders are currently trying to push it as a distribution standard for e-books. This means a couple of very interesting things. If the ePub people have their way, publishers will no longer have to produce e-books in different formats for different e-book vendors; they publish in just ePub, and demand that everyone else (say, Amazon) convert ePub to their own proprietary format. And it’s really simple to do that, primarily because ePub’s built on a nearly 100% XML base - itself a highly convertible format. Is it worth it? As of late 2008 Sony announced that their reader would now support the ePub format, and publishers (or at least, the ones who have vested interest in a digital book future) have been relatively supportive of ePub over others. If the IDPF people get their way and ePub becomes the industry standard (or even if it becomes just a distribution standard), ePub would well be worth it. I’m fairly optimistic that ePub will win - at the very least, I want it to win - but the road to that future is far from clear-cut: Amazon has yet to announce any plans about ePub compatibility. They’re the one major player who’s yet to come around to ePub, and for what it’s worth - I think that it’s going to take a bit of time, some elbow grease, and a lot of arm wrestling to get them to see things from the publisher’s point of view. But give it time. It should happen … eventually.

5. Adobe’s PDF format is probably the most known amongst the e-book formats I’ve discussed so far^[3]. There’s not much to talk about: PDFs are simple, familiar, and easy to use regardless of medium, plus they’ve been around long enough for everyone to know, more or less, what a pdf file looks like. And because the PDF format is so old, it’s not likely that you’ll ever meet anyone with a computer that can’t read the PDF file format. Is it worth it? Hell, yes.

The Format That Wins

I want to make a case here that the primary ebook format we’re going to work with is probably going to be whichever ebook format wins on the iPhone. The Apple developer conference, WWDC, happened not very long ago, and several very interesting things became clear during that conference, most of it worrying news to the rest of the mobile phone industry, but good news for the rest of us. Here’s what Daring Fireball’s John Gruber has to say:

On the whole, there was a palpable sense that the iPhone is a peer to the Mac in Apple’s eyes. This isn’t about counting how many sessions were devoted to each. Nor is it an indication that the Mac as a platform is slowing. Quite the opposite in fact — Apple is selling more Macs than ever, and, knock on wood, there’s a strong consensus amongst developers that Snow Leopard is going to be the best release of Mac OS X yet. It’s simply that for however fast the Mac is growing, the iPhone is growing far faster.

But the two platforms are symbiotically intertwined. The Monday schedule at WWDC is static. In the morning comes the keynote, which the press attends and where all public announcements are made. After lunch, though, there comes what is effectively a second keynote, this time with material aimed squarely at developers. A technical keynote, as compared to the morning’s marketing keynote, if you will. This technical keynote has for as long as I can remember been titled “Mac OS X State of the Union”. This year the title changed to “Core OS State of the Union”.

Hence the symbiosis: Apple now has two full-fledged developer platforms, Mac OS X and iPhone OS, derived from one core system. Neither felt more important than the other this year at WWDC, which is remarkable considering that one of them hadn’t even shipped two years ago.

But look at their vectors — their relative rates of growth — and ponder how much longer until WWDC begins to feel like an iPhone developer conference with a Mac developer track. My answer: next year. In other words, I think it will have taken just three years for the iPhone to supplant the Mac as Apple’s primary platform. By 2011 it will be obvious.

It’s simply a matter of users. During Phil Schiller’s keynote, he showed a graph of the “OS X” user base over time, with steady growth over the first part of this decade followed by a sharp jump from 25 to 75 million over the past two years. This figure was widely mis-cited, however, as showing growth in “Mac OS X” users. It did not. The graph said “OS X”, not “Mac OS X”, and what Apple meant to show were the combined number of users of Mac OS X and iPhone OS. It was a very misleading and poorly-designed chart.

This doesn’t prove anything on its own, but stick with me for a bit. I’ve been seeing several articles arguing the point that AT&T isn’t providing immediate MMS and tethering support due to fear that their network would crash the very instant a million or so iPhone users decide to connect their devices. And I’ve noticed that the iPhone is itself a remarkably tactile platform, one perfect for reading books, and that we’ve already seen a number of apps showing us just that: that reading, and reading on your iPhone, is one hell of a revelatory experience. We’ve also been hearing rumours of an Apple tablet, with all the touchy goodness associated with their current multi-touch technology, and having that released in the not-too-distant-future would mean bringing the tactile interface to a fully-fledged operating system. And that, lastly, all those people connecting to an online network on such a small device will be a community of captive, fanatical users limited by the processing capabilities of their phones, but not by their phone’s features … making the iPhone all at once better than any ebook reader out there (cough the Kindle cough) but also perfect for reading text on the go.

But all of the above are small, fragmented pieces of information, hardly worth talking about, individually. It’s when you look at them from a broader perspective that things begin to become a lot more exciting, particularly from a digital-fiction point-of-view. Allow me to pull it all together for you: Apple sees the iPhone as a peer to their traditional Mac platform; the iPhone is a superior tactile device perfect for on-screen reading; the iPhone has a fanatical userbase that is connected to the Internet, one that downloads and consumes content through the iPhone itself; and Apple is a master at enabling 3rd-party (software) innovation. Put two and two together and you’d realize that this platform is ready for just the right ebook app^[4] to come along, and whichever one it is - be it Amazon’s Kindle app, or an Eucalyptus-type reader, or even one that we’ve never heard about - whichever one that is, that app will be the turning point that defines our industry. Want to know which format you should end up supporting? Watch the iPhone, and watch it closely.

____________________________________________________
^1. HTML isn’t really a programming language, but XML resembles it in the sense that both have very simple opening and closing tags as a foundation, like, say: <head></head> or <blockquote></blockquote> ↩

^2. Don’t worry too much about how XML works with other languages - that bit’s not relevent to this article ↩

^3. Though I must note here that the PDF is really more of a document format, not an ebook one. ↩

^4. This is dependent on one more factor: the app must have seamless integration with an online store, which in turn must be stocked with a good collection of ebook titles. In this aspect, at least, Amazon seems to have a clear lead, but no more so than if Apple decides to enter the ebook market themselves. If they do, or if some publishers decide to take things into their own hands and cobble together an online store/app combination, then I’m willing to bet that things will get very interesting, very fast. ↩

I’ve spent the last couple of days working on Novelr’s first collaraborative project. What this project does is it attempts to answer the question: ‘why do you read online fiction?’ and most of it is still, I must admit, in bits and pieces. But let’s examine the answers to that question, and ask ourselves some other related, and difficult, questions about this field we’re in.

For instance, let us consider that a large amount of people reading online fiction are writers themselves. One of the main community efforts in Novelr has been about how we can get more readers (ie: non-writing, non-creating people) to the medium, to consume what we writers are publishing. We want consumers of online fiction, and we must admit that ideally, we want an audience who are not participants - who do not produce works of fiction themselves. So what does that mean? It means that we’re currently writing for other writers. What is troubling with that assumption? Does it tick you off that the only reason other writers are reading your work is because they, too, want to be read by you? I’m now talking about an I’ll read your work if you read mine policy, and indeed that very topic has been explored on Novelr before in a guest post. But what’s wrong with it? Are you, like some of us, happy that you’re been read, to hell with the writer/reader dichotomy? What does an acceptance of this situation mean?

The first thing that springs to mind when we talk about an audience of equal creators is the blogosphere. People write blogs for a small audience, and it’s highly likely that a portion of that audience are bloggers themselves, and that you read and comment on their blogs to reward them for coming to visit your blog. The more successful blogs (say, Techcrunch) have a larger reader to blogger ratio, and they return a smaller amount of comments than a less successful blog (say, your Mum’s) would. Another example of a community of equal creators is the photo sharing site Flickr. Your contacts post photos and you post photos and everyone looks and comments at each other’s photos because, like us, I’ll read yours if you read mine.

The upshot of my above paragraph is that an audience of equal creators is the accepted norm in many areas of the online world. It is Internet culture. And even if this were not true, and that your blog commands a small readership of non-bloggers, consider: what is to prevent any or all of them from starting up their own blogs? Nothing? Nothing. In a medium where the barriers to entry (or creation) are almost nil, a community of creators are quite inevitable. Taking all of the above into consideration, and also taking into consideration that by the very act of writing your blook you are inspiring your readers to start their own blooks, are we likely, as a bunch of writers, to ever find an audience of ‘just’ readers? Is it alright if we don’t? What differences are there if we compare this model to the model of the book, the publisher, and the bookstore? 

You’ll find the answers unflattering, I believe, and I’d rather not answer them for you. You can tell me your thoughts in the comments area of this post. But here’s something to chew on before I step back: there may well come a day where the amount of people who want to write books outnumber the amount of people who want to read them. Indulge me and close your eyes: imagine this book future for a little while. Now wouldn’t that be strange? Yes, I can hear your voices now: that would be strange indeed.

In the last post by Gavin we talked about how and why a publishing industry slump will help online fiction. In the comment storm that followed James of JPS/fact presented a counter-argument as to why online fiction is not yet an alternative to the traditional publishing world. James and I were supposed to do a Q&A post on Novelr, but due to time constraints (mine, mostly) we have settled on me writing this post, with him editing it. The arguments and ideas forthwith are, at the core, his.

First, a recap. We know that the traditional publishing industry is upon dark times - an obvious parallel would be the music industry, which was grappling with piracy and the Internet before iTunes came along and blew everything up. In the previous post Gavin wrote that the time is ripe for a similar thing to happen in Book World - and I agree with him. But before we begin discussing how best to blow things up let us talk about the challenges that are unique to us - and online fiction - in particular.

Quality

The first point James brings up is that online fiction suffers from chronic quality drought. The problems we have with quality are two fold: first of all we do not have a legion of editors, proofreaders, people who are familiar with text and who constantly hound at authors (again and again and again) to polish up, jettison chapters, rewrite characters, rethink themes and the sort. Secondly, we have little (as yet) serious works in online fiction. Traditional print fiction does not suffer from these problems - their editorial processes are so tight we accuse them (rightly, it seems) of being patronizing to new authors, and I’ve personally lost count of the amount of Book Awards designed to promote an ever-escalating bar of quality for new novels. They also have an old, long-standing gauntlet of academics and critics through which new novels are thrown into … online, all we have is The Blooker Prize.

How are we faring on these points? Not very well, I’m afraid: we’re still figuring out an editing process for online fiction (in the comments section we’ve got a lot of talk about readers being editors - I do think, however, that there is a limit to the effectiveness of this method) - however, as for quality I am confident we will win out in the end. The quality of blooks now are a lot better than they were one year ago, when I first started Novelr - and as we continue to experiment with the form and the function of the screen we will only get better and better at presenting stories online.

Accessibility

Online fiction isn’t as portable as the dead-tree version. We need batteries, we need a screen; that screen isn’t easy on the eyes; we have yet to build a globally accepted standard for electronic books. I have dealt with this problem before on Novelr: like James, I believe it is impossible to port an offline work to the digital world without significant change. Rather, writing has to be tweaked to suit the way we read things on a screen. And that’s leaving out things like hypertext and images - which, used wisely, help boost the immersive power of a story.

We have another problem in this area, however: did you know that only 27% of Internet users read blogs? And if we look at reading in a broader sense we have to admit that we are losing our kids to video, music and games. How many Gen Ys know the pleasure of turning to the last page of a book? If they do read, it is in bites - on blogs and newspaper websites, never more than a few lines of information. We will have to fight to get them to realize stories are another form of entertainment - just because they don’t like the reading they do in school doesn’t mean that reading isn’t fun.

But back to the technology - despite what most critics say I believe we’re in a far better position than we care to admit. I am writing this on a beautiful glossy LCD screen, and Amazon’s Kindle makes some headway in solving the screen and battery problem, though it is too expensive and too rare at the moment for any real impact. But this is what I am excited about: I am following a little known technology called Seadragon very closely - below is a demo of the technology being put to its paces in front of a live audience. My breath caught as I watched it. Tell me if yours does as well.