Why GitHub is Important for Book Publishing

How do you organize large numbers of people for a common purpose? For millenia, the answer has been some sort of hierarchical organization. An army, or a feudal system topped with a king. To reach global scale, these hierarchies propagated customs and codes for behavior: laws, religions, ideology. Most of what you read in history books is really the history of these hierarchies. It wasn't possible to orchestrate big efforts or harness significant resources any other way.

In the 20th century, mass media redistributed much of this organizational power. In politics, charismatic individuals could motivate millions of people independently of the hierarchies that maintain command and control. But for the most part, one hierarchy got swapped for another. In business, production innovations such as Henry Ford's assembly line needed the hierarchy to support the capital investments.

I think the history of the 21st century will be the story of non-hierarchical systems of human organization enabled by the Internet. From this point of view, Wikipedia is particularly important not only for its organization of knowledge, but because it demonstrated that thousands of people can be organized with extremely small amounts of hierarchy. Anyone can contribute, anyone can edit, and many do. Bitcoin, or whatever cryptocurrency wins out, won't be successful because of a hierarchy but rather because of a framework of incentives for a self-interested network of entities to work together. Crowdfunding will enable resources to coalesce around needs without large hierarchical foundations or financial institutions.

So let's think a bit about book publishing. Through the 20th century, publishing required a signification amount of investment in capital- printing presses, warehouses, delivery trucks, bookstores, libraries, and people with specialized skills and abilities. A few large publishing companies emerged along with big-box retailers that together comprised an efficient machine for producing, distributing and monetizing books of all kinds. The transition from print to digital has eliminated need for the physical aspects of the book publishing machine, but the human components of that machine remain essential. It's no longer clear that the hierarchical organization of publishing is necessary for the organization of publishing's human effort.

I've already mentioned Wikipedia's conquest of encyclopedia publishing, by dint of its large scale and wide reach. But equally important to its success has been a set of codes and customs bound together in a suite of collaboration and workflow tools. Version tracking allows for easy reversion of edits. "Talk pages" and notifications facilitate communication and collaboration. (And edit-wars and page locking, but that's another bucket of fish.)

Most publishing projects have audiences that are too small or requirements too specific to support Wikipedia's anyone-can-edit-or-revert model of collaboration. A more appropriate model for collaboration in publishing  is one widely used for software development.

Modern software development requires people with different skills to work together. Book publishing is the same. Designers, engineers, testers, product managers, writers, and subject domain experts may each have an important role in creating a software application; authors, editors, proofreaders, illustrators, designers, subject experts, agents, and publicists may all work together on a book. Book publishing and software can be either open or proprietary. The team producing a book or a piece of software might number from one to a hundred. Books and programs can go into maintenance mode or be revised in new editions or versions. Translation into new languages happens for both. Assets from one project can be reused in other projects.

Open source software has been hugely successful over the past few decades. Along the way, an ecosystem of collaboration tools and practices has evolved to support both open source development and software development in general. Many aspects of this ecosystem have been captured in GitHub.

The "Git" in GitHub comes from git, an open source distributed version control system initially written by Linus Torvalds, the Linus behind Linux. It's fast, and it lets you work on a local code repository and then merge your changes with a repository stored somewhere else.

In just two sentences, I've touched on several concepts that may be foreign to many book publishing professionals. Microsoft Word's "track changes" is probably the closest that most authors get to a version control system. The big difference is that "track changes" is designed to facilitate collaboration between a maximum of two people. Git works easily with many contributors. A code "repository" holds more than just code, it can contain all the assets, documentation, and licenses associated with a project. And unlike "track changes", Git remembers the entire history of your project. Many book publishers still don't keep together all the assets that go into a book. And I'm guessing that publishers are still working on centralizing their asset stores instead of distributing them!

Git is just one of the useful aspects of GitHub. I think the workflow tools are perhaps more important. Developers talk about the workflow variants such as "git-flow" and "GitHub-flow", but the differences are immaterial to this discussion. Here's what it boils down to: Someone working on a project will first create a "feature branch", a copy of the repository that adds a feature or fixes a bug. When the new feature has been tested and is working, the changes will be "committed". Each set of changes is given an identifier and a message explaining what has been changed. The branch's developer then sends a "pull request" to the maintainers of the repository. A well crafted pull request will provide tests and documentation for the new feature. If the maintainers like the changes, they "pull" the changes into the main branch of the repository. Each of these steps is a push of a button on GitHub, and GitHub provides annotation, visualization and commenting tools that support discussions around each pull request, as well as issue lists and wiki pages.

The reason the workflow tools and the customs surrounding their use are so important is that anyone who has used them already knows how to participate in another project. For an excellent non-programming example, take a look at the free-programming-books repository, which is a basic list of programming books available online for free.  As of today, 512 different different people have contributed a total of 2,854 sets of changes the the repository, have expanded it to books in 23 languages, and have added free courses, screencasts and interactive tutorials. The maintainers enforce some basic standards and make sure that the list is free of pirated books and the like.

It's also interesting that there are 7,229 "forks" of free-programming-books. Each of these could be different. If the main free-programming-books repo disappears, or if the maintainers go AWOL, one of these forks could become the main fork. Or if one group of contributors want to move the project in a different direction from the maintainers, it's easy to do.

Forking a book is a lot more common than you might think. Consider the book Robinson Crusoe by Daniel Defoe. OCLC's WorldCat lists 7,459 editions of this book, each one representing significantly more effort than a button push in a workflow system. It's common to have many editions of out-of-copyright books of course, but it's also becoming common for books developed with open processes. As an example, look at the repository for Amy Brown and Greg Wilson's Architecture of Open Source Applications.  It has 5 contributors, and has been forked 58 times. For another example of using GitHub to write a book, read Scott Chacon's description of how he produced the second edition of Pro Git. (Are you surprised that a founder of GitHub is using GitHub to revise his book about Git?)

There's another aspect of modern software engineering with GitHub support that could be very useful for book publishing and distribution. "Continuous integration" is essential for development of complex software systems because changes in one component can have unintended effects on other components. For that reason, when a set of changes is committed to a project, the entire project needs to be rebuilt and retested. GitHub supports this via "hooks". For example, a "post-commit" hook can trigger a build-test apparatus; hooks can even be used to automatically deploy the new software version into production environments. In the making of a book, the insertion of a sentence might necessitate re-pagination and re-indexing. With continuous integration, you can imagine the correction of a typo immediately resulting in changes in all the copies of a textbook for sale. (or even the copies that had already been purchased!)

A number of startups have recognized the applicability of Git and GitHub to book publishing. Leanpub, GitBook, and Penflip are supporting GitHub backends for open publishing models; so far adoption has been most rapid in author communities that already "get" GitHub, for example, software developers. The company that is best able to teach a GitHub-like toolset to non-programmers will have a good and worthy business, I think.

As more people learn and exercise the collaboration culture of GitHub, new things will become possible. Last year, I became annoyed that I couldn't fix a problem I found with an ebook from Project Gutenberg. It seemed obvious to me that I should put my contributions into a GitHub repo so that others could easily make use of my work. I created a GitHub organization for "Project GitenHub". In the course of creating my third GitenHub book, I discovered that someone named Seth Woodward had done the same thing a year before me, and he had moved over a thousand Project Gutenberg texts onto GitHub, in the "GITenberg"  organization. Since I knew how to contribute to a GitHub project, I knew that I could start sending pull requests to GITenberg to add my changes to its repositories. And so Seth and I started working together on GITenberg.

Seth has now loaded over 50,000 books from Project Gutenberg onto GitHub. (The folks at Project Gutenberg are happy to see this happening, by the way.) Seth and I are planning out how to make improved quality ebooks and metadata for all of these books, which would be impossible without a way to get people to work together. We put in a funding proposal to the Knight Foundation's NewsChallenge competition. And we were excited to learn that (as of Jan 1, 2015) the Text Creation Partnership has added 25,000 texts from EEBO (Early English Books Online) on GitHub. So it's an exciting time for books on GitHub.

There's quite a bit of work to do. Having 50,000 repositories in an organization strains some GitHub tools. We need to figure out how to explain the GitHub workflow to potential contributors who aren't software developers. We need to  make bibliographic metadata more git-friendly. And we need to create a "continuous integration system" for building ebooks.

Who knows, it might work.

Update January 30: Our NewsChallenge proposal is being funded!!!

In 2013, eBook Sales Collapsed... in My Household.

2013 was not the year the ebook industry was expecting. We hoped that ebooks would continue their explosive year-on-year revenue growth, and that the replacement of print by digital would proceed apace. We suspected that the growth of ebook sales might moderate, because, as HarperCollins CEO Brian Murray told Publisher's Weekly, "Nothing grows by triple digits for too long." But just as CDs replaced vinyl and digital downloads replaced CDs, it seemed obvious that the the age of the printed book was nearing its end; the century of the ebook was dawning.

We got a few things right. Internationally, ebook sales growth was strong. Print continued its slow decline. Bookstores continued to close. But for some reason, ebook sales in the US stopped increasing. And even started declining!

There are many possible explanations for this turn of events. There are technicalities with the data collection, particularly with publishers such as Amazon's imprints that don't report their sales numbers. Young Adult sales dropped steeply, as there was no smash hit to follow on the huge success of Hunger Games. 50 Shades of Gray didn't turn out to be a lasting relationship. And there's been a downward trend on prices, particularly as publishers start to use dynamic pricing to stimulate sales. But it seems to me that something in the environment is changing, more than just a market maturation.

Amazon probably has enough data to understand what's happening, but they're notoriously opaque about reporting numbers. On the other hand, they're quite good about reporting to customers what they've bought. So I decided to analyze my own household's Amazon data. I had the impression that my family was spending less on ebooks, but I wasn't sure, because they still seem to spend all hours of the day reading. The results were kind of shocking.

The graph shows my household Kindle ebook purchases from 2009-2013. As you can see, 2013 marked a steep drop from the 2009-2011 peak years of about $1000 per year.

I don't buy Kindle ebooks myself (I buy ePub only, so I can hack on them) but other members of my household have bought quite a lot. The average price paid is about $7, and this has held quite steady. But in 2013, Kindle purchases stopped almost completely, and they were not replaced by purchases on other platforms.

Based on detailed "interviews" with the subject ebook purchasers, here are some non-factors in this collapse:

1. "Netflix-for-Books" services. Nobody but me has heard of them.
2. Kindle Owner's Lending Library. Despite an well-used Amazon Prime subscription, they haven't figured out how to use it for ebooks.
3. Our public library. Nobody but me has used it for ebooks.
4. Piracy. As if!

The two main reasons for this spending collapse turn out to be:

1. The Kindle acquired in early 2009 reached end-of-life due to a cheaply made power cord, and was replaced by an iPad. The lack of in-app purchase for the Kindle App has resulted in a significant impediment to Kindle purchases. The iBookStore has not attracted a single ebook purchase.
2. The iPad owner now spends the vast majority of her reading time in fan-fiction websites, mostly fanfiction.net and ArchiveOfOurOwn.org. Same for the iPad borrower, but a different mix of websites.

I find it worrying that the Justice Department pursued an big antitrust suit against Apple and 5 of the big 6 publishers, won, and despite making an issue of Apple's in-app purchase ban in iOS, it seems to have lost the argument with Judge Cote. We'll see how that turns out.

It's worth paying close attention to the fan fiction sites. After all, 2012's biggest revenue engine for the book industry, 50 Shades, was a repackaged fanfic. On an iPad with a decent internet connection, the fanfic sites work better than ePubs. They link and they script. Just try making a link from one ePub to another and you'll get my point.  They deliver content in smaller, more addictive chunks, and they integrate popular culture MUCH more effectively than books do, for reasons relating primarily to copyright. The authors are responsive and deeply connected to readers; they often ARE the readers!

There's a fanfic site to appeal to every reader; I highlighted Wattpad earlier this year. ArchiveOfOurOwn.org ("AO3"), a project of the Organization for Transformative Works, a non-profit, experienced the growth in 2013 that was missing from the ebook sector. The number of works hosted by AO3 doubled to just under a million works, covering almost 14,000 "fandoms". (A good example of a fandom is the "Dragonriders of Pern" fandom, which currently hosts 534 works). Fanfiction.net, an advertising supported site, hosts almost 2000 fandoms and over 1.3 million works, more than half of which are in the Harry Potter or Twilight fandoms. Game oriented discussion forums also engage in fanfiction. (Popular in my house is spacebattles.com)

My anecdata might be completely anomalous, although Amazon, a very data-driven company, seems to be aware of the same phenomena. They've been making the Kindle into a full featured tablet to go head-to-head with the iPad. They've also launched a fanfic site called Kindle Worlds, which has 15 worlds and 341 works.

Early stage venture capitalist Josh Kopelman says that many of the best opportunities for startups are not those in expanding markets. "We love investing in technologies and business models that are able to shrink existing markets. If your company can take $5 of revenue from a competitor for every $1 you earn – let's talk!",  he has written on his firm's website. Kopelman founded Half.com in the early days of the internet, a company which shrank the book market by getting people to resell the books they had just bought for a fraction of the price of a new book. Microsoft's Encarta shrank the Encyclopedia business from $1.2B to $600M before Wikipedia shrank the business by another 90%.

In 2014, I'm guessing it's the book publishing industry's time to shrink. A convergence of tech startups, tech monsters, and tech non profits seems to be ready for the assault. The fanfic sites, the Wattpads, the Project Gutenbergs and the Manybooks, the Readmills, the Leanpubs and the Smashwords (and I hope the Unglue.its); these are people building the foundations of a creative industry that will flourish even if the ebook sales collapse that I see around me spreads to your house as well.

Happy New Year!

PW: "Author as Entrepreneur"

My article for Publisher's Weekly on crowdfunding for books, Author as Entrepreneur, has emerged from behind the paywall. It's a follow-up of my blog post from April,  Book Publishing After the JOBS Act Revolution. An excerpt:

“From a business perspective, I think the business model of crowdfunding book equity is not practical,” said Ash Kalb, a tech lawyer and venture capitalist who founded Singularity & Co., which crowdfunds e-book editions of classic science fiction. “For 99.9% of books, the overhead costs of dealing with a large number of unsophisticated investors are going to be too high, and one lawsuit from a disgruntled investor wipes out the business. But really, it all depends on the bar the SEC raises with the rules, so we shall see.” 

In England, where equity crowdfunding is already legal, Web sites such as Seedrs and CrowdCube think they have found ways to address the problems. CrowdCube has raised over £6.7 million from more than 30,000 investors for 46 businesses. Seedrs has funded 21 startups with more than £1 million total. 

Figuring out how to churn out small publishing startups on an assembly line won’t be the only barrier to making crowdpublishing work. A bigger obstacle may be the changes in transparency that crowdfunded books require. 

“Have you ever seen a 10-K or a securities offering? asks Kalb. “Do you really want to list risk factors at the start of your book? ‘No one may like this book. I may be a bad author. My wife may leave me. Werewolves may not be as popular next year...’”

Since it was originally published in the print edition, the web version is missing the links, so here they are:

• Unglue.it
• PubSlush
• Unbound
• Singularity & Co.
• TenPages
• CrowdCube
• Seedrs
• Hollow World KickStarter

There have been some recent development on the securities regulatory front. The SEC has decided to drop its prohibition against "general solicitations" of securities for sale as long as they are available only to accredited investors, as required by the JOBS Act. (I don't understand how the SEC gets to decide whether to implement a law, just as I don't understand how Apple could have been an ebook monopolist before it sold its first ebook.) This is a LONG way from legalization of equity crowd funding, but at least some movement is happening.

Four Corners of the Sky: Chapters 39-41

This is installment 7 of my Big Library Reading diary, covering chapters  39-41, through the end of Part 2.

I had been reading Four Corners on my train commute into Manhattan for Book Expo America. But on Friday Overdrive pushed an iPad update that somehow left my Overdrive app in a half-updated state. So that slowed me down a bit. I need to get it in gear; I have only 5 days left on my checkout!

Maybe it's the influence of BEA, but there seemed to be a LOT of alcohol consumed in these 3 chapters. So I have a few things to say about that today.

Sgt. Dan Hart drinks ALMOND LIQUEUR????? Is that an alusion to a movie I didn't see? Because if Brad was drinking almond liqueur I would think nothing of it, but Dan Hart? What kind of love interest (I got that right, didn't I?) drinks almond liqueur? And after a bottle of Cuervo, too. Annie drinks pitchers of mojitos after saying she doesn't drink. Also, Brad's bottle of beer.

Speaking of mojitos, congratulations to the Evoke team, which won the Publishing Hackathon book discovery competion with their website that makes emotional connection of characters in books more salient. The mojitos at the afterparty were just AMAZING, they set this reading diary back about 10 chapters worth.

So Melissa Skippings is a vodka martini girl. Good for her.

Zemanta is recommending blog posts about Nutella Martinis. So they win the competition for drink discovery via books.

Next Diary Entry

• How to Make a Nutella Martini (crazyadventuresinparenting.com)
• Nutella Martini (neatorama.com)
• Famous cocktails: from silver screen to bars and clubs (brandsandfilms.com)
• Lavender Liqueur (annashortcakes.wordpress.com)
• Get creative with quince: a recipe for Quince Liqueur (toogoodtowastesa.com)

Hack the Publishing Hackathon

Why a publishing hackathon?

Book discovery needs innovation. It’s never been easier to get a book into a reader’s hands—just one click. But, with over 10,000 books published each year on every topic imaginable, how do people find out about them? There are fewer bookstores to help readers discover exciting new authors and ideas. There’s currently no digital experience that replicates the serendipity of browsing bookshelves. Recommendation engines are fairly primitive – they know what you bought, but they don’t know why. It’s a disruptive opportunity that hasn’t been explored.

Seriously, the sponsors of this event don't think book discovery has been explored? I guess they were too busy suing Mr. Google to notice that Google Books is a pretty good discovery tool. I suppose they never thought to ask Mr. Wikipedia how many books are published every year.

All in all, I find the description of this hackathon INSULTING to just about every developer that's worked in the general vicinity of the book industry.

Umm. Mr. Steinberger. If you and Perseus really want to promote discovery innovation, then perhaps you have heard of Goodreads? They're driving some decent discovery of books. Maybe it doesn't count if Mr. Amazon is buying them. Perhaps you've heard of Amazon? They popularized the "If you liked this, maybe you'll like..." feature that everyone in the publishing industry tries to copy. If you don't like Goodreads, maybe I can introduce you to LibraryThing, which has been driving valuable book discovery in more ways than I can list here. I know that "library" in their name is a big turnoff for your big 6 colleagues, but libraries are huge book discovery machines. I don't suppose you want them to disrupt anything. And umm DP.LA????

People mostly discover books by word of mouth. Some  innovators promoting social reading include Readmill (who had their own publishing hackathon) and (giving props to the NYC home team) ReadSocial and the stuff Bob Stein has been exploring. And Kobo, Copia and Zola are doing some amazing things to integrate book discovery with ebook selling and reading environments. I've written previously about Jellybooks' fresh approach to discovery.

And some more on libraries. When I was at OCLC, we worked on real simple problems like "how do you discover the other editions of the same book?" and we found that publishers had NO CLUE what they'd published 5 years previous. So yeah, we did our bit.

But I'm coming to the hackathon anyway. because despite the ridiculous framing, this event has some clueful backers. NYPL for one. Small Demons for two. And they're even wasting prize money on a new age library metadata thingy. (I might be wrong about the wasting part.)

I'm hoping that some people will be interested in rethinking ebook front matter. Unglue.it needs books to work better all by themselves. The best discovery instrument for a book is the GDMF book, to my mind. So let the book do some work. With a little javascript. And no more DRM, thank you very much!

Dandelion Publishing is the Worst Idea Ever

Spring has arrived in New Jersey. Newark's Branch Brook Park is home to the the nation's largest collection of blossoming cherry trees in the nation. You can't go outside without being stunned by something living and beautiful. Also, the weeds like it. I spent Sunday pulling dandelions. They have long, deep roots, and they love depositing thousands of baby dandelions anywhere you try to grow anything else.

Which reminded me of the stupidest thing I heard at London Book Fair two weeks ago. It was in Neil Gaiman's "Dandelion" talk at the Digital Minds conference. Apparently Gaiman thought it went over like a lead balloon, but I was there and the London digital publishing establishment just loved it. You can watch it here:




Or (quicker) read the transcript at the Falls into Writing blog :

Mammals, which all of us are, I hope, put a tremendous amount of effort into raising our children, into what we do. Child – it’s, you know, a solid 15, 16, 17, 18 years. They go onto higher education.  It could be 22, 23 years, and you’re still raising these things. And you put effort and you put life into it. Dandelions don’t care. They just have thousands of seeds, and they throw them to the wind. And there’s a level in which, as time goes on, I’m enjoying throwing things to the wind. 

Two days ago, I was in New York, bored over lunch, and I started drawing on the paper tablecloth. And I just did a drawing on the paper tablecloth. And at the end of the meal, I was getting up, and my wife looked down, and she said, ‘Are you doing anything with that?’ 

And I said, ‘No, just leaving it behind for them to throw away.’ 

And she said, ‘We should do something with it. It’s a great drawing.’ So she stole it, folded it up, walked outside, put it under a rock by the restaurant, and twitted a photograph of herself putting it under the rock and the location. And she said, ‘You can retweet that. Some of your fans, somebody’ll find it. It’ll make them happy.’ 

And I said, ‘Okay.’ And we walked the 70 seconds to our hotel. Went up to our room. 

And she said, ‘Oh. No point in you retweeting it. Somebody’s already found it.’

and later:

The truth is, whatever we make up is likely to be right. It’s time for dandelions. Embrace the old as we embrace the new because we’re on the frontier, and there are no rules on the frontier.

The reason this is a bad idea for publishing, and maybe the reason Gaiman thought he had bombed, is that book publishing is all about suppressing the weeds and fertilizing the flowers. Suppressing weeds is fashionably called "curation" and the fertilizer is politely called "marketing".

It's only the dregs of publishing that makes money off of weeds. And dandelions are weeds in most contexts. The righteous publishers treat their books like children, and yes, they put lots of effort into them. My organic gardening friends assure me that dandelions are delicious in salads; by the same token, I have a friend who works at Marcal who assures me that books make excellent pulp.

Even the most inexperienced self-publisher knows that scattering seeds to the wind is a recipe for financial disaster in book publishing. A seedling book of the very best variety requires careful nurture. To think you can hide a masterpiece under a rock and expect anything other than worms is the height of narcissism.

Also, dandelions are spammers. Please don't be a spammer.

(To Neil Gaiman: the rest of your talk was just brilliant; if you want to unglue one of your books, we'll be happy to fawn over you like teenage beliebers.)

Book Publishing After the JOBS Act Revolution

On Monday, the US Senate confirmed Mary Jo White as the new head of the Securities and Exchange Commission (the SEC). The book industry will never be the same. No, that's not a non sequitur.

A year ago, a bipartisan majority in Congress passed the "Jumpstart Our Business Startups" or JOBS Act, and President Obama signed the legislation on April 5, 2012, thus accomplishing the first major overhaul of the nation's securities laws in over forty years. Perhaps the most far-reaching provision of the JOBS Act is the legalization of equity crowd funding. But not quite yet. The SEC has not published the rules that would implement the JOBS Act, and so it's not yet legal to crowd fund a business, unless your crowd is all millionaires.

#strangersproject

"But what about Kickstarter?" you may be asking yourself. "Isn't that legal?"

Kickstarter and businesses like it (Unglue.it, which I run, is one) take great pains to prevent projects on the site from offering any sort of tangible equity. Instead of equity-based crowd funding, these site offer rewards-based projects and products. People become backers on Kickstarter because of the rewards they get if the project a success. In many ways, Kickstarter is just a site where creators pre-sell products that don't exist. A supporter on Unglue.it might get a signed manuscript or some other reward,  but it's the intangible equity of making a book free to the world that drives the site.

By contrast, the backers on an equity-based crowd funding platform could receive shares in a business.  So on a crowd-funding site for books, the backers might become investors in the individual books, and would make money if the books turned a profit. If you happened to invest in Fifty Shades...

From an author's point of view, this would completely change the game of publishing. Instead of relying on an insider network of literary agents who market book properties to publishers, an author would put proposals on a book-funding site. They'd line up a team of free-lance editors, illustrators, designers and developers, and the literary proposal would look like a mini-business plan. (More likely the author would seek help from a new class of social-media-savvy literary product-manager-agents who specialize in marketing to the crowd funders.) The crowd – probably consisting of voracious readers hoping to earn a little money from their obsession- would fund the books that had the best chance of success. That same crowd would be the marketing vanguard for the book when it's finally published; how can corporate publishing compete with that?

There's nothing intrinsic about crowd-funding that restricts this sort of fund-raising to unknown authors looking for a first advance. The JOBS act restricts the amount raised from "unqualified investors" to $1,000,000, so the really big name authors would have to tap the "qualified investor" funding market. (An individual with more than a million dollars in assets excluding home and vehicles is considered "qualified")

Once equity crowd-funding becomes established for books (and it WILL happen!), incumbent publishing houses will have lost, at a stroke, their oligopoly on books as investment vehicles. Already, publishers are outsourcing their design, editorial, production, distribution and sales functions; providing capital is their last bastion of essential function. They will have to participate in the new markets or they will dissipate into irrelevancy.

The reason the SEC has not issued the new regulations implementing JOBS is apparently because the previous Chairwoman, Mary Schapiro, had reservations about opening the gates to crowd-funding. The SEC's mindset is to protect ordinary investors from being fleeced by Wall Street sharks.  One would assume that President Obama's new appointee will be motivated to implement what could be a signature accomplishment of the Obama administration. So Monday's confirmation of Mary Jo White could signal the start of a new era of American investment.

Tim Draper at SVCrowdFund
by Return On Change

I had not paid attention to the possibilities of equity crowd funding for books until I attended last week's "SVCrowdFund" conference in Palo Alto. There I met many of the people who had been instrumental in getting the JOBS Act introduced and passed. There was electricity in the air, and my head is still spinning.

What's needed to make equity crowd funding for books a reality is a platform that has both crowd funding and publishing functions. Mostly, investors in books need to be protected from the sort of accounting shenanigans that prevent advances from earning out and royalties from amounting to more than a few pennies per copy. They also need some insurance against authors who don't deliver their promised manuscripts. All parties need sound legal agreements, business plan templates and investment entities. But above all, none of it will work without copious transparency and openness.

I've been trying to think some of these things through. It's a pretty big project, needing a variety of expertise. If you'd like to join me and others in some discussion, please let me know, and I'll organize something. Or leave a comment.

Resources
Community/Crowd Funding for Books

• Unglue.it
• PubSlush
• Unbound
• Singularity & Co.
• Knowledge Unlatched
• TenPages

Disaggregated Publishing

• Smashwords
• BiblioCrunch
• PressBooks
• CreateSpace
• LuLu

JOBS Act

• Text of the Bill (thomas.loc.gov)
• SEC impedes crowd-funding revolution (bankrate.com)
• Slow-moving SEC blamed for Blocking JOBS Act (politico)
• JOBS Act, One Year Later: Hang Tight, Equity Crowdfunding Is Coming (news.terra.com)
• Massolution Crowdfunding Report
• Crowdfunding Industry Celebrates First Anniversary of JOBS Act Despite Delays (techcrunch)

Random House's eBook Price Hikes are GOOD for Libraries. IF...

Random House, the last of the Big 6 Publishers to allow libraries to purchase and lend ebooks on a 1-copy-1-patron (pretend-its-print) basis, said last month that it was going to raise its pricing for libraries. The new pricing isn't set in stone, but Library Journal has reported that libraries are being asked to pay as much as three times the price of a print copy for a lendable ebook from Random House.

The general reaction from the library world was nicely summed up by ALA President Molly Raphael in an official press release: "In a time of extreme financial constraint, a major price increase effectively curtails access for many libraries, and especially our communities that are hardest hit economically."

Well, yes. But 5 years from now, libraries may well look back on Random's move and recognize it as the beginning of a new, healthier relationship between public libraries and trade publishers, one that recognizes libraries as an important player in the "reading ecology". Here are the reasons why I think it might happen:

• eBooks aren't books! So why should the price of a library-lendable eBook be locked to the price of a print book? Once the prices of lendable ebooks are allowed to float, market forces will move them up and down. For some books, high profile best sellers for example, the market price for a lendable ebook might be 5 or 10 times the print price. A year later, that same book would have to be steeply discounted to be sellable in the library market. Books without a buzz, or by a new author might be offered to libraries well below the print price, in an attempt to prime the market and spread the word.
• High prices for library-market books are nothing new! In academic markets where libraries make up a significant fraction of the buyers, prices are already over $100 per copy. That's because the  sales impact of inter-library loan and other forms of library sales-substitution is built-in to the price.
• Higher prices give libraries more leverage. This is the most important benefit of higher prices for  lendable ebooks. Libraries aren't being forced to buy the 3x ebooks- they will consider prices and their limited budgets before investing in them. They'll need to demand digital product features tailored to libraries to make them worth premium prices. 

This last point is the big IF. With its move, Random House has made clear what it wants out of a new relationship with libraries- more cash per copy. In return, libraries need to demonstrate what they expect for their money. What should libraries require from their premium ebooks in exchange for premium prices? Here's my list:

1. Portability - the ebooks shouldn't be locked to the distribution platform of a particular vendor; most libraries have existed longer than Overdrive, Adobe, Apple and Amazon combined and libraries would like to continue existing after those companies have been long forgotten. Their ebooks should persist as well.
2. Transferability -  libraries can make their ebook assets go a lot farther if they can be traded to other libraries or library consortia.
3. Privacy - libraries should never be forced to expose their users to the prying eyes of anybody!
4. Accessibility -  libraries will increasingly be relied on to provide text-to-speech and other accessibility technologies to users who need them.
5. Integrability - libraries don't want to be sources of friction, they want to provide integrated information environments. Library systems will increasingly provide capability such as annotation, discussion, advanced discovery tools and social interaction; they won't be able to do that if their ebooks are walled off behind third party DRM.

But, back in the real world,  most public libraries that offer ebooks are having difficulty keeping their digital shelves stocked due to overwhelming user demand for ebooks. If ebooks cost 3 times what they did last year, the availability will be 3 times worse. How can this situation get back into balance? I have three suggestions. First, if ebooks don't expire, as in the Harper-Collins scheme, the supply of ebooks will grow over time so that even if long wait times for hot titles are the norm, plenty of 5-year-old ebooks will be there to read for library users. Second, libraries can steer users to ebooks that don't have pretend-it's-print lending limits: those in the Public Domain or in the Creative Commons.

Here's an idea for a way that a smart publisher could help a library convert its print collection- offer a 1 for 3 (or 1 for 2) p for e  trade-in. The publisher's sales of new books would improve by suppressing competition from used books, and the library would gain inventory of older books to slake reader book-thirst.

Both libraries and publishers need to move on from backward-looking economic models. The time to start doing so is now. We can make it happen.

eBook Markets Need eBook Quality Standards

Yes, the Kindle is UL rated!

Underwriter's Laboratory (UL) issued its first standard, covering "tin clad fire doors", in 1903. It then became easier for architects to specify fire-resistant doors for new buildings, which no doubt was a boon to tin clad door manufacturers, who no longer had to compete with doors made with too-thin tin. The UL® labels now let consumers buy all sorts of electrical products without thinking about whether their new Amazon Kindle will burst into flames in the middle of Maharaja's Mistress.

Think about all the things you didn't have to think about today. If you nuked a mug of water for tea this morning, you probably didn't consider whether the microwave's magnetron would fry you. You probably don't even know that your microwave oven has a magnetron. Our modern civilization is built on being able to not think about these things. Quality standards such as those developed by UL help us to think less, and help marketplaces sell more.

Unfortunately, if you're an avid ebook reader, in 2011 you have to think more than you want to about ebook quality. When Neal Stephenson's new novel, Reamde, came out, early purchasers of the book were dismayed to find that it was rife with typographical errors. (But not the title. That "typo", for ReadMe, is intentional!) Amazon was forced to suspend sales.

I've been watching an important effort on ebook quality. It's worth supporting. The entry deadline for the Publishing Innovation Awards is this week, November 15. Entrants submit ebook files which are evaluated for quality, innovation and design. New this year is the "QED" seal, which is awarded to entrants that satisfy a checklist of basic ebook quality no-brainers:

1. Front matter: the title does not open on a blank page.
2. Information hierarchy: content is arranged in such a way that the relative importance of the content (heads, text, sidebars, etc) are visually presented clearly.
3. Order of content: check of the content to be sure that none of it is missing or rearranged.
4. Consistency of font treatment: consistent application of styles and white space.
5. Links: hyperlinks to the web, cross references to other sections in the book, and the table of contents all work and point to the right areas. If the title has an index, it should be linked.
6. Cover: The cover does not refer to any print edition only related content.
7. Consumable Content: The title does not contain any fill-in content, such as workbooks and puzzle books, unless the content has been re-crafted to direct the reader on how to approach using the fill-in content.
8. Print References: Content does not contain cross references to un-hyperlinked, static print page numbers (unless the ebook is intentionally mimicking its print counterpart for reference).
9. Breaks: New sections break and/or start at logical places.
10. Images: Art is appropriately sized, is in color where appropriate, loads relatively quickly, and if it contains text is legible. If images are removed for rights reasons, that portion is disclaimed or all references to that image are removed.
11. Tables: Table text fits the screen comfortably, and if rendered as art is legible.
12. Symbols: Text does not contain odd characters.
13. Metadata: Basic metadata for the title (author, title, etc.) is in place and accurate.

Next year, I hope they add a checklist item for typographical errors. If a publisher can produce print with minimal errors, there's no excuse to allow them in digital books. As the Reamde debacle showed, even typos can create significant customer service expenses for retailers.

A few years from now, it's likely that any ebook that doesn't meet these standards will be unsaleable; for now, a QED seal is a great way for publishers to realize the value of making a good digital product, and for readers to be able to think less.

Notes:

1. In building Unglue.it, we've realized that we need to give book lovers some assurance that the ebooks they support for ungluing will be of a quality that they will be proud to have contributed to. We'll point to QED as a reference point for the quality we expect from unglued ebooks.
2. I read the print version of Reamde. I thought the spin-up was Stephenson's best, but there was a lot of carnage as things spun globally out of control.

The United Nations of Reading

The Internet Archive

I had a great time at Books in Browsers, even though I completely lost my voice on the second day. The assembled talent and brainpower made almost every moment a thrill. When my talk from the morning of the first day was given prominent mention in the New York Times' Bits Blog, I got so excited that I couldn't pay attention to an amazing talk on annotation of medieval manuscripts.

But the most important talk of the two days was Brian O'Leary's closing presentation, which prompted the Twitter backchannel to unanimously elect him the "Secretary-General of the United Nations of Publishing".

Here's his abstract:

Although business models have changed, publishers and their intermediaries continue to try to evolve their market roles in ways that typically follow the rules for “two-party, one-issue” negotiations.  In an environment in which the negotiations are better framed using models for “many parties, many issues”, these more limited approaches have made the design of a flawed ecosystem even worse, shifting burdens onto valued intermediaries (libraries and booksellers, among others).

Content abundance, coupled with improvements in available technologies, gives us an opportunity to reshape the competitive framework.  This talk will examine options to apply the principles of effective game design to create a set of new, targeted and evolving business models for content dissemination in an era of abundance.

O'Leary talked about the changes occurring in the entire ecosystem of what used to be called "publishing": authors, agents, publishers, distributors, retailers, libraries, and of course readers. He noted that relationships throughout the ecosystem were being renegotiated without an awareness of the effects of these changes on the rest of the ecosystem. As a result, frameworks, arrangements and processes that could benefit the entire ecosystem were not being given the consideration they deserve.

The future of EPUB

O'Leary pointed to the discussions leading to the United Nations Convention on the Law of the Sea as a possible inspiration for a reading-ecosystem way forward. The breakthrough in those discussions was the introduction game-theory models that helped the parties see the effects of agreements and provisions on all stakeholders in the Law of the Seas negotiations. If a similar sort of model could be developed for the activities surrounding publishing, it might be possible to do a lot more that to "save publishing". Intelligent, collaborative application of digital technologies should be able to increase the effectiveness of an industry whose purpose is to promote reading, education, culture and knowledge.

According to O'Leary, we need to figure out ways to fund the sort of research that could be the basis of modeling for the reading ecosystem. One possibility would be to create a cross-industry organization to do so.

If such an organization were created, I hope that its membership mirrors the composition of Books in Browsers attendees. Many inhabitants of the reading ecosystem were represented, despite the technology emphasis of the meeting- publishers, librarians, agents, academics, authors, designers. The contrast with last week's DPLA Launch meeting was striking- hardly any publishers or authors were in evidence at DPLA. It seems to me that with everything that's at stake, we could do a lot worse than to listen some more to Brian O'Leary.

Update (10/31/11): The text of O'Leary's talk is posted here.)

In Defense of the Book as a Container

Podcasting has turned radio into a new medium. From the New York Times Magazine:

...the value of a media product does not come from being fast. It comes from being timeless. ...It wouldn't make sense, [Abumrad] said, to devote the effort to seduce, disturb and engage the listener if "Radiolab" epidsodes were merely broadcast once and disappeared.

Exactly, I thought, Radiolab is like a book.

Back in October, Brian O'Leary posted a rather long essay on his blog, called "Context First". The essay was well received, and he presented versions of the essay at conferences around the world. I excerpt:

my idea in a nutshell is this: book, magazine and newspaper publishing is unduly governed by the physical containers we have used for centuries to transmit information.  Those containers define content in two dimensions, necessarily ignoring that which cannot or does not fit.

Worse, the process of filling the container strips out context – the critical admixture of tagged content, research, footnoted links, sources, audio and video background, even good old title-level metadata – that is a luxury in the physical world, but a critical asset in digital ones.  In our evolving, networked world – the world of “books in browsers” – we are no longer selling content, or at least not content alone.  We compete on context.

I propose today that the current workflow hierarchy – container first, limiting content and context – is already outdated.  To compete digitally, we must start with context and preserve its connection to content.

Something about O'Leary's essay has been gnawing at me ever since, but until reading yesterday's article on Radiolab, I couldn't put my finger on what it was that bothered me.

The implication of O'Leary's "Unified Field Theory of Publishing", is that in order to compete, print publishing needs to break away from the limiting forms of the print medium and become something new, in jazzy harmony with our contextually dynamic digital world, filled with links and metadata and APIs.

Bear in mind, O'Leary's [a consultant who's worked in both the magazine and book industries, is] dead right about magazines. But I've decided his implication is dead wrong about books. We need to understand what it is about the book that makes it a container of media that will persist into the digital world. It's NOT context. The wonderful thing about the book as container is the same thing that lifts Radiolab as podcast above Radiolab as radio. It's the timelessness.

So as we evolve the ebook, I think we need to be aware of and nurture its potential for timelessness. If we put the context first, as O'Leary urges, then all we have left is a website.

Long live the content container formerly known as the book!


Notes:

1. As we develop plans for the Gluejar business, we need to add some definition to the "containers" that get released as creative commons ebooks. One thing we might do to enhance "timelessness" is to add a digital signature that verifies that the content of the ebook hasn't been altered.
2. [Update] This conversation has continued over on Brian O'Leary's blog. There's also a remarkable reflection, in French, at SoBookOnline, that does a great job of laying out the ideas and questions that Brian and I are wrestling over.

Consumer Demand for Pirated eBooks Stopped Growing in 2010

Online piracy of ebooks has been a persistent worry for book publishers who look at the successes and failures of other media that have moved to digital forms. A surprising number and variety of ebooks are easily availabile on file sharing websites and peer-to-peer networks that use bitTorrent and similar protocols. The possibility that this availability will cut into sales of licensed ebooks and even print books is a scary one for an industry that has had many decades of relative stability. At Digital Book World in January, Brian Napack, President of Macmillan, "delivered a passionate call to arms for publishers to fight piracy in the ebook space or risk permanent damage to the underpinnings of publishing as a commercial enterprise".

Adding to the ebook piracy hysteria have been studies of the prevalence of ebook piracy produced by Attributor, a company that sells anti-piracy services. I've previously written critically about Attributor's report that purported to find evidence that "Online Book Piracy Costs U.S. Publishers Nearly $3 Billion".

In their most recent report, Attributor has taken a rather clever approach to the measurement of ebook piracy. Instead of trying to track downloads, Attributor has begun to use Google Trends to gain an understanding of consumer demand for ebooks. Although there are many potential difficulties in using Google for this purpose, Google Trends is a powerful and useful tool for gaining insight into the things that web users around the world are looking for.

Attributor presents their data along with an alarming narrative of growing and pervasive ebook piracy, and points to the iPad as a contributing factor to an increase in demand for pirated ebooks. After playing around with Google Trends for a while, I've come to the conclusion that Attributor has narrowly selected data to fit their narrative; taken as a whole, Google Trends data broadly supports a rather different narrative: that the growth of consumer interest in pirated ebooks slowed significantly in 2009 and stopped in early 2010.

To understand how Google Trends informs the debate about the prevalence of ebook piracy, it helps to understand what activity is being measured. Google Trends measures the frequency that search terms are used. A consumer looking for a free copy of a particular work will typically search on the book title, adding  terms like "free" or "download" or "pdf" to locate downloadable files. A more sophisticated strategy, one that is quickly learned, is to add the name of a preferred download site. If the user prefers peer-to-peer networks, the word "torrent" can be added to locate "seed" files for the item. The file sharing sites most commonly used for this purpose are currently RapidShare, Megaupload, 4shared, and Hotfile. To use Google trends to measure the demand for a pirated ebook, you give it keywords that reproduce these searches. For example, demand for Stephanie Meyer's book Breaking Dawn can be assessed with a query such as this one.

To assess the overall state of ebook piracy, I used data from this query. Note that since the search is for ebooks generically, there's no telling for sure that the ebooks being searched for are really pirated; for the purposes of this study, I assumed that none of the ebooks being searched for are legally available on these sites. Calling them "pirated books" may be inaccurate, but I'll use that term anyway.

Some features of the data are immediately apparent. First of all, searches for pirated ebooks have increased a great deal over the past 5 years. It's worth noting however, that the most intense interest measured by Google occurs in India, the Philippines, Indonesia, Vietnam, Malaysia, Singapore, and eastern Europe. Less than half the search volume comes from the US. It's also easy to see seasonal peaks that obscure the shorter term trends. The peak periods for pirate ebook seeking are the December holidays and the beginning of September, presumably because of the start of school.

To eliminate seasonal variations, I computed the year over prior year growth of pirate ebook search activity. The resulting plot is quite smooth. After a few years of 100% per year growth, 2008 showed a clear slowing of growth. This slowing of growth continued up to the beginning of 2010, and then  flat-lined. Since February of 2010, the growth of interest in pirated ebooks has stopped completely.

It should be noted that this stabilization has occurred during a period of strong sales of ebook reader devices, including Kindle, Nook, and the iPad. Indeed, the unveiling of the iPad was coincident with the stabilization of demand for pirate ebooks.

It's hard to know for sure what's happening, but one interpretation of these patterns is that a broad increase in consumer-friendly availability of properly licensed ebooks over the last 2 years has squelched the growth of demand for ebooks from illicit sources. In that light, the remaining demand can be interpreted as a sign of poor availability for appropriately priced ebooks on college campuses and in developing countries.

While this data has to be seen as an encouraging sign for the book publishing industry, it's too soon to know if it will last. It's entirely possible that too-high prices, cumbersome DRM, or new technologies could reinvigorate the demand for illicitly shared ebook files. For the moment at least, the book publishing industry can exhale.

Aggregating Deep Discount Readers of eBooks

The book publishing industry should be terrified of readers like me. Over the last year, I have purchased a grand total of one new book. Why only one? I have a huge stack of books, both print and digital, in my aspirational reading queue. I read plenty of books, but there are many more books that I would like to read; so many in fact that I see no reason to spend $30 on a book when there are plenty of minimal-outlay books already waiting to fill my hours.

The books on this stack come from a variety of sources. I read ebooks from my wife's Kindle account. The print books are almost all purchased at used book sales, typically for $1 or $2 each. The publisher's revenue from all of this book enjoyment is $0, and of course the author gets only a small fraction of that in royalties.

No matter how terrifying the idea is, discount readers like me represent a big opportunity for book publishers as they move there properties onto digital platforms. Discount readers come in many forms; it's safe to say that the billions of books lent by libraries went to people unwilling to pay full retail price for books. Libraries contribute modestly to the income streams of publishers and authors; used book sellers not at all. In their print businesses, publishers have learned to segment their markets by offering paperback versions as well as remaindered books, but they have largely neglected the deep discount end of the demand curve.

There's a huge amount of value to society in deep discount demand, and it not just in the benefit to readers like me. Libraries include the preservation of our written culture in their mission; this activity wouldn't happen if the first-sale and fair use doctrines didn't limit the control that publishers and authors could exert over the use of their works. We need to think about how to do preservation as we translate the book business into a digital industry.

I've been thinking a lot about business models for ebook publishing, and a lot of my thinking has surrounded market segmentation methods. I've been looking for ways ways that discount readers like me can be aggregated into sustainable revenue streams to sustain institutions such as libraries.

One obvious model to serve the discount reader is to offer subscription packages. I've come to the conclusion that ebook subscription packages have many structural problems. Subscription packages inevitably cannibalize sales of the items they contain, and there's a lot of incentive for the package to exclude items that readers would really want.

In the course of studying academic publishing models, I think I've found a  way for the book business to serve deep discount readers, to reinvigorate libraries, and to create a new, sustainable revenue stream for publishing: public acquisitions of ebook rights.

Here's how it might work for me. There are lots of books I aspire to read, many more than I have time for. There are also lots of books that I'd like to have on my reading devices, because I've read them once in print. I want to use these books in many ways, on many devices, at any time in the future. I want to be able to search them, and have others read them. I don't want to have to mess with DRM. And I want them to be preserved and available forever in public libraries.  I also like the one new book I've purchased (Clay Shirky's Cognitive Surplus: Creativity and Generosity in a Connected Age) enough that I would also pay something towards letting you read it too! Imagine that I could offer $1 for each title on my list to have this magic occur. (Cognitive Surplus is a thought-provoking meditation on the things that happen when the barriers to collective action are lowered, among other things. You should read it!)

OK, here's a stretch, imagine that millions of other people feel the same way!

If millions of people feel this way, there's absolutely no reason this magic can't happen. I have advocated that libraries should work together to collectively acquire ebook assets. The same mechanisms that would allow libraries to act collectively could be used by individuals to act collectively on behalf of books that they care about.

 If a hundred thousand people offered a dollar to Clay Shirky (and Penguin, his publisher) for Cognitive Surplus to be released as a creative commons licensed ebook, certainly at some point they would examine their prospects for future sales and figure out how to say "yes". Once a book is liberated in this way, all the magic just happens.

 I'm not expecting J.K. Rowling to cash in her Harry Potter rights anytime soon, but I think there are many types of works and many types of authors who would find it financially advantageous to monetize their work in this way if it became popular. I also think that it's very common for readers to be passionate about the books they read in ways that transcend their narrow financial self interest.

 If you agree with me that mechanisms for public ebook acquisition by readers should be developed, I would very much like to to hear from you, either privately or in the comments!