How to Fund a Public eBook Library with Tax Deductions

In California, newly (re)elected governor Jerry Brown has proposed to zero out state funding for libraries. In Britain, government cuts could lead to as much as a quarter of librarians lose their jobs over the next year. In New York, the Queens library has responded to budget cuts by zeroing out its acquisition budget, instead of cutting opening hours. In my home town of Montclair, New Jersey, a historic branch building is being closed and the reference department is being eliminated.

Given the worldwide climate of cutting libraries, the recent proposal for the establishment of a "Digital Public Library of America" (DPLA) would seem to be either the product of a castles-in-the-sky delusions, or the result of watching too many Star Trek reruns. But it's worth thinking through the DPLA for a few moments before dismissing it as unrealistic.

Despite the complexities, the fundamental idea of a National Digital Library (or NDL) is, at its core, straightforward. The NDL would make the cultural patrimony of this country freely available to all of its citizens. It would be the digital equivalent of the Library of Congress, but instead of being confined to Capitol Hill, it would exist everywhere, bringing millions of books and other digitized material within clicking distance of public libraries, high schools, junior colleges, universities, retirement communities, and any person with access to the Internet.

I don't quite understand whether or how a National Digital Library could be restricted to its citizens; it would effectively be a world library, but that's a quibble, or perhaps a warning about territorially restricted rights, but there you get the idea. It should be an international effort. The Sloan funded planning initiative should go a long way towards fleshing out the idea.

The most difficult part of building such a library will be finding stuff to put on its virtual shelves. There's public-domain material to start. Cooperative scanning and digitization of such material would create an important resource for scholars, and much of this work has already been done. Almost two million volumes worth of digitized scans of public domain works  are already available (with varying availability) through the HathiTrust cooperative. Some of these scans were made as part of the Google Books project.

Another source of books for DPLA would be material that is out of print but not out of copyright. The Internet Archive suggests that many of these works can be made available to the public as if they were physical books, as long as a physical book is held as in reserve as the basis for doing so. It not clear whether doing so would be consistent with copyright law, except perhaps for books without economic value. Manyrightsholders might be willing to donate their rights to DPLA, just as many rightsholders appeared to be willing to allow use by Google for modest compensation.

Unfortunately, many rightsholders are absent and can't be found. Many rightsholders lack good documentation of rights or may be uncertain whether rights have been properly reverted. The Google Books Settlement Agreement attempted to settle or circumvent litigation surrounding these issues; the agreement has thus far not been acted on by the court. It's quite possible that legislative action would be required to allow a digital public library to freely provide this sort of content. Good luck with that.

What's missing from the proposed DPLA is any way to stock it with content that people actually want to read. Publishers that I've talked to are loathe to give up their backlists because they believe the backlists may become valuable through digitization and enhancement. Often the rights surrounding their backlists are so murky that it would be a significant expenditure just to clear rights. The only backlist acquisitions an NDLA would be likely to afford are those that are truly worthless.

There IS, however, a way, in the existing tax code, to direct tax money toward acquiring eBooks for a DPLA, or for any library, for that matter. It's called the deduction for charitable donations. As long as doing so is consistent with their charitable purpose, any recognized charity could collect donations toward the acquisition of rights to an ebook. If the ebook was then made available under a public license, such as one of the creative commons licenses, it would further that public charitable purpose.

For example, a charity focused on finding a cure for a disease (e.g. Huntington's) could ask supporters, acting together, to buy out the digital rights to books and journal articles relating to research, treatment and patient care for that disease. Similarly, a museum devoted to the history of New York City tenements could ask supporters to acquire works of fiction that bring to life the period and lifestyle of immigrant living in lower Manhattan in the early 20th century. Libraries could join together to support all sorts of books that promote their public purposes of promoting literacy, economic development, and cultural enrichment.

Donations made in this way would be fully tax deductible, at least in the US. I don't know the situation in other countries, but I note that many European countries have unfavorable treatment for eBooks under their value added tax (VAT) systems. I learned at Digital Book World that Italy has a 4% VAT for print books, but a 20% VAT for eBooks, which are considered to be a service. The VAT for charitable donations is, of course, 0%.

Readers of this blog will recognize that this is another way to present "ungluing ebooks". I've been working on ways for libraries and consumers to join together in a market to acquire the right to put ebooks into the public commons. This could happen through a collective of libraries or through a web site like Kickstarter, or both. Once acquired for the public commons, the ebooks could be made available through DPLA or any other library or ebook distribution service.

Apart from the tax advantages, the benefit of working with charitable institutions to crowd-fund the acquisition and liberation of ebooks is that early validation of the concept will be easier. No one really knows whether people will put real money towards the movement of ebooks into the public commons. Working on a project catalog much smaller than "all the world's books" with a target audience more focused than "people who love books" will allow us to more accurately gauge the willingness of the public to support public ebooks.

I've spent a lot of time this month explaining my ideas for ungluing ebooks. The questionthat keeps recurring is whether I plan to act on these ideas by forming a non-profit. The most persuasive argument for this is that the idea of acquiring the rights to an ebook for the public commons is much easier to understand when put in the context of a public charity, such as public television or public radio. The whole idea of free books is so foreign to people that people assume that free ebooks means pirated ebooks.

For now, though, I'll be pursuing this via my for-profit company, Gluejar, Inc. That's because I talked to my lawyer. He pointed out that it's much easier to deduct the costs of a speculative business venture from my taxes than it would be to convince the IRS that this weirdo crowd-funded public ebook idea is really a charitable endeavor.

I'd like to acknowledge Juliet Sutherland, Director of Distributed Proofreaders, and Richard Cave, I.T. Director of the Public Library of Science for very helpful contributions to the ideas in this post. I'm not an accountant or lawyer; this article is not any sort of tax advice.

Internet Archive Sets Fair-Use Bait With Open Library Lending

Here's the most important thing I've learned about intellectual property law: the lawyers who say "yes" when you ask if you can do something are much, much more expensive than the lawyers who say "no".

Brewster Kahle, the founder, and through a foundation, the funder, of the Internet Archive, can afford a very expensive lawyer. He sold Alexa to Amazon for about $250 million of Amazon stock. He'll need that expensive lawyer; on Tuesday, the Internet Archive announced that its Open Library had started to facilitate the lending of out-of-print (but in-copyright) digitized books, a move that seems designed to spur a legal reaction from publishers.

To some extent, this isn't really news. Kahle has been advocating digital lending of books for some years now. In 2001, he published an article in D-Lib Magazine advocating the use of Inter-Library Loan (ILL) for digital materials. In October, the Internet Archive unveiled its Bookserver software, whose goal was to enable the lending of digital materials over the internet, and the University of Toronto was one of the original partners in its development.

The lending library announcement was also modest in scope. The big numbers were associated with programs unlikely to stir any controversy. Over a million out-of-copyright works are available, and Open Library has integrated access to the 70,000 ebooks licensed to subscriber libraries through Overdrive. Only 187 of the available books fall into the category of un-licensed out-of-print but in-copyright books.

In another sense though, the announcement, which was fed directly to the Wall Street Journal, was a declaration of war on barriers to fair use of digitized books. From the Journal's article:

The effort could face legal challenges from authors or publishers. Paul Aiken, the executive director of the Authors Guild [...] said "it is not clear what the legal basis of distributing these authors' work would be." He added: "I am not clear why it should be any different because a book is out of print. The authors' copyright doesn't diminish when a work is out of print."

Mr. Kahle said, "We're just trying to do what libraries have always done."

Having to receive prior permission from a copyright owner in order to scan a book is onerous, said Mr. Blake, of the Boston Library. "If you own a physical copy of something, you should be able to loan it out. We don't think we're going to be disturbing the market value of these items."

Stewart Brand, author of the 1988 book "The Media Lab"—one of the scanned books that will now be available for loan—said he didn't mind seeing his title made available this way. Mr. Kahle at the Internet Archive asked his permission, he said, and he gave it because he thinks digitizing books has the potential to improve knowledge.

The fact that at least one author was asked for permission suggests that the Archive is being very careful about what it chooses to make available through the lending program. A look at the 187 items in the lending library supports this view. There are

• Works by well-known copyright reform advocates such as Brand and Lawrence Lessig.
• Obsolete computer books. Example: Microsoft Windows 98 at a glance
• Older books likely to be orphan works: example- Alice James: her brothers~her journal, the posthumously published diary of Alice James.
• An in-copyright version of an out-of-copyright work: Brace Lineage, a genealogical work apparently based on an older version.
• Other genealogic works which might be considered to be data compilations. Example: Twelve generations in America
• Spanish language works published in Guatemala. Example: Regimenes agrarios
• A collection of stories by anonymous pregnant teens: You look too young to be a mom: teen moms speak out on love, learning, and success

In short, if you wanted to take legal action to stop the digital lending library, each of the books included in the lending library would pose some sort of problem for you.

It does not appear that the Internet Archive is attempting to rely on the statutory exceptions for libraries in US copyright law. These exceptions have rather technical requirements, and the lending library program does not appear to have been crafted to take advantage of these exceptions. Rather, it appears to be staking out fair use grounds. As James Grimmelmann writes:

The argument here would likely center on the Archive’s nonprofit purpose, the negligible harm to the market for some long-out-of-print books (quite possibly including some orphan works), and the nearby public policies of first sale and library exceptions. The natural counter-argument, however, is that distributing complete copies of books for readers to consume is so close to the core of copyright’s rights and goals that fair use simply cannot stretch that fair. These are non-transformative, substitutive, complete copies of expressive works—so while the Archive would have an argument, the fair use factors arguably tip 4-0 against it. Should it win, it would be a revolution in fair use caselaw. A good revolution, for some, but a revolution nonetheless.

Consider the case of Digital Systems by Ronald Tocci, published by Prentice Hall in 1977. The lending library offers only the first edition; a tenth edition was published in 2006. It would be hard for Prentice Hall to argue that offering a single copy of the 1977 version would reduce its economic value, while it would be easy to argue that sales of the 2006 edition will be enhanced. In addition to the probable difficulty in proving damages, it's likely that Prentice Hall might have difficulty proving that it has electronic publication rights for the 1977 edition, as author contracts of that era could not have anticipated today's internet distribution channels.

The Internet Archive's legal strategy would appear to be one of fair use creep, a sort of adverse possession by the public domain. If no one steps forward to tell the Internet Archive to stop lending these works, then the public gains a sort of right-of-way to use them. Even if a lawsuit occurs, it's quite possible that a jury would consider single-copy lending use of any of the 187 to be fair, even if the majority of copyright lawyers might disagree. If enough works become available in this way, then a political constituency for library lending of ebooks could develop and strengthen.


It looks to me as though the Archive is setting a trap, hoping that someone will take the bait and file a lawsuit despite problematic subject matter. The publicity exemplified by the Wall Street Journal article then looks like chum on the water to trick the legal sharks of publishing into striking on poisoned bait.

And here I was about to go swimming.

Business Idea Number 3: Gluejar Book Search

A few years ago, I was invited to give a talk about the future of libraries at a library staff retreat. After the talk, the speakers were given a special tour of the library, which had recently undergone renovation. I was struck by the loneliness of the stacks. So many books, so much knowlege, so little usage.

As OCLC's Lorcan Dempsey has recently observed, the lawsuit over Google Book Search and its proposed settlement has highlighted the limitations on libraries' ownership of their book collections. There are many things that libraries would like to do with their books that they are prevented from doing by copyright law. The possibility that the Google Books service will enable libraries to reanimate their lonely book collections is the reason that libraries have, for the most part, been sympathetic to Google's digitization program.

One session at last week's Code4Lib conference sharpened my awareness of how libraries are struggling to acheive this reanimation on their own. There were 3 different presentations, from Stanford, NC State (3.65 MB ppt), and University of Wisconsin, Oshkosh, on "virtual bookshelves". The virtual bookshelf tries to enliven the presentation of an electronic library catalog by trying to reproduce part of the experience of browsing a physical library- sometimes the book you really need is sitting there next to the book you're looking for. It's an idea based on a sound user-interface design principle: try to present information in ways that that look familiar to the user.

The virtual bookshelf is not a new idea. Google has even been awarded a patent on virtual bookshelves- see the commentary here and here. Given that Naomi Dushay (who presented the Stanford work) wrote about Virtual Bookshelves in 2004, it appears to unlikely that the Google patent (filed in 2006) will apply broadly at all.

While the virtual bookshelf is a sensible and practical incremental improvement on the library catalog interface, it's also backward looking. People looking for information today want to search inside the books, not just "browse the stacks". But libraries don't have the ability (today) to search inside the books that they think they own.

Google Books could enable libraries to do just that. Google is spending huge sums of money to digitize books in libraries and make them searchable. When they got sued for doing this, the library community looked forward to having questions surrounding the fair use of digitized books settled in court. For example, while it's pretty clear that using digitization to create an full-text index of a book would be allowed as fair use, the display of "snippets" (as done by Google) may or may not be held to be a fair use of the page scans. When a settlement of the lawsuit was announced, much of the library community was disappointed that these fair-use questions would not be settled.

Google Books already allows users to set up book collections of their own and search them. The results come with snippets (see pictures), but if the settlement is approved, Google's ability to show snippets with vastly reduced infringement liability would leave it with a dominant position in libraries because of its ability to search inside huge numbers of books. If the settlement is not approved, Google's dominance would be similar, except that a copyright decision could shut down Google Books at some time in the distant and irrelevant future.

Some aspects of the settlement create holes in Google's index. As part of the settlement, rights holders can exclude their works from Google's index. Google's publisher partner program allows publishers to create these holes today. For example, even if you add Tolkein's "the Two Towers" in your Google library, Google won't let you search inside it. Only limited research uses can be made of the digitized works; as the Open Book Alliance's Peter Brantley has argued, it's very hard to tell what sort of innovations might arise from the availability of large numbers of digitized texts as data; the same goes for indices of these works.

Many other works have been excluded from the settlement. Works published only outside the US, Canada, UK and Australia, as well as works published in the US, but not registered with the copyright office, are not covered by the settlement. Works other than books, such as newspapers, magazines, and other periodicals are also excluded.

For these reasons and others, I've begun talking to people about "Gluejar Book Search". Gluejar Book Search would be a business focused on collecting, aggregating and redistributing full-text indices of copyrighted material. To comply with copyright law, it would focus on indices that can be distributed without infringinging copyright, and would help provide libraries and publishers with tools  to produce copyright-safe index documents.

I've frequently encountered the assertion that digitizing all the books in libraries is prohibitively expensive, and that only Google (or possibly the government) could possibly have the financial resources to do it. For example, Ivy Anderson reports an estimate by the California Digital Library that digitization of the 15 million books in the libraries University of California would take a half a billion dollars and one and a half centuries. There are two coutervailing arguments. First, the cost of book digitization software and equipment has rapidly fallen, and will continue to fall. Last year, I wrote about the Dan Reetz' DIY book scanner, but even commercial devices capable of both image aquisition and OCR are currently available for as little as $1,400. I described how it could cost as little as $10,000,000 to put scanners in 10,000 libraries to enable scanning of 5,000,000 books per year.

The other factor that could drastically lower the cost of producing digital full-text indices of all types of copyrighted materials is the drastically lower technical demands of an indexing system compared to that of an archival imager. Archival imagers produce huge scanned image files because of the need for high resolution in an archival image. The resulting demands on storage hardware are significant and expensive. In contrast, an index file can be quite small; the laptop I'm typing on could store indices for 3,000,000 books; I estimate that full-text indices of all the worlds books would today require at most ten commercially available hard drives.

Gluejar Book Search would be fueled by two main revenue streams. The first stream would come from customized search services to enable library patrons to search inside the library's books. The second stream would be to provide aggregated feeds of index files to mass-market and specialized search providers- Google's competitors, and book retailers such as Amazon and its competitors. Google may even want to acquire index files for works it has been asked to remove from its own index, such as the Tolkein book mentioned above.

A possible third revenue stream would come from partnerships with rightsholders willing to permit page or snippet display in exchange for link traffic. If a Book Rights Registry comes into existence, it's possible that many business models could be arranged without prohibitive transaction costs.

Part of the revenue from Gluejar Book Search could be returned to libraries, publishers and other institutions that have contributed index files to the aggregation. Libraries could choose to use these funds to fund further digitization; alternatively, they may prefer to contribute to an Open-Access index.

The success of Gluejar Book Search would depend to a significant extent on its ability to reach critical mass. If it could reach index 80% of a library's book collection, it would deliver significant value to the library. (That statement is based purely on conjecture- email me or leave a comment if you agree or disagree!) Critical mass might be rapidly attained by working closely with publishers and by partnering with low-cost digitization providers and existing content aggregators so obtain indices for the most widely held books. Once critical mass is obtained, the "long tail" could be addressed by encouraging the particpation of large numbers of libraries around the world.

A Gluejar Book Search business would require a significant but not huge raise of capital, if for no other reason than to address litigation risk. Although I believe the legal position of building copyright-safe book indices is secure, there are bound to be litigious rightsholders with a poor grasp of fair use under copyright. The other big risks involve Google. Google might well develop services that greatly undercut Gluejar Book Search's revenue streams. Finally, the "copyright-safe" approach might be completelyundermined if courts in many countries were to rule decisively for an expansive view of fair-use.

If you want to know more about Gluejar, read this post. I have been exploring many possibilities about "what to do next", and I've written about other ideas, as well. As always, I'm interested in feedback of all kinds. Over the next few months, I hope to develop this and other ideas in more depth, so stay tuned.

Copyright-Safe Full-Text Indexing of Books

As the February 18 hearing on the revised Google Books Settlement Agreement draws near, I think its timely to explore some issues surrounding full-text indexing of books. It's important to realize that when Google began its program of scanning books in libraries, it chose to do so in a way that entered the gray zone of fair use. Google continues to maintain that its scanning activities are perfectly legal, and fair use advocates welcomed the Publishers' and Authors' lawsuit because it had the potential to clarify ambiguities around fair use. No matter where the court decided to draw the line, the both fair use and rightsholder control would be able to extend into the zone of current uncertainty.

Overlooked in the controversy is the fact that Google could have chosen a safer course in its effort to make full-text indices of books. In this article, I'll argue that it's possible to make full-text indices of books in a way that steers well clear of copyright infringement. But first, I should note that playing it safe would not have been a good plan for Google. By pushing fair use to its limits, Google assured itself a favorable competitive position. In a lawsuit, Google could have lost on 90% of the fair use they were claiming and would still have ended up 10% ahead of where a safe course would have taken them. Google is large enough that even a 10% victory in court would have paid off in the long run. As it is, Google chose to settle the lawsuit under terms that put them in a better position than they would have occupied by playing it safe, and potential competitors don't gain the benefits of a fair-use precedent.

I make two assumptions about copyright in devising an copyright-safe indexing method:

1. You can't infringe the copyright to a work if you don't copy the work.
2. If you can't reconstruct a work from its index, then distributing copies of the index doesn't infringe on the work's copyright.

Just in case these assumptions are weak, my fall-back position is that indexing is clearly a fair use under US copyright law.

First, the fall-back assumption: full-text indexing is allowed as fair use under US copyright law. Indices are allowed as "transformative uses". Judge Robert Patterson's decision (pdf, 195K) in the "Harry Potter Lexicon" case gives an excellent background of this jurisprudence and concludes:

The purpose of the Lexicon’s use of the Harry Potter series is transformative. Presumably, Rowling created the Harry Potter series for the expressive purpose of telling an entertaining and thought provoking story centered on the character Harry Potter and set in a magical world. The Lexicon, on the other hand, uses material from the series for the practical purpose of making information about the intricate world of Harry Potter readily accessible to readers in a reference guide. To fulfill this function, the Lexicon identifies more than 2,400 elements from the Harry Potter world, extracts and synthesizes fictional facts related to each element from all seven novels, and presents that information in a format that allows readers to access it quickly as they make their way through the series. Because it serves these reference purposes, rather than the entertainment or aesthetic purposes of the original works, the Lexicon’s use is transformative and does not supplant the objects of the Harry Potter works.

The author of the Lexicon lost his case not because his indexing was not allowed, but rather because he copied too much of J. K. Rowling's creative expression in doing so.

Second, you have to copy to infringe copyright. A more accurate statement is this: You have to either make a copy or a derivative work to infringe copyright. The second piece of this can be a bit more confusing, because "derivative work" has a specific meaning in copyright law. A translation into another language is an example of a derivative work. Indices are not derivative works. The law considers indices to be more akin to metadata. I might need access to a book to count the number of figures it contains, but a report of the number of figures in a book and what page they're on is in no way a derivative work. The copyright act defines a derivative work as

a work based upon one or more preexisting works, such as a translation, musical arrangement, dramatization, fictionalization, motion picture version, sound recording, art reproduction, abridgment, condensation, or any other form in which a work may be recast, transformed, or adapted.

If you make copies by scanning, however, as Google is doing, you must also establish that your use is allowed as fair use. If you don't, then you don't even need to reach the fair use provision.

The last assumption gets more technical. The simplest form of a word index is a sorted list of words with pointers to the occurrence of the word within the text. So an index of that last sentence might look like this:

a    5,9
form    3
index    7
is    8
list    11
occurrence    18
of    4,12,19
pointers    15
simplest    2
sorted    10
text    24
the    1,17,20,23
to    16
with    14
within    22
word    6,21
words    13

It doesn't take a computer science degree to see that it's easy to reconstruct the sentence from this index. For that reason this form of index is equivalent to a copy. If you remove the position pointers, however, the index loses enough information that the sentence cannot be reconstructed. So if we take the words on a page of text and sort the words in each sentence, then sort the word-sorted sentences, we get an index of a page that can't be used to reconstruct text, but can be used to build a useful full-text index of a book.

The trickiest step of completely copyright-safe indexing is producing the page index from a book without producing intermediate copies of the pages. In a conventional scanning process, a digital image of a page is stored to disk and the copy is passed to OCR software. Indexing software then works on the OCR text. A scanning process that was fastidious about copyright, however, could scan lines of text word by word and never acquire an image large enough to be subject to copyright.

US courts have considered the loading of a copyrightable work into a computer's RAM storage to constitute copying, but scanning sufficient to produce an index can in principle be done without requiring that to occur. (For an excellent law review article on the RAM-copying situation, read Jonathan Band and Jeny Marcinko's article in Stanford Technology Law Review.) Also, even sentences of more than a few words can be considered copyrightable works, as I discussed in an article from November.

Another possible way to avoid copying is to build a black-box indexer. A closer look at the RAM-copying precedent, MAI SYSTEMS v. PEAK COMPUTER suggests that a non-copying scanning indexer can be built even if page images exist somewhere in RAM. In that case, the court reasoned that the software copy could be viewed via terminal readouts, system logs, and that sort of thing. If a closed-box indexing system were built so that page images resident in RAM could never be "perceived, reproduced, or otherwise communicated", then there is a fair chance that a court would find that copying was not occurring.

I'm a technologist, not a lawyer. I would welcome comment and criticism from experts of all stripes on this analysis. For example, I've not considered international aspects at all. There are many technical aspects of copyright-safe indexing that would need to be sorted out, but doing so could open the way to countless transformative uses of all the books in the world.

2020: Fewer Libraries, More Locations

In the middle of the block I live on, there are two fire hydrants right next to each other. The reason for this is that half the block is in the town of Glen Ridge, New Jersey, and half the block is in Montclair, New Jersey. In the past there were incidents when fire trucks from the two towns rushed to the scene of a fire alarm, only to get into lengthy discussions about which town had responsibiity. That doesn't happen any more. In 1991, Glen Ridge closed its fire department and contracted with Montclair to supply fire protection services.

Will the same sort of consolidation happen with libraries? I think it will.

In my "Ten Predictions for the Next Ten Years" article, my first prediction was that the number of public libraries in 2020 would be half of what it is today. I also predicted that the number of public library locations would increase by 50%. I got plenty of feedback on Twitter that these predictions needed some explanation. Roy Kenagy thought that my prediction couldn't possibly apply to Iowa, where "new [libraries] sprout like weeds and people tend to them as their own".

There were two considerations, book digitization and the shift to e-books, that led me to these predictions, and neither is peculiar to New Jersey. I admit, though, that New Jersey's high taxes and density of services affected my estimate of the magnitude of coming changes.

Over the next ten years, book digitization will completely change the way most people use libraries. Instead of browsing the stacks or searching a catalog, people will increasingly make use of full text indexes and digitized resources to find books. This already happens with Google Books. They will then try to obtain the physical book in the library, or alternatively, use an e-book reader. Public libraries will need to adapt their physical plants to accommodate this changed usage pattern. Stacks will become more warehouse-like; public spaces will have fewer books and more coffee. Patrons will demand larger collections, but will accept less physical access to print. Home delivery of library materials will become much more common.

At the same time, libraries will struggle to adapt to the e-book economy. The most likely outcome will be a shift to licensed resources. Publishers will discover the benefits of putting much larger numbers of titles into e-book subscription packages such as those currently offered by Overdrive, Netlibrary, Ebrary, and others. When these packages can be used on patrons' Kindles and other e-readers, libraries will need to have them.

All of these trends will put pressure on libraries to work together on shared services, and ultimately to merge. Larger libraries will more effective at delivering both print books and e-books, and patrons will care less about where the print books are stored when they're not being lent. Smaller libraries will find it difficult to support the technical and operational expertise needed to run the public library of 2020.

While the shift to digital media will cause library organizations to become larger and fewer through mergers, it will also allow branches to be effective at smaller sizes. Without the need to store a critical mass of books, tiny, storefront branches will become more practical and cost efficient. Guys in vans carrying books will become more important. When people go to their local branch, they'll be able to use the free Google Books terminal (libraries are to get one free for every building) or other computers, check out some books, then have a coffee and socialize for an hour or so until the van makes its hourly delivery. Or they'll do their shopping rounds and come back to pick up the bag of books waiting for them. Establishing branches in shopping areas is not only a smart thing for libraries to do, it's also very cost-efficient.

In my own town, it seems that almost every year there's talk of closing the branch to save money. If you look at it, you can see why- the building is massive and has to be very expensive to operate. Eventually it will be shuttered and sold, but a storefront branch down the block could deliver the same services and cost much less to run. Does it make sense for the town high school to run its own library? Not really, but that could be another branch. We'll have fewer libraries, but more locations.

While consolidation and mergers will reduce the number of libraries, it can't be ignored that public library budgets are being slashed, and some libraries are being closed for purely financial reasons. Part of this is that the perceived value of libraries is less than it used to be. Many critical information services that used to be available only through libraries are now readily available through the internet.

There's also the possibility that public library services could be outsourced. My town's library gets annual funding of $3.8 million, roughly $100/resident, paid through our property taxes. If, in 2020, people are mostly reading books on e-readers, how much will they be willing to spend on a library? Will people prefer to spend the $100 on a commercial e-book subscription? I'm not sure, but I'm guessing that some towns will go the outsourcing route.

As I've said before, I'm an optimist about the ability of libraries to adapt to changes in media. If you look carefully at my picture of the Montclair Public Library van guy, you'll notice that what he's collected from this drop box is a newspaper, some VCR tapes and a whole bunch of DVD's. The print books were in another box, and there was not a single e-book to be carried to the main library. Makes you wonder...

Update 1/8: Some follow-up here.

Copyright Enforcement for eBooks: Cultural Life Preserver or Orwellian Nightmare?

I'm a 7 MPH speeder. When I'm on an empty highway with a 65 MPH speed limit, I drive 72. This puts my car in roughly the 70th percentile of car speed. But when some idiot comes zooming past at 85, I cheer when I see him stopped by the cops 5 minutes later.

Last time I was in England, I was appalled to find that cameras had been installed along some of the motorways that would send you a speeding ticket automatically if you averaged more than the speed limit. I told the limo driver that Americans would elect a black president long before we'd tolerate speed cameras on the freeway. I was right.

I'm no legal theorist, but I know better than to think that human behavior is determined by laws- laws only work as far as they reflect a social consensus. It's true for driving and it's also true for reading, listening to music, and watching videos. As behaviors change due to the introduction of technology, society is forced to modify social norms for behavior.

The book publishing industry is at the beginning of a technology driven change in the way that people read books, and the shape of the consensus that emerges will determine how creative production is sustained. (Same for news, but that's another story entirely!)

Social consensus has a lot of inertia because if people and institutions don't have to change, they won't. Think about how social consensus evolved when music became digital. It used to be that people expected to be able to listen for free, via radio, and expected to pay to "keep" the music. Once paid for, people expected to be able to share their records with friends in a variety of ways.

When music became digital with the advent of the compact disk, very little changed, at least for a while. The addition of internet distribution, however, allowed Napster to stretch the "sharing" behavior so as to cover free listening and threaten the buy-to-own behavior. The music industry responded with legal action, but its failure to provide convenient, authorized activities to cover accustomed behaviors gave Napster an effective monopoly on digitally distributed music. If not for the social habit of paying-to-keep music, the music industry may well have collapsed. With the takedown of Napster and the rise of authorized services like iTunes, Pandora and Spotify, the music industry has begun to successfully reshape user behavior forged by easy unauthorized file sharing, but its mistakes have clearly hurt.

The movie industry has had more luck with the onset of digital distribution. People still expect to watch TV for free, and to pay for premium entertainment at the movies. The internet bandwidth needed to easily move video files has become available at about the same time as distribution sites such as Hulu and Netflix, so pirates have never had much of a monopoly on digital movie distribution. YouTube offers a flood of free video content, and it works with rightsholders to identify and remove unauthorized uses of their work. Large amounts of unauthorized distribution has occurred, but the movie industry has responded with both the carrot and the stick, by providing enhanced in-theater experience, inexpensive secondary distribution channels, making deals with YouTube and providing specialized DVD content while pursuing takedowns and ostentatiously prosecuting copyright infringements. Certainly the movie industry has made some missteps, but a blockbuster movie can still gross a billion dollars.

People have always expected to pay to own books, but once bought, the books could be freely borrowed from friends or libraries, and a vibrant used-books market makes older works available at very low cost. The biggest change brought about by digital distribution is the flood of free material available on a huge variety of websites, from blogs to wikis to traditional news.

It's not clear how book (including ebook) sales will be impacted by unauthorized distribution of digital copies. Although I've noted that it's relatively easy to find and identify unauthorized copies of works like Harry Potter and the Deathly Hallows, it's not likely that people will change their book buying behavior unless they have to. That's why I find it surprising that J. K. Rowling and her publishers are giving the pirates a near monopoly on the digital version of that particular book.

I've heard publishers say that they've learned from the example of the music industry that the threat of piracy makes DRM (digital rights management) a necessity for distribution of ebook content. In fact, almost the opposite is true. Publishers have been distributing books for hundreds of years without DRM. A potential pirate doesn't need to crack any encryption; they need only buy a single copy of the book and scan it. I wrote about the advent of cheap book scanners in October; Wired has a recent article.

Pirating a book is somewhat more difficult than pirating a song, but comparable to pirating a movie. The first step is to acquire a digital copy. Popular books are easy to obtain and a professional pirate would likely remove the binding with a saw and feed the pages into a high-speed copier/scanner. (Until the DVD comes out, a pirate typically sits in a theater and films the movie; the DRM on DVD's is trivial to crack.)

The digital file would then either be seeded onto a peer-to-peer network or uploaded to a file distribution or streaming site similar to rapidshare. Studies by Arbor, Cisco, and Sandvine suggest that P2P networks are declining in popularity compared to the file distribution sites, especially in countries with high broadband penetration.

In a peer to peer network such as those using the BitTorrent protocol, the work is divided between tracker sites and the peers which provide the actual files. The use of many peers allows high-volume distribution without needing a high bandwidth internet connection. Since the RIAA and others began filing lawsuits against people thought to be involved in providing files, the remaining networks have adopted social-networking and encryption to make sure that they can no longer be easily monitored.

File distribution sites are being used more and more as broadband connections become widespread. These sites have many legitimate uses, and will respond to takedown notices when illicit content is identified on their sites (although the in some countries, the takedowns are processed with the underwhelming speed of a bank's electronic funds transfer.) The links and metadata for the illicit files mostly appear on third party sites, which complicates any enforcement action. Ironically, sites such as Rapidshare have become so popular that to use it easily you really have to purchase a premium subscription!

Still, digital book piracy has already begun to appear in significant amounts. According to Brad Beautlich, Sales Director at DtecNet, text books, including law and medical textbooks, are now frequently appearing on the content distribution sites and torrent indexes favored by copyright infringers. These tend to be expensive items sold in cost-sensitive markets, which increases the incentives for unauthorized use. The sites appear to have very few books that have been cracked from digital versions; most of the book content currently available is clearly derived from scanned print.

The lack of pirated e-reader files (such as kindle or epub files) is consistent with the profile of e-reader early adopters, who tend to be to be older and not particularly price sensitive. I assume it's because older users tend to have bad eyes and full shelves. They're unwilling to install P2P client software or be attracted by the sort of advertising found on file index sites. Readers in developing countries may be in different situations.

DtecNet is a company that has been providing detection services to media companies. They offer to seek out, document and help to take down unauthorized content from web sites and file sharing networks. Their task can be difficult, as they need to scan and monitor indexing sites that may cloak the identity of a file ("NITM2" instead of Night in the Museum 2) and figure out from user comments in multiple languages whether a file is genuine or not.

Beautlich suggests that although monitoring from his company would be expensive ($4000-5000/month for a Harry Potterish project), an early investment in copyright enforcement by the book industry might more effective than a strategy of waiting for a larger threat to arrive.

Another strategy to modify user behavior is being pursued by Audible Magic. Audible Magic has a rather different business model from DtecNet. Instead of working for rightsholders, Audible Magic provides content identification services to ISPs, educational institutions, and content distribution services, helping them minimize their liability for copyright infringement. In the US, the Higher Education Opportunity Act (HEOA) of 2008 requires colleges and universities to have "A plan to 'effectively combat' copyright abuse on the campus network using 'a variety of technology-based deterrents'."

Audible Magic provides an appliance that attaches to a router or gateway within the client's network. The appliance "listens" to network traffic, and when is recognizes copyrighted content being transferred in ways that connote unauthorized use, it either logs a report or attempts an intervention. According to Jay Friedman, Audible Magic's Vice President for Marketing, over 100 University campuses are using their systems. Pricing depends on the amount of bandwidth used by the university and can be as little as a few thousand dallars a year.

Interventions are positioned in a "graduated response" model. For example, a user's next webpage download might be replaced by a page suggesting that unauthorized activity may have occurred, along with a reminder of an institution's usage policies. Continued infractions might result in the user being put in a "timeout", followed by a human mediated intervention.

If you find it big-brotherish to have an "appliance" looking over your shoulder so see whether your infringing copyrights, you wouldn't be alone. The Electronic Frontier Foundation has warned that Audible Magic's service offering is "no magic bullet", and is concerned that this type content monitoring would be a threat to individual privacy rights. It's one thing for a universities and corporations to be proactive in avoiding copyright infringement liability, but imagine what it would be like if this sort of monitoring were a legal requirement! Public Knowledge has published an excellent overview of the issues surrounding this sort of network monitoring.

In fact, international treaties and legislation requiring ISPs to adopt "three strikes" graduated response policies culminating in loss of internet connection is being considered in Europe and other parts of the world. While many book publishers would be horrified to buy into these sorts of copyright enforcement regimes, at the same time they are aghast at the prospect of having their content pirated and their livelihoods destroyed.

Think about the speed limit monitor in the accompanying photo. Based on my observation it is very effective at modifying the behavior of drivers. 7-MPH speeders like myself become 1-MPH speeders. I don't think anyone minds being monitored by this sign- there is confidence that it's not doing anything other than measuring and displaying our speed. In contrast, hidden speed traps seem evil- they don't slow people down unless they own radar detectors; the egregious speeders are not the ones who get caught! Copyright enforcement for ebooks should be as much like that as possible. As Princeton's Ed Felten has observed, the ideal copyright enforcement system exhibits maximal compliance and minimum prosecution. Especially for books, monitoring systems should be as open as possible and visible to users to maximize compliance and to create confidence they are not also snooping on reading habits.

It's interesting to read about the experiences of a university that implemented monitoring of P2P networks to comply with HEOA. Illinois State's Digital Citizen Project's summary of "Escalated Response System Testing Utilizing Audible Magic Copysense" (pdf, 1.5 MB) is valuable reading. While it's hard to be sure that Illinois' program was effective (you can't measure events that have evaded detection), I found it interesting that Illinois State students expressed minimum complaints or concern about the program.

A company with content identification technologies similar to those of Audible Magic is Nexicon. Both companies have agreements in place to work with YouTube to help to identify copyrighted material in uploaded videos, but Nexicon's business model aligns them with enforcement-oriented rightsholders. Here's how Nexicon President Sam Glines describes their flagship services:

Through our GetAmnesty and PayArtists solutions, we share with the rights holders settlements collected via the DMCA notices sent to infringers. The copyright holder sets the dollar amount per infringement - in the case of PayArtists and for Frank Zappa, the settlement amount is $10 per infringement. Nexicon’s MARC platform is capable of sending 95 million DMCA notices each day. Nexicon’s MARC platform monitors billions of illegal downloads of copyrighted material on a daily basis.

Nexicon has recently been involved in controversial takedown notices which Prof. Mike Freedman of Princeton's Center for Information Technolgy Policy describes as "inaccurate enforcement". In addition to defending Frank Zappa's copyright interests, Nexicon, a public company, boasts about fighting child pornography. At the same time, it appears to be associated with a New Jersey company that represents pornography publishers in their battle against copyright pirates. It can be hard for a technology company to control how their customers employ technology, but I would like to see more clear and coherent explanations of what happened to Freedman than Nexicon has provided to date.

Identification of ebooks is rather a different endeavor than identification of video or audio files. Copyrighted content in audio and video files can be identified in a number of ways, including watermarking, hashing and fingerprinting. As its name implies, Audible Magic's roots are in the audio fingerprinting area, and its huge library of 7 million song fingerprints is a significant asset, but they increasingly need to use textual clues such as those required for eBook identificationand are interested in further developing book-related identification techniques. As I've written previously, textual fingerprints are surprisingly effective at identifying books, even using a single sentence.

Book publishers preparing to fight piracy need to first and foremost have their content ready to be identified. While metadata, epub files and the like will be useful in locating and identifying pirated content that includes OCRed text, scanned images of books are also likely to be useful for the development of content recognition systems. If book publishers don't at least have scan files of every book they own, now is the time to start for them to start scanning!

Enforcement is only one weapon in the fight against book piracy, and it is the one weapon that most quickly loses effectiveness, as the techniques of copyright evaders evolve. One potential weapon that should be avoided is the dirty trick. If book publishers are unable to learn from the Sony rootkit fiasco, they will get all the ill will and lawsuits they deserve.

The shaping of societal behavior is a hopeless endeavor if the stick is wielded without a corresponding carrot. Any psychologist will tell you that most powerful tool in modifiying human behavior is positive reinforcement. If ebooks are to succeed commercially, publishers must use every means possible to reward people who purchase ebooks. I hope to write more about this soon, but I believe that positive reinforcement is the best lens to look at DRM with. DRM will fail unless its users believe it is rewarding them with convenience and ease of use, and with sufficient reward, it is also unnecessary - that is the lesson of iTunes.

As the era of digital books dawns, book publishers should expect that business models will change. Their mission, if they choose to accept it, is not only to deal with unauthorized use, but also to lead users to a social consensus that benefits everyone.

Update: In this post, I managed to overlook Attributor. Here's a post about them.

Links

Copyright Enforcement/Monitoring Companies

• Attributor
• Audible Magic
  
• Civolution
  
• DtecNet
  
• Nexicon
  
• Red Lambda
  
• SafeMedia
  
• Vobile
  

Privacy Organizations

• Center for Information Technology Policy
  
• Electronic Frontier Foundation
  
• Public Knowledge
  

Content Industry Organizations

• MPAA
  
• RIAA
  
• SIAA

The Uniqueness of Sentences and J. K. Rowling's (Non)Infringement of Tanya Tucker

Have you ever heard someone say something unusual and wonder to yourself if anyone in the history of humanity had ever said that before, ever? It happens a lot more than you might think.

In the discussion of my article on copyright salami, I suggested that copyright based on content as short as a sentence would not be very robust. I had reasoned that if the sentences were short enough, the would be a high probability that the same sentence had already appeared in a copyrighted work, or even in a work that was in the public domain. I imagined building huge databases of sentences that had already been used so as to clear them for reuse.

I decided to do some testing first. I chose a page at random (p. 447) from my (print) copy of J. K. Rowling's Harry Potter and the Deathly Hallows. I extracted the sentences, and put each sentence into Google and into Google Book Search. The results surprised me.

My first test sentence was

"Get - off - her!" Ron shouted.

With only 5 words, none of them uncommon, I expected to get a a few close matches. The book search produced zero hits, and no results at all close. The general Google search was more interesting. Of the 7 hits, all of them exact matches, the top two of seven hits appear to be properly attributed fair use quotations from the book. Two other hits were to complete, unauthorized copies of the book. One of these, on SlideShare, offers this disclaimer:

"hey here i got this book in pdf format .. am i violating anything after .. uploading this stuff over here ... just let me know .. if any issue come in existence, will remove it

Although the item has had 34,000 views, it pdf itself appears to have been removed from SlideShare. The pdf posted by a Filipino web designer on his web site, though, is still available (and has been since August) and is of quite good quality.

The oddest hits are to a site which masquerades as a "game ranking" portal site.

RPGRank is a real-time online game ranking system which provide a best MMORPG ranking portal for both players and games of all genre with the exclusive news, press release, review, preview, interview, trailer and vedio. RPGRank strive to provide all gamers things that they never experienced before by newest game beta keys, live-event, and online tournamentsa with attractive giveaways from games.

It appears that this site generates pages of random text for the benefit of search engines by extracting sentences from books and feeding the sentences to Google in a random order. This site has convinced Google to index "about 318,000" pages of its meaningless "content", and offers to sell "background" advertising space on the site at $1200 per month.

The last hit appears to be to a site which is presenting a Vietnamese translation of the book alongside the complete English text. Although I can't read Vietnamese, I doubt very much that it is authorized use. Vietnam joined the Berne convention only 5 years ago, so this is certainly an illegal infringement.

Of the 26 sentences on page 447, I could find only three that had been used in places that Google knows about. The first, "Leave him alone, leave him alone!" is a line from a Tanya Tucker song. The second, "Harry's stomach turned over.", has been used in James Edward Amesbury's "bloody but weakly conceived thriller", A Sporting Chance and in D. Edwards Bradley's Harry's War.

The third,"Harry did not answer immediately." is firmly in the public domain, having done duty as a complete sentence in Smith Hempstone's A Tract of Time, as a fragment in Frances Elizabeth G. Carey-Brock's 1867 My father's Hand: and Other Stories, and in Adam Williams' 2007 gripping adventure of modern China, The Dragon's Tail.


Three sentences comprising bits of dialog: "Been Stung", "And your first name?", and "Vernon Dudley", turned up numerous matches to fragments of sentences in Google. It was also amusing to see matches for the sentence "What happened to you, ugly?" This phrase matched two people-search sites which specialize in feeding Google pages with text like "What happened to Joe Smith?" Apparently there is someone who uses the screen name "you_ugly", and the people search engines just leapt to the wrong conclusions!

Most of the sentences on page 447 appear to be purely original to J. K. Rowling. Was she lucky, or were the odds stacked in her favor? Word frequencies for English have been measured, so we can easily generate a simplistic estimate of sentence occurrence rate. Ignoring the proper name "Ron", the words "Get", "off", "her" and "shout" have occurrence frequencies of 0.22%, 0.046%, 0.22%, and 0.0055%, respectively. Multiplying these occurrence rates gives us a weighted occurrence probability of this combination of 1 per 8 trillion. If you had the entire population of earth speaking random four-word English sentences they might come up with this combination in a day or two. Add "Ron" into the mix, and they might take the greater part of a year to generate the sentence J. K. Rowling wrote.

For context, it's interesting to guess at the total number of sentences that humanity has written or spoken. It's estimated that 100 billion humans have lived so far. If those humans spent 16 hours a day for an average of 65 years generating 3 sentences per minute, we'd be up to about 20 million trillion sentences. The real number is probably a factor of 100 to a thousand less (half of us are men, after all!). This estimate roughly agrees with estimates of others that all the words ever spoken could be archived using 10 exabytes of storage.

Ten exabytes is not as much storage as it used to be. The Internet Archive currently has 0.003 exabytes; although Google is quite secretive about its hardware deployment, it seems likely that their current storage capacity is in excess of 10 exabytes. Yesterday, Google announced a pricing plan where they'll rent you 0.000016 exabytes for $4096 per year. I'll do the math for you. If you want to store everything anyone has ever said, Google will rent you the space for only $2.5 billion dollars per year!

Given that Google will soon have digitized a large fraction of the world's books, there are a few things we can learn from this exercise.

• It will soon be very easy for Google to detect unauthorized copies of books in its index, and presumably to remove them. The benefit to publishers of doing this would hugely outweigh any damages they're suffering from the Google Books digitization program. Why have publishers overlooked getting this to happen as part of the agreement settling their lawsuit?
• It will not be difficult for Google to accurately de-duplicate the Google Books index.
• J.K. Rowling's hesitancy to release her books in ebook format is really, really stupid.

Before you get distracted with something useful, do this: pick about 5 random words, make a sentence from them, and become the first human ever to say that sentence. Depending on what you do next, you may also be the last!

Rehashing the Copyright Salami

I got a lot of feedback on my post on "Copyless Crowdscanning" from a variety of people. The comments, taken as a whole, do an excellent job of illuminating the ambiguities and difficulties of copyright law as applied to digitization.

Some people who read the article, such as Robert Baruch, assumed that I was trying simply to evade copyright by distributing the copying among many people and then enabling users to reassemble books in their browser. Many critics doubt that a salami strategy is much of a defense against copyright infringement, because a judge will see a copy appearing on a users browser, and won't be impressed at the details of how the copy was assembled. James Grimmelmann wrote me:

The problem is in your assumptions; two of them depend on a belief that copying small portions (or making small portions available) is categorically not infringement. It probably isn't, taken alone. But that doesn't mean we can aggregate those small amounts and have them stay non-infringing, any more than we can integrate infinitesimals and have them stay infinitesimal. I think a court would wind up saying that the activities of the various people in the system could be treated as part of a common plan of action, at which point there's full-text copying going on, and thus potential infringement.

I agree with this assessment, and thus a "copyless crowdscanning" organization would have to figure out how to make the reconstitution of the fulltext impossible. And as I explained, engineers hate to hear that something is impossible!

My main interest, however, was not to make salami by reassembling slices, but to make hash. I was assuming that an index of a book is not a copy of a book or a derivative work, but rather a collection of facts about the book. I am willing to grant that if a copy of a significant fraction of a book can be generated from a collection of facts, then that collection is equivalent to a copy, but if not, then it is neither a copy nor a derivative work.

I recognize however, that there are lawyers willing to argue that any index is inherently a derivative work and that rights holders should be able even to control the indexing of their work. It's clear from looking at cases such as The Harry Potter Lexicon case and Seinfeld Aptitude Test case that judges use a variety of tests to determine if a work is indeed a collection of facts. In the latter case, infringement was established because the "facts" were fictional. Because judges look at a variety of factors, we can't remove copyright law from the picture just because there's no copying. Still, the judge in the Harry Potter Lexicon case was pretty clear that an index per se is a transformative and thus allowed use of the work:

... the Lexicon identifies more than 2,400 elements from the Harry Potter world, extracts and synthesizes fictional facts related to each element from all seven novels, and presents that information in a format that allows readers to access it quickly as they make their way through the series. Because it serves these reference purposes, rather than the entertainment or aesthetic purposes of the original works, the Lexicon’s use is transformative and does not supplant the objects of the Harry Potter works.

Another set of criticisms said that the slices in my salami were too big. Grimmelmann says "A scan of a single page is probably enough to infringe." Richard Nash asserts that "copying a sentence *is* prima facie infringement." And Wes Felter, in a comment, pointed at this article which goes into the rather deep philosophical and abstract problems created by information reducible to a number that can be "coloured" with infringing intent.

We could modify the crownscanning software to use an even smaller slice, of course, but here we get to an interesting question: what is the smallest bit of text that can be protected by copyright? I think it would be silly to argue that single words could be copyrighted. As poetry, I'm sure a thousand word sentence could be. But as a practical matter, I'm guessing most any single sentence could be reused fairly. In the US, at least. And it would be really difficult for anyone to prove that a particular sentence had not been previously used in a copyrighted work. But unfortunately there's no obvious rule.

Is there any objective principle that could be used to determine the copyrightability of text fragments? A computer scientist might argue that a relevant measure should be the probability that a given text fragment could be generated at random, and if there were a lot of math-major judges, that might work. It seems to me that copyright law should at the very least be built using solid blocks. A legal apparatus built on a foundation of copyrightable 3-word sequences, for example, would quickly melt into uselessness.

The squishyness of today's copyright system imposes a huge cost on both users and owners of copyright. If the rules were clearer, rightsholders would find it easier to monetize their work, and society would benefit from the increased non-infringing use. For example, there could be a decision that said that reuse of less than 10 words cannot by itself be infringing and that 100 words is by default infringing. You don't need any calculus to enjoy a good bite of salami!