How to Dig for Book Data Treasure

To me, surest indicator of an impending doom for book publishing is hearing a publisher cite the advertising of Attributor, an anti-piracy solutions company, as if it were science. It's not the attitude towards piracy that bothers me, that's entirely sensible. It's the implied devaluation of honest data that depresses me.

There's hope though. I've gotten to know quite a number of people throughout the reading ecosystem with whom I can use the word "data" as high praise, roughly equivalent to the word "gold". If you're reading this, chances are you're a member of this secret society, and what follows is a sketch of a treasure map.

In a recent post, I promised to suggest ways that we might measure the effects of library ebook lending on book sales. If you think about it, there are many parallels between attempting such a measurement and previous studies that have tried to measure the effect of ebook piracy on book sales. Unfortunately, the only objective study I know of was a small study done by Brian O'Leary, and the effects observed in that study were small and in a direction counter to popular narratives (and thus rarely noted in the sort of presentations that cite Attributor advertising).

In that study, O'Leary looked for time-domain correlations between sales figures for books from two publishers and the appearance of the same books on BitTorrent. A similar study focused on library Lending could be much more compelling, because library circulation data is a much more direct measure of distribution than any sort of torrent tracking, and librarians are much better than pirates at sharing data.

With the cooperation of booksellers, library circulation and holdings could be compared and correlated to store-by-store sales. For example, you could look at a book that's held in a significant fraction of libraries and look for correlations (positive AND negative) between areas where a library is circulating the book and stores where the book is selling. You've have to remove regional and demographic variance, of course, but with enough data, almost anything is possible.

With the cooperation of a large publisher, rigorous experiments could be done. Scientific experiments derive rigor from the use of controls. To prove that lending influences sales, it's not enough to do lending and look for sales. A rigorous experiment would have both a trial where books are lent and an identical trial where the same books are not lent.

One way to control a lending experiment would be to make a random selection of a publisher's catalog available for lending. Imagine if Penguin had worked with the library community on an experimental withholding of a random part of its catalog from Overdrive. The sales could be analyzed for patterns and trends.

It's important that data analysis of this sort be done objectively by researchers with integrity. In any large collection of data, it's possible to focus on data which supports one narrative over another. If lending-sales studies were done, my guess is that some types of books would show correlations very different from others.

I've used the word "cooperation" several times already. I'm not so naïve as to think that data sharing will materialize out of thin air. Perhaps the sort of eco-system wide organization envisaged by the same Brian O'Leary could be the vehicle to make data treasure digging possible. Opportunity in Abundance for the win!

Real Research Gets Reproduced

It's not often that I'm identified as a physicist, as Richard Curtis did in his commentary on my followups of Attributor's piracy "demand" report. But it's true, I worked in materials physics research at Bell Labs from 1988-1998.

Crystal structure of YBCO

Those were great years to be in materials physics. In 1986, two guys at IBM Zurich discovered some amazing new superconducting materials. By the end of that year, a team in Japan had reproduced their results; a group I was a part of at Stanford did the same after talking with the Japan group in December. By March, so many groups around the world had made exciting discoveries that the American Physical Society meeting in New York became known as "the Woodstock of Physics".

A blue semiconductor laser.

A few years later, a guy in Japan reported that he had made a semiconductor light emitting diode (LED) glow blue. His work was a lot harder to reproduce; it took years for anyone to come close to what his team reported; although he published many details, it was hard work. I even sawed one of his LEDs in half to try to understand how it worked. Today, my kitchen (and the screen of my MacBook) is lit by white LED's made from that same semiconductor.

Around that time, some chemists in Utah announced a truly amazing discovery: they saw fusion reactions occurring in palladium electrochemical cells. Since they were respected electrochemists, their results were taken seriously, and lots of people tried to reproduce the incredible results. The promise of a seemingly magical, unlimited power source seemed almost too good to be true. This time, however, nobody could reproduce the results. Some scientists saw odd things happen, but they were different in every lab. At Bell Labs, the scientists trying to reproduce so-called "cold fusion" became convinced that the guys in Utah were being led astray by their excitement.

In science, it's usual that a surprising result will only be accepted once it has been reproduced by someone else. My scientific training has sometimes gotten me in trouble in the world of libraries and publishing. When presented with something that seems surprising to me, I ask for the evidence. In cultures that are more comfortable assigning and recognizing authority, my questions have sometimes been seen as irritants.

It's been that way with my questions about the Attributor report. I was surprised at some of the findings, and I tried to reproduce them. My results can't reproduce some of the key findings reported by Attributor. It would be nice to better understand the factor of a hundred difference between my results and those of Attributor; much might be learned from such an analysis. Attributor is a company that sells anti-piracy services; one would hope that the data they report is somehow rooted in fact, even though they benefit from overestimates of privacy.

In Richard Curtis' article, Jim Pitkow, Attributor's CEO, is quoted:

Our study’s rigorous methodology ensured highly accurate results that align with actual consumer behavior. We analyzed 89 titles, using multiple keyword permutations per title, across different days of the week, with very high bids to ensure placement – each of which is fundamental in guaranteeing accuracy and legitimacy. Each of these variables impact the findings, and analyzing all variables together produce highly accurate results. We stand by our research, and we’re confident that the study addresses an accurate portrayal of the consumer demand for pirated e-books.

If Attributor really stands by its research, it will make it easier for people like me to reproduce their results. In particular, they should publish the complete list of the "869 effective keyword terms" used as keywords for their Google AdWords experiment. There are mistakes they might have made in permuting and combining search terms; they might also have thought of a class of effective search terms that my study totally overlooked. As it stands, it's impossible to know.

I can understand why Attributor might not want to release their search term list. First of all, they should expect people to try to tear it to shreds. The marketing department isn't going to like that. That's what happened to the superconductor guys, the blue LED guy, and cold fusion guys. They stood behind their work, and let the scientific community look for weaknesses and make their own judgments.

Cold fusion didn't pan out, and Pons and Fleischmann, the Utah guys, tried for years to figure out what it was they measured. Bednorz and Müller, the guys in Zurich, won the Nobel Prize. Shuji Nakamura, the LED guy, won a Millenium Prize and a lawsuit.

It may be easier to do a followup study without the worry of spurious searches for widely known terms. But at this point, Attributor customers and the book industry as a whole stand to learn a lot from understanding where the irreproducibility of Attributor's study is coming from. Publishers need that information to plan out a response to the threat of ebook piracy, and their needs should come first- no matter what the marketing department says.

Consumer Interest in Pirated eBooks is Even Lower Than I Thought

My recent posts following up on Attributor's most recent study on demand for pirated ebooks have been republished on TeleRead, probably the longest running blog covering ebooks and related topics. Paul Biba, the current editor, has been doing a great job bringing together interesting articles from many different perspectives.

The Teleread discussion on my article Attributor ebook piracy numbers don’t add up had some interesting contributions. Jim Pitkow, CEO of Attributor, suggested that the difference between the data I got from Google AdWords and Attributor's data might be a result of a difference in methodology. While I used Google's predicted traffic numbers, Attributor used numbers from an actual AdWords campaign. For example, Attributor bought the keyword “lost symbol free ebook” along with 868 others and counted how many impressions were generated by Google.

AdWords predicts a total query volume of 62 per day for the keyword “lost symbol free ebook”. If each of the other keywords did the same volume of queries, Attributor should have seen 53,900 impressions per day for its ad campaign. It's not at all clear how 53,900 impressions turns into 1.5-3 millions searches for pirated content; that would be a factor of 30-60 larger than the traffic predicted by the AdWords estimator tool. Pitkow's comment seemed to suggest that the result of an actual advertising campaign would be different from the estimate, possibly accounting for the discrepancy.

So I did an actual advertising campaign of my own to see if this was true.

I've used AdWords in the past with amazingly cost-effective results. I would spend about five dollars a month at five cents per click to advertise a service that sold for thousands of dollars per year. My experience, albeit somewhat dated, was that the estimator tool overestimated, not underestimated, the actual traffic. I was interested to see if this was still true.

I constructed a "free ebook survey" landing page for my ad campaign, and added StatCounter analytics so I could see who clicked on my ad. I bought the keyword suggested by Pitkow: “lost symbol free ebook”.

AdWords gives you a number of settings to fine-tune your ad campaign. For example, I checked all the boxes for geographical coverage so my ads would be seen in as many countries and languages as possible.

AdWords offers three important options that determine the distribution of the ads. You can choose to advertise on Google only, Google and its "search partners" or on Google, it's search partners, and Google's "display network". I spoke with Pitkow last week, and he indicated that Attributor's study used both Google and its search partners.

The initial results of my ad campaign were alarming. In just four hours, I accumulated 7,000 impressions and 19 clicks; my campaign halted because my bids were too low. This traffic level would easily support Attributor's estimates. But when I looked into it, I found that I had included the "display network" without meaning to. What's more, the referring sites were really junky. I couldn't imagine who would use sites like "Lost World TV", NDParking and Sebaidu. The 61 cents I spent in those four hours may well have gone to a bunch of sites engaging in click fraud.

A full week of advertising on just Google, by contrast, resulted in a grand total of 15 impressions, much less than Googles estimate of 62 per day. I next added Google's search partners to my campaign, and got an impression rate about three times higher.

Even with the search partners, the reported search volume is about a tenth of the predicted volume. I must therefore revise the estimate I made about "consumer demand for pirated ebooks". Instead of 100,000-300,000 searches per day, 10,000-30,000 per day throughout the world seems to be a better estimate.

As a result of this experiment, the Attributor numbers are even more inexplicable than before. It's worth noting however, the one area where there's no disagreement. Both my investigations and Attributor's show that consumer interest in piracy is mostly located outside the US, UK, and Canada. Jane Litte's recent post on the geographical restrictions conundrum for ebooks (and comments thereto) does an excellent job of describing why that may be so.

Attributor eBook Piracy Numbers Don't Add Up

In my article on "Consumer Demand for Pirated eBooks", I showed that Google Trends data tells a very different story from the one that anti-piracy services vendor Attributor derived from the very same data. I did not comment, however, on the headline that Attributor gave for its press release. The key finding of the report heralded by that release was that "Daily demand for pirated e-books can be estimated at 1.5-3 million people worldwide." This result has garnered some significant attention, because the number is quite large.

Extracting numbers using the tools used by Attributor is rather involved, and it's taken a while for me to carefully examine the available data. After doing this work, I've decided that when Attributor wrote "can be estimated at 1.5-3 million", they left out the word "blindly". As far as I can tell, Attributor is recklessly inflating the magnitude of ebook piracy; using the very same traffic measurement tools, I estimate the truth to be about 10% of the number they claim.

The Attributor numbers come from data generated by Google's AdWords service. AdWords is designed to help advertisers select advertising keywords and to manage budgets. For example, AdWords will tell you that the keyword "PDF" is used in approximately 101 million searches per month, worldwide, or 3.32 million searches per day. "PDF" is a keyword that a searcher might use in the course of a search for a pirated ebook, so you could reasonably assume that some percentage of these searches involve a consumer looking for a book they can avoid paying for. The trouble with this assumption is that most searches that include "PDF" have nothing to do with ebooks.

Another AdWords tool designed to assist Google advertisers is the keyword suggestion tool. In practice, you use this tool to refine keywords. Here is a table of the top ten refined searches for "PDF":

Keyword	percent of "pdf"
filetype pdf	36.69%
doc to pdf	6.03%
pdf download	3.30%
pdf to swf	3.30%
pdf to xls	2.70%
free pdf	2.70%
pdf free	2.70%
pdf to word	2.21%
pdf to rtf	2.21%
php pdf	1.81%

Of these, it's reasonable to assume that some percentage of the "pdf free" and a smaller fraction of the "pdf download" searches are related to consumers trying to avoid paying for books. The other searches are clearly unrelated to books. We can further use the keyword suggestion tool to refine these estimates. My review of over 700 refined keywords indicates that at most 4% of PDF searches, or 132,000 per day, are looking for ebooks of any kind.

A review of AdWords' suggested refinements for the term "rapidshare" reveals that searcher interest in ebooks is negligible compared to that for movies, TV, music and games. For example, Rapidshare is a "file-locker" site, and might be expected to appear in search terms for illegally distributed files. Of 743 suggested keywords, only one, accounting for 0.24% of "rapidshare" queries, or about 4,000 per day, is clearly related to ebooks:

Keyword	percent of "rapidshare"
files rapidshare	13.45%
rapidshare download	6.03%
download rapidshare	6.03%
download from rapidshare	6.03%
rapidshare megaupload	4.93%
free rapidshare	3.29%
rapidshare free download	2.70%
free rapidshare downloader	2.70%
free rapidshare download	2.70%
rapidshare download free	2.70%
free download rapidshare	2.70%
rapidshare free downloader	2.70%
download rapidshare free	2.70%
free rapidshare downloads	2.70%
download free rapidshare	2.70%
rapidshare searcher	2.19%
rapidshare search	1.80%
search on rapidshare	1.80%
dvdrip rapidshare	1.21%
rapidshare file	1.21%
rapidshare windows 7	1.21%
rapidshare mp3	1.21%
rapidshare dvd	0.99%
windows 7 rapidshare	0.81%
movie rapidshare	0.54%
rapidshare movie	0.54%
rapidshare upload	0.54%
upload rapidshare	0.54%
rapidshare downloader	0.44%
rapidshare file download	0.44%
rapidshare music	0.44%
music rapidshare	0.44%
download rapidshare files	0.36%
movies rapidshare	0.36%
rapidshare files download	0.36%
rapidshare windows xp	0.36%
720p rapidshare	0.36%
rapidshare premium accounts	0.30%
rapidshare password	0.30%
xbox 360 rapidshare	0.30%
game rapidshare	0.30%
password rapidshare	0.30%
rapidshare game	0.30%
rapidshare premium account	0.24%
premium account rapidshare	0.24%
rapidshare account premium	0.24%
premium rapidshare account	0.24%
rapidshare generator	0.24%
rapidshare engine	0.24%
rapidshare engine search	0.24%
up rapidshare	0.24%
rapidshare software	0.24%
software rapidshare	0.24%
rapidshare ebook	0.24%

Harry Potter and the Twilight Saga make appearances farther down the list, but only the titles that exist as movies.

Although direct interest in ebook torrents is so small that AdWords can barely measure it (~1500 searches per day), torrent search sites can give us another way to estimate the magnitude of interest in pirated ebooks. According to "KickassTorrents", the torrents active recently had this composition:

movies	30.04%
music	27.62%
tv	16.22%
apps	13.76%
games	5.52%
anime	5.43%
ebooks	1.42%

About 1.4 million searches using the keyword "torrent" are made on Google daily, according to AdWords. If the distribution of searches mirrors the distribution of files, this would indicate that searches for ebook torrents comprise about 46,200 per day.

All in all, I estimate that about 210,000 searches made on Google per day represent possible interest in pirated ebooks. About 30,000 of these come from the US. The "real" number for all countries could be as high as 300,000 or as low as 100,000. The 1.5-3 million numbers reported by Attributor are not within the range of plausibility.

One difficulty with using Google AdWords to gain insight into piracy is that it measures only a "shadow cast by piracy", as expressed by a commenter on my previous post. Nonetheless, AdWords sheds considerable light on patterns of demand. For example, the tools show clearly that it's common for people to search for movies and TV shows and acquire them extralegally. Also, they indicate that most of the demand, about 82%, for pirated ebooks comes from outside of the US, UK and Canada. Publishers should plan antipiracy strategies accordingly, based on data that can be confirmed independently.

Update: I have a followup post.

Consumer Demand for Pirated eBooks Stopped Growing in 2010

Online piracy of ebooks has been a persistent worry for book publishers who look at the successes and failures of other media that have moved to digital forms. A surprising number and variety of ebooks are easily availabile on file sharing websites and peer-to-peer networks that use bitTorrent and similar protocols. The possibility that this availability will cut into sales of licensed ebooks and even print books is a scary one for an industry that has had many decades of relative stability. At Digital Book World in January, Brian Napack, President of Macmillan, "delivered a passionate call to arms for publishers to fight piracy in the ebook space or risk permanent damage to the underpinnings of publishing as a commercial enterprise".

Adding to the ebook piracy hysteria have been studies of the prevalence of ebook piracy produced by Attributor, a company that sells anti-piracy services. I've previously written critically about Attributor's report that purported to find evidence that "Online Book Piracy Costs U.S. Publishers Nearly $3 Billion".

In their most recent report, Attributor has taken a rather clever approach to the measurement of ebook piracy. Instead of trying to track downloads, Attributor has begun to use Google Trends to gain an understanding of consumer demand for ebooks. Although there are many potential difficulties in using Google for this purpose, Google Trends is a powerful and useful tool for gaining insight into the things that web users around the world are looking for.

Attributor presents their data along with an alarming narrative of growing and pervasive ebook piracy, and points to the iPad as a contributing factor to an increase in demand for pirated ebooks. After playing around with Google Trends for a while, I've come to the conclusion that Attributor has narrowly selected data to fit their narrative; taken as a whole, Google Trends data broadly supports a rather different narrative: that the growth of consumer interest in pirated ebooks slowed significantly in 2009 and stopped in early 2010.

To understand how Google Trends informs the debate about the prevalence of ebook piracy, it helps to understand what activity is being measured. Google Trends measures the frequency that search terms are used. A consumer looking for a free copy of a particular work will typically search on the book title, adding  terms like "free" or "download" or "pdf" to locate downloadable files. A more sophisticated strategy, one that is quickly learned, is to add the name of a preferred download site. If the user prefers peer-to-peer networks, the word "torrent" can be added to locate "seed" files for the item. The file sharing sites most commonly used for this purpose are currently RapidShare, Megaupload, 4shared, and Hotfile. To use Google trends to measure the demand for a pirated ebook, you give it keywords that reproduce these searches. For example, demand for Stephanie Meyer's book Breaking Dawn can be assessed with a query such as this one.

To assess the overall state of ebook piracy, I used data from this query. Note that since the search is for ebooks generically, there's no telling for sure that the ebooks being searched for are really pirated; for the purposes of this study, I assumed that none of the ebooks being searched for are legally available on these sites. Calling them "pirated books" may be inaccurate, but I'll use that term anyway.

Some features of the data are immediately apparent. First of all, searches for pirated ebooks have increased a great deal over the past 5 years. It's worth noting however, that the most intense interest measured by Google occurs in India, the Philippines, Indonesia, Vietnam, Malaysia, Singapore, and eastern Europe. Less than half the search volume comes from the US. It's also easy to see seasonal peaks that obscure the shorter term trends. The peak periods for pirate ebook seeking are the December holidays and the beginning of September, presumably because of the start of school.

To eliminate seasonal variations, I computed the year over prior year growth of pirate ebook search activity. The resulting plot is quite smooth. After a few years of 100% per year growth, 2008 showed a clear slowing of growth. This slowing of growth continued up to the beginning of 2010, and then  flat-lined. Since February of 2010, the growth of interest in pirated ebooks has stopped completely.

It should be noted that this stabilization has occurred during a period of strong sales of ebook reader devices, including Kindle, Nook, and the iPad. Indeed, the unveiling of the iPad was coincident with the stabilization of demand for pirate ebooks.

It's hard to know for sure what's happening, but one interpretation of these patterns is that a broad increase in consumer-friendly availability of properly licensed ebooks over the last 2 years has squelched the growth of demand for ebooks from illicit sources. In that light, the remaining demand can be interpreted as a sign of poor availability for appropriately priced ebooks on college campuses and in developing countries.

While this data has to be seen as an encouraging sign for the book publishing industry, it's too soon to know if it will last. It's entirely possible that too-high prices, cumbersome DRM, or new technologies could reinvigorate the demand for illicitly shared ebook files. For the moment at least, the book publishing industry can exhale.

Deconstructing the Attributor Book Piracy Study

It was a lot of fun to have a post slashdotted and read by 100 times more people than any of my previous posts, however, the fact that it made fun of the way a study on piracy was "spun" overshadowed some serious points.

1. The study in question, from Attributor, in fact has some substance beneath the spin.
2. The number of books people get from libraries is comparable to the number they get from bookstores.

First, the Attributor study. Looking past the silly projections of book industry lost sales, there is some useful information there. In the study, illicit copies of 913 books were found on four one-click download sites which display download counts. The reported download counts for these 913 books totaled 3.2 million over 90 days. On average, each book was downloaded 3,500 times total, or 39 times per day.

These numbers are in rough agreement with a much smaller sample I collected for myself. I looked at only 10 books in a single category on a single site, and observed that download totals ranged for 0 to 5000, with an average of 578.

I followed up with some questions for Rich Pearson, the General Manager at Attributor. Most interesting to me, and probably of likely concern to publishers, was this: of 913 titles, chosen from Amazon lists to get a broad distribution rather than to focus on popularity, Attributor was able to find illicit copies of 90% of the books they looked for. I had expected that the most popular titles would be easy to find, but this breadth surprised me.

The extrapolations made to translate these download numbers into economic impact, while understandable from an marketing point of view, are multiply lacking in rigor.

The first assumption made by the study is that the "downloads" reported by  download sites represent potential readers of books. Attributor does not have any special relationship with the download sites to gain access to download statistics, they simply rely on the download counters published by the sites.  Assuming that the counts are not simply manufactured (and I've seen download sites that do just this) it's unlikely that the download sites bother to distinguish between robot activity and human activity. Robot activity is important because many of the download sites do not offer search themselves. This is a tactic that allows them to be unaware of, and thus not liable for,  the content that they host. The sites depend on other sites linking to them to drive traffic; they monetize the traffic by throttling downloads and offering "premium" subscriptions to people want to remove the throttles.

The second assumption made in the Attributor study is the way that they extrapolate numbers reported by 4 sites to the 25 sites that were monitored. What they did was to weight the sites based on the distribution of the 52,000+ takedown notices that Attributor has sent out since launching their monitoring service in July of '09. One problem with this is that while the scope of takedowns was limited to Attributor customers, the 913 monitored books were limited to non-customers. If Attributor's customers were focused in textbooks, for example, this would result in a bias towards sites used to host illicit textbooks. The other problem is "lamp post" bias- the extrapolation is based on what Attributor can find; sites hidden from Attributor would result in undercounting.

Third, the sampling extrapolation to cover the entire industry has significant uncertainty. I worry most about price bias. The 1,132 downloads of the book Freakonomics were reported, which seems insignificant against sales of 2.5 million copies. Freakonomics, ranked #77 at Amazon,  can be bought new for $9.35. By contrast, Architect’s Drawings, which was downloaded over 10,000 times and ranks #547,491 at Amazon,  is a $60 hardcover. (The high download count is likely due to use as a textbook). Thus the "lost sales" for Architect’s Drawings will be hugely overweighted in the extrapolation.

Finally, while the Attributor study does note that the actual effect of downloads on sales is speculative, there is a significant question as to whether the people downloading the book copies could have purchased the books even if they wanted to. It is evident from the chatter around the book downloads that many of the downloaders speak Spanish, French, Portuguese, Arabic, and Indonesian. According to Pearson, Attributor scans sites in many languages linking to illicit books, including Chinese, Japanese, Korean, French, Spanish, German, Italian, Czech, Polish, Russian, Portuguese and Austrian. The impact of piracy is likely to be very different in export markets. Not that this is shocking news to anyone.

Attributor is currently targetting its service (which starts at around $10,000) at large publishers, though it is working with a reseller called Author Guard to service smaller publishers. It points to its superior ability to find and take down illicit content as its comparative advantage. (I've previously surveyed other companies in this space; somehow I managed to overlook Attributor.)

Although many publishers will look at the numbers and conclude that services like Attributor are not worth the expense, I think the real danger from piracy is collective rather than individual, and thus the response should be collective rather than individual. The book publishing industry will only suffer if piracy becomes so widespread that piracy gains cultural acceptance. To prevent this from happening, book publishers need to lead the culture with both carrot and stick. Legal, allmost-free access to books must be provided such as occurs today in libraries, while illicit content should be taken down in as efficient manor as possible. This means that services such as Attributor's should be deployed on behalf of the entire industry, perhaps through a consortium, and not just by individual publishers. Finally, the book industry needs to figure out how to use ebooks to effectively address the needs of the developing world, or else huge markets will be forever out of reach.

The book publishing industry is entering some scary times and needs to decide who its friends are. A don't know whether technology companies with scary marketing will prove to be reliable friends or not. Amazon and Apple might end up being saviours, but publishers shouldn't expect them to be buddies. I'm pretty sure of one thing, though. Libraries are definitely not the enemy.

Offline Book "Lending" Costs U.S. Publishers Nearly $1 Trillion

Hot on the heels of the story in Publisher's Weekly that "publishers could be losing out on as much $3 billion to online book piracy" comes a sudden realization of a much larger threat to the viability of the book industry. Apparently, over 2 billion books were "loaned" last year by a cabal of organizations found in nearly every American city and town. Using the same advanced projective mathematics used in the study cited by Publishers Weekly, Go To Hellman has computed that publishers could be losing sales opportunities totaling over $100 Billion per year, losses which extend back to at least the year 2000. These lost sales dwarf the online piracy reported yesterday, and indeed, even the global book publishing business itself.

From what we've been able to piece together, the book "lending" takes place in "libraries". On entering one of these dens, patrons may view a dazzling array of books, periodicals, even CDs and DVDs, all available to anyone willing to disclose valuable personal information in exchange for a "card". But there is an ominous silence pervading these ersatz sanctuaries, enforced by the stern demeanor of staff and the glares of other patrons. Although there's no admission charge and it doesn't cost anything to borrow a book, there's always the threat of an onerous overdue bill for the hapless borrower who forgets to continue the cycle of not paying for copyrighted material.

To get to the bottom of this story, Go To Hellman has dispatched its Senior Piracy Analyst (me) to Boston, where a mass meeting of alleged book traffickers is to take place. Over 10,000 are expected at the "ALA Midwinter" event. Even at the Amtrak station in New York City this morning, at the very the heart of the US publishing industry, book trafficking culture was evident, with many travelers brazenly displaying the totebags used to transport printed contraband.

As soon as I got off the train, I was surrounded by even more of this crowd. Calling themselves "Librarians", they talk about promoting literacy, education, culture and economic development, which are, of course, code words for the use and dispersal of intellectual property. They readily admit to their activities, and rationalize them because they're perfectly legal in the US, at least for now.

Typical was Susanne from DC, who told me that she's been involved in lending operations for over 15 years. This confirms our estimate that "lending" has been going on for over ten years, beyond even Google's memory. Our trillion dollar estimate may thus be on the conservative side. Of course, it's impossible to tell how many of these lent books would have been purchased legally if "libraries" were not an option, but we're not even considering the huge potential losses to publishers when "used" books are resold for pennies on the black markets.

The communications backbone for this vast enterprise appears to be Twitter. Already, there is constant chatter on the #alamw10 hashtag. Most messages are clearly coded references to illicit transactions. For example a trafficker with the alias "@libacat" tweets "Have to be on the bus to the airport at 6:41 tomorrow morning to make it to the airport to get on my plane to #alamw10". At first glance, it seems like a mundane tweet about travel plans, but the breathtaking ordinariness and triple redundancy is more likely a secret code. How else to understand @scolford's (correction: retweet of @SonjaandLibrary replying to @BPLBoston) tweet; "curling my toes in joy at the thought of visiting your library"?

I've attended this meeting before. When I register for the book lending confab, I'll be presented with an encrypted document labeled the "program", which once decoded, will tell me where I can meet other book traffickers, discuss arcane trafficker lore, and drink trafficker beer. It's thick with secret code words like YALSA, LITA and NMRT, and no apparent rhyme or reason in its layout, evidently to frustrate outside investigators. I'll be lucky if I can find a bathroom.

Two places I'll be sure to find this weekend will be the OCLC Blog Salon on Sunday evening and the Chinatown Storefront Library on Saturday afternoon. Say hello if you see me.

A more serious post on Attributor is forthcoming.

Update: here's my post on "Deconstructing the Attributor Study".