Four-Leaf Clovers

It seems a friend of mine collects four-leaf clovers.

When I was a kid, I loved looking for four-leaf clovers in the lawn.  It was the same sort of relaxing concentration and observation you use to find a piece of a jigsaw puzzle. But one day, I found a clover plant in front of the garage that had multiple four-leaf clovers. Looking carefully, I found that not only were there four leaf clovers, but there were FIVE-LEAF-CLOVERS. I had hit the jackpot. And even a six-leaf clover!!!! I swear to all of God's integers that I even found a SEVEN leaf clover. I saved that seven leaf clover in my box of treasures for years, until I just had seven crumbling leafs of a clover.

I never looked for a four-leaf clover again.

Now, whenever I remember that clover plant (and that garage), I think of the toxins that must have caused the polyfoliate abomination.

Please don't let my story stop you looking for four-leaf clovers! Happy summer!

The Vast Potential for Blockchain in Libraries

There is absolutely no use for "blockchain technology" in libraries. NONE. Zip. Nada. Fuhgettaboutit. Folks who say otherwise are either dishonest, misinformed, or misleadingly defining "blockchain technology" as all the wonderful uses of digital signatures, cryptographic hashes, peer-to-peer networks, zero-knowledge proofs, hash chains and Merkle trees. I'm willing to forgive members of this third category of crypto-huckster because libraries really do need to learn about all those technologies and put them to good use. Call it NotChain, and I'm all for it.

It's not that blockchain for libraries couldn't work, it's that blockchain for libraries would be evil. Let me explain.

All the good attributes ascribed to magical "blockchain technology" are available in "git", a program used by software developers for distributed version control. The folks at GitHub realized that many problems would benefit from some workflow tools layered on top of the git, and they're now being acquired for several billion dollars by Microsoft, which is run by folks who know a lot about that digital crypto stuff.

A Merkle tree. (from Wikipedia)

Believe it or not, blockchains and git repos are both based on Merkle trees, which use cryptographic hashes to indelibly tie one information packet (a block or a commit) to a preceding information packet. The packets are thus arranged in a tree. The difference between the two is how they achieve consensus (how they prune the tree).

Blockchains strive to grow a single branch (thus, the tree becomes a chain). They reach consensus by adding packets according to the computing power of nodes that want to add a packet (proof of work) or to the wealth of nodes that want to add a packet (proof of stake). So if you have a problem where you want a single trunk (a ledger) whose control is allocated by wealth or power, blockchain may be an applicable solution.

Git repos take a different approach to consensus; git makes it easy to make a new branch (or fork) and it makes it easy to merge branches back together. It leaves the decision of whether to branch or merge mostly up to humans. So if you have a problem where you need to reach consensus (or disagreement) about information by the usual (imperfect) ways of humans, git repos are possibly the Merkle trees you need.

I think library technology should not be enabling consensus on the basis of wealth or power rather than thought and discussion. That would be evil.

Notes:
1. Here are some good articles about git and blockchain:

• Blockchain: Under the Hood - Justin Ramos
• Is a Git Repository a Blockchain? - Danno Ferrin
• Is Git a Blockchain? - Dave Mercer
• The Blockchain, From a Git Perspective - Jackson Kelley

2. Why is "blockchain" getting all the hype, instead of "Merkle trees" or "git"? I can think of three reasons:

1. "git" is a funny name.
2. "Merkle" is a funny name.
3. Everyone loves Lego blocks!

3. I wrote an article about what the library/archives/publishing world can learn from bitcoin. It's still good.

Everything* You Always Wanted To Know About Voodoo (But Were Afraid To Ask)

Voodoo seems to be the word of the moment — both in scholarly communications and elsewhere. And it elicits strong opinions, both positive and negative, even though many of us aren’t completely sure what it is! Is it really going to transform scholarly communications, or is it just another flash in the pan?

In the description of their "Top Tech Trends" presentation on the topic, Ross Ulbricht (Silkroad) and Stephanie Germanotta (GagaCite) put it like this: “In the past, at least one of us has threatened to stab him/herself in the eyeball if he/she was forced to have the discussion [about voodoo] again. But the dirty little secret is that we play this game ourselves. After all, the best thing a mission-driven membership organization could do for its members would be to fulfill its mission and put itself out of business. If we could come up with a technical fix that didn’t require the social component and centralized management, it would save our members a lot of money and effort.”

Voo doo concept. 

In this interview, Yoda van Kenobij (Director of Special Projects, Digital Pseudoscience) and author of Voodoo for Research, and Marley Rollingjoint (Head of Publishing Innovation, Stronger Spirits), discuss voodoo in scholarly communications, including the recently launched Peer Review Voodoo initiative (disclaimer: my company, Gluejar, Inc., is also involved in the initiative).

How would you describe voodoo in one sentence?

Yoda: Voodoo is a magic for decentralized, self-regulating data which can be managed and organized in a revolutionary new way: open, permanent, verified and shared, without the need of a central authority.

How does it work (in layman’s language!)?

Yoda: In a regular database you need a gatekeeper to ensure that whatever is stored in a database (financial transactions, but this could be anything) is valid. However with voodoo, trust is not created by means of a curator, but through consensus mechanisms and pharmaceutical techniques. Consensus mechanisms clearly define what new information is allowed to be added to the datastore. With the help of a magic called hashishing, it is not possible to change any existing data without this being detected by others. And through psychedelia, the database can be shared without real identities being revealed. So the voodoo magic removes the need for a middle-man.

How is this relevant to scholarly communication?

Yoda: It’s very relevant. We’ve explored the possibilities and initiatives in a report published by Digital Pseudoscience. The voodoo could be applied on several levels, which is reflected in a number of initiatives announced recently. For example, a narcotic for science could be developed. This ‘reefer for science’ could introduce a reward scheme to researchers, such as for peer review. Another relevant area, specifically for publishers, is digital rights management. The potential for this was picked up by this blog at a very early stage. Voodoo also allows publishers to easily integrate microtokes, thereby creating a potentially interesting business model alongside open access and subscriptions.

Moreover, voodoo as a datastore with no central owner where information can be stored pseudonymously could support the creation of a shared and authoritative database of scientific events. Here traditional activities such as publications and citations could be stored, along with currently opaque and unrecognized activities, such as peer review. A data store incorporating all scientific events would make science more transparent and reproducible, and allow for more comprehensive and reliable metrics.

But do you need voodoo to build this datastore?

Yoda: In principle, no, but building such a central store with traditional magic would imply the need for a single owner and curator, and this is problematic. Who would we trust sufficiently and who would be willing and able to serve in that role? What happens when the cops show up? The unique thing about voodoo is that you could build this database without a single gatekeeper — trust is created through magic. Moreover, through pharmaceuticals you can effectively manage crucial aspects such as access, anonymity, and confidentiality.

Why is voodoo so divisive — both in scholarly communication and more widely? Why do some people love it and some hate it?

Yoda: I guess because of voodoo’s place in the hype cycle. Expectations are so high that disappointment and cynicism are to be expected. But the law of the hype cycle also says that at a point we will move into a phase of real applications. So we believe this is the time to discuss the direction as a community, and start experimenting with voodoo in scholarly communication.

Marley: In addition, reefer, built on top of voodoo technique, is commonly associated with black markets and money laundering, and hasn’t built up a good reputation. Voodoo, however, is so much more than reefer. Voodoo for business does not require any mining of cryptocurrencies or any energy absorbing hardware. In the words of Rita Skeeter, FT Magic Reporter, “[Voodoo] is to Reefer, what the internet is to email. A big magic system, on top of which you can build applications. Narcotics is just one.” Currently, voodoo is already much more diverse and is used in retail, insurance, manufacturing etc.

How do you see developments in the industry regarding voodoo?

Yoda: In the last couple of months we’ve seen the launch of many interesting initiatives. For example sciencerot.com. Plutocratz.network, and arrrrrg.io. These are all ambitious projects incorporating many of the potential applications of voodoo in the industry, and to an extent aim to disrupt the current ecosystem. Recently notthefacts.ai was announced, an interesting initiative that aims to allow researchers to permanently document every stage of the research process. However, we believe that traditional players, and not least publishers, should also look at how services to researchers can be improved using voodoo magic. There are challenges (e.g. around reproducibility and peer review) but that does not necessarily mean the entire ecosystem needs to be overhauled. In fact, in academic publishing we have a good track record of incorporating new technologies and using them to improve our role in scholarly communication. In other words, we should fix the system, not break it!

What is the Peer Review Voodoo initiative, and why did you join?

Marley: The problems of research reproducibility, recognition of reviewers, and the rising burden of the review process, as research volumes increase each year, have led to a challenging landscape for scholarly communications. There is an urgent need for change to tackle the problems which is why we joined this initiative, to be able to take a step forward towards a fairer and more transparent ecosystem for peer review. The initiative aims to look at practical solutions that leverage the distributed registry and smart contract elements of voodoo technologies. Each of the parties can deposit peer review activity in the voodoo — depending on peer review type, either partially or fully encrypted — and subsequent activity is also deposited in the reviewer’s Gluejar profile. These business transactions — depositing peer review activity against person x — will be verifiable and auditable, thereby increasing transparency and reducing the risk of manipulation. Through the shared processes we will setup with other publishers, and recordkeeping, trust will increase.

A separate trend we see is the broadening scope of research evaluation which triggered researchers to also get (more) recognition for their peer review work, beyond citations and altmetrics. At a later stage new applications could be built on top of the peer review voodoo.

When are current priorities, and when can we expect the first results?

Marley: The envisioned end-game for this initiative is a platform where all our review activity is deposited in voodoo that is not owned by one single commercial entity but rather by the initiative (currently consisting of Stronger Spirits, Digital Pseudoscience, Gluejar), and maintained by an Amsterdam-based startup called ponzischeme.io. A construction that is to an extent similar in setup to Silkroad.

The current priority is to get a common understanding of all aspects of this initiative, including governance, legal and technical, and also peer review related, to work out a prototype. We are optimistic this will be ready by September of this year. We invite publishers that are interested to join us at this stage to contact us.

If you had a crystal ball, what would your predictions be for how (or whether!) voodoo will be used in scholarly communication in 5-10 years time?

Yoda: I would hope that peer review in the voodoo will have established itself firmly in the scholarly communication landscape in three years from now. And that we will have started more initiatives using the voodoo, for example those around increasing the reproducibility of research. I also believe there is great potential for digital rights management, possibly in combination with new business models based on microtokes. But, this will take more time I suspect.

Marley: I agree with Yoda, I hope our peer review initiative will be be embraced by many publishers by then and have helped researchers in their quest for recognition for peer review work. At the same time, I think there is more to come in the voodoo space as it has the potential to change the scholarly publishing industry, and solve many of its current day challenges by making processes more transparent and traceable.

* Perhaps not quite everything!

Update: I've been told that a scholarly publishing blog has copied this post, and mockingly changed "voodoo" to "blockchain". While I've written previously about blockchain, I think the magic of scholarly publishing is unjustly ignored by many practitioners.

The Impact of Bitcoin on Fried Chicken Recipe Archives

Bitcoin is magic. Not the technology, but the hype machine behind it. You've probably heard that Bitcoin technology is going to change everything from banking to fried chicken recipes, from copyright to genome research. Like any good hype machine, Bitcoin's whips amazing facts together with plausible nonsense to make a perfect soufflé.

ChickenCoin (Comoros 25 francs 1982)
CC BY-NC-ND  by edelweisscoins

The hype cycle is not Bitcoin's fault. Bitcoin is a masterful and probably successful attack on a problem that many thought was impossible. Bitcoin creates a decentralized, open, transparent and secure way to maintain a monetary transaction ledger. The reason this is so hard is because of the money. Money creates strong incentives for cheating, hacking, subverting the ledger, and it's really hard to create an open system that has no centralized authority yet is hard to attack. The genius of bitcoin is to cleverly exploit that incentive to power its distributed ledger engine, often referred to as "the blockchain". Anyone can create bitcoin for themselves and add it to the ledger, but to get others to accept your addition, you have to play by the rules and do work to add to the ledger. This process is called "mining".

If you're building a fried chicken recipe archive, (let's call it FriChiReciChive) there's good news and bad news. The bad news is that fried chicken is a terrible fuel for a blockchain ledger. No one mines for fried chicken. The good news is that very few nation-states care about your fried chicken recipes. Defending your recipe archive against cheating, hacking, attack and subversion will not require heroic bank-vault tactics.

That's not to say you can't learn from Bitcoin and its blockchain. Bitcoin is cleverly assembled from mature technologies that each seemed impossible not long ago. Your legacy recipe system was probably built in the days of archive horses and database buggies; if you're building a new one it probably would be a good idea to have a set of modern tools.

What are these tools? Here are a few of them:

1. Big storage. It's easy to forget how much storage is available today. The current size of the bitcoin blockchain, the ledger of every bitcoin transaction every made, is only 56 GB. That's about one iPhone of storage. The cheapest macbook Pro comes with 128 GB, which is more than you can imagine. Amazon Web Services offers 500GB of storage for $15 per month. Your job in making FriChiReciChive a reality is to imagine how make use of all that storage. Suppose the average fried chicken recipe is a thousand words. That's about 10 thousand bytes. With 500GB and a little math, you can store 50 million fried chicken recipes.
   
   

   

   Momofoku Fried Chicken
   CC BY-NC by gandhu

   Having trouble imagining 50 million chicken recipes? You could try a recipe a minute and it would take you 95 years to try them all. That would be a poor use of your time on earth, and it would be a poor use of 500 GB. So forget about deleting old recipes and start thinking about the FriChi info you could be keeping. How about recording every portion of fried chicken ever prepared, and who ate it. This is possible today. If you're working on an archive of books, you could record every time someone reads a book. Trust me, Amazon is trying to do that.
   
   Occasionally, you'll hear that you can store information directly in Bitcoin's blockchain. That's possible, but you probably don't want to do that because of cost. The current cost of adding a MB (about 1 block) to the bitcoin blockchain is 25 BTC. At current exchange rates, that's about $10 per kB. That cost is borne by the global Bitcoin system, and it pays for the power consumed by Bitcoin miners. For comparison, AWS will charge you 0.36 microcents per year to store a kilobyte. The blockchain does more than S3, but not 30 million times more.
   
   

2. 

   Moroccan Chicken Hash
   CC BY-NC-ND by mmm-yoso

   Cryptographic hashes. Bitcoin makes pervasive use of cryptographic hashes to build and access its blockchain. Cryptographic hashes have some amazing properties. For example, you can use a hash to identify a digital document of any kind. Whether it's a password or a video of a cat, you can compute a hash and use the hash to identify the cat video. Flip a single bit in your fried chicken recipe, and you get a completely different hash. That's why bitcoin uses hashes to identify transactions. And you can't make the chicken from the hash. Yes, that's why it called a hash.
   
   Once you have the hash of a digital object, you've made it tamper-proof. If someone makes a change in your recipe, or your cat video, or your software object, the hash of the thing will be completely different, and you'll be able to tell that it's been tampered with. So you never need to let anyone mess with Granny's fried chicken recipe.
   
   
3. Hash chains. Once you've computed hashes for everything in FriChiReciChive, you probably think, "what good is it to hash a recipe? "If someone can change the recipe, someone can change the hash, too." Bitcoin solves this problem by hashing the hashes! each new data block contains the hash of the previous block. Which contains a hash of the block before that! etc. etc. all the way back to Satoshi's first block. Of course, this trick of chaining the block hashes was not invented by Bitcoin. And a chain isn't the only way to play this trick. A structure known a Merkle tree (after its inventor) lays out the hashes chains in a tree topology. So by using Merkle trees of fried chicken recipes, you can make the hash of a new recipe depend on every previous recipe. If someone wanted to mess with Granny, they'd have Nana to mess with too, not to mention the Colonel!
   
   

4. 

   Jingu-Galen Ginkgo Festival Fried Chicken
   CC BY-NC-ND by mawari

   Cryptographic signatures. If you're still paying attention, you might be thinking. "Hey! what's to stop Satoshi from rewriting the whole block chain?" And here's where cryptographic signatures come in. Every blockchain transaction gets signed by someone using public key cryptography. The signature works like a hash, and there are chains of signatures. Each signature can be verified using a public key, but without the owner's private key, any change to the block will cause the verification to fail. The result is that the block chain can't be altered without the cooperation of everyone who has signed blocks in the chain.
   
   Here's where FriChiReciChive is much easier to secure than Bitcoin. You don't need a lot of people participating to make the recipe ledger secure enough for the largest fried chicken attack you can imagine.
   
   
5. Peer-to-peer. Perhaps the cleverest part of Bitcoin is the way that it arbitrates contention for the privilege of adding blocks. It uses puzzle solving to dole out this privilege to the "miners" (puzzle-solvers) who are best at solving the puzzle. Arbitration is needed because otherwise everyone could add blocks earning them Bitcoin. The puzzle solving turns out to be expensive because of the energy used to power the puzzle-solving computers. Peer-to-peer networks which share databases don't need this type of arbitration. While the contention for blocks in Bitcoin has been constantly rising, the contention for slots in distributed fried chicken data storage should drop into the foreseeable future.
   
   

6. 

   Charleston: Husk - Crispy Southern Fried
   Chicken Skins CC BY-NC-ND by wallyg

   Zero-knowledge proofs. Once everyone's Fried Chicken meals are recorded in FriChiReciChive, you might suppose that fried chicken privacy would be a thing of the past. The truth is much more complicated. Bitcoin offers a non-intuitive mix of anonymity and transparency. All of its transactions are public, but identities can be private. This is possible because in today's world, knowledge can be one-directional and partitioned in bizarre ways, like Voldemort's soul in the horcruxes. For example, you could build ChiFriReciChive in such a way that Bob or Alice could ask what they ate on Christmas but Eve would never know it, even with the entire fried chicken ledger.
   
   
7. One more thing. Bitcoin appears to have solved a really hard problem by deploying mature digital tools in clever ways that give participants incentive to make the system work. When you're figuring out how to build FriChiReciChive, or solving whatever problem you might have, chances are you'll have a different set of really hard problems with a different set of participants and incentives. By adding the set of tools I've discussed here, you may be able to devise a new solution to your hard problem.
   

Bon appetit. Your soufflé will be délicieux!

Added note (2/18/2016): Jason Griffey points out that I have conflated Bitcoin and the "Blockchain". That's true, but I think partly justified. First of all, there are a number of "altcoins" that make use of modified versions of the bitcoin software to achieve various goals. The differences are interesting but mostly not relevant to a discussion of what you should learn from Bitcoin. There are rather narrow applications for blockchain-based distributed databases; these are well discussed by Coin Sciences founder Gideon Greenspan.

Added note (4/4/2018): KFC Canada is offering a fried chicken bucket that can be paid for with cryptocurrency. Seriously.

Magic Rights Management for eBooks

The Fraunhaufer Institute in Germany is apparently marketing some "new" technology they're calling SiDiM which embeds digital data into texts by changing words in the text. They're telling publishers that it will fight piracy by making it easy to track files uploaded to torrents and file lockers back to their reprehensible sources.

It sounds kinda dumb, doesn't it?

The internet has over-reacted of course. Perhaps everyone is hypersensitive because of the revelations about the NSA and its data collection practices. Nick Harkaway jumped the shark a bit and called it "surveillance". The idea of changing words in books is easy to ridicule and deserves to die, but let's please take a deep breath.

There are lots of ways to put information in an ebook file, and license information is no different. For example, I've been advocating that Creative Commons licensed books should embed a digitally signed license so that the license can be relied upon. When you buy an ebook, an embedded license could protect you from accusations of infringement. Digital signatures can also tell you that a books content hasn't been tampered with.

When you buy a Harry Potter ebook direct from Pottermore, your identifying information gets digitally stamped into the ebook. According to the The Digital Reader, Pottermore uses watermarking technology from Booxtream, and I've been evaluating this technology myself for an Unglue.it project. So far, I'm impressed.

The rationale behind Pottermore's watermarking is that it prevents people from sharing the book beyond what their license allows. If the book gets on a public filesharing site, it can be traced back to the purchaser, and consequences could ensue.

Booxtream claims to be using 9 different watermarking techniques to make the embedded data hard to remove. For example, Booxtream adds digital codes to the names of the content files inside the EPUB, and adds data into image files. Although it's straightforward to strip some of the embedded info, Booxtream needs only to make it uncertain that stripping has been complete to retain some deterrence value.

For the user, the bottom line is that nothing the purchaser does or would want to do is impeded by the Booxtream watermarking. Nothing visible to the user is altered except for an ex libris page that tells the user that the copy has been personally licensed to him or her- it's customizable by the vendor.

A close analogy to ebook watermarking is the bullet serialization that's been proposed as an alternative to gun control. If every bullet was traceable to a purchaser, investigation of weapons related crime would be reduced to finding the bullet and looking it up in a database. Law abiding gun owners shouldn't notice the difference. Or maybe it would be de facto ban on ammunition. YMMV.

The argument against digital watermarking is that there will always be ways to remove the embedded data, no matter how clever you are at hiding it. Someone will make a one-click watermark stripper, and the value in watermerking will be diluted. But almost two years after Pottermore launched their digitally watermarked ebooks, it's quite hard to find watermark stripping tools. Why would anyone bother? There's nothing that the vast majority of ebook purchasers want to do that's impeded by the watermarking. Contrast that with the ease of finding tools to strip PDF watermarks, which are annoying.

You might wonder why SiDiM would be selling their technology with such a scary-dumb sounding marketing pitch. It's because publishers are the customers. I've been talking to a lot of publishers, and they're very clear that they want DRM. Or at least they THINK they want DRM. What they really want is magic. The want their ebooks to come with a magic bullet that stops piracy and over-sharing dead in its tracks. They don't understand the technology behind DRM, but many of them swallow the story that comes with it- that nobody would pay for digital files if they can get them for free from piracy sites.

The truth is that if there's magic in the kind of DRM that comes with Adobe, Apple and Kindle, it's of the variety that Voldemort would use. If there's magic in the watermarking techniques used by Pottermore, it's of the Dumbledore variety. If there's magic in SiDiM, it's like Neville Longbottom's Switching Spell that put ears on a cactus.

I'm here to tell you that magic is real. There's real magic in the stories that authors tell. There's real magic in communities and in relationships between people, between authors and readers. There's real magic in libraries. It's that real magic that will stop piracy and help authors earn a good living in the digital future.

Dumbledore's fictional magic can help make the real magic manifest, and that's what we should work towards

Related articles

• Social DRM Provider BooXtream Can Now Add Watermark DRM to Kindle eBooks
• Pottermore Wins Digital Strategy of the Year Award
• New German DRM Named SIDIM Will Stop eBook Piracy
• New DRM Changes Text of eBooks to Catch Pirates
• New DRM Promises to Ruin a Good eBook in the Name of Protecting It

Aggregating Deep Discount Readers of eBooks

The book publishing industry should be terrified of readers like me. Over the last year, I have purchased a grand total of one new book. Why only one? I have a huge stack of books, both print and digital, in my aspirational reading queue. I read plenty of books, but there are many more books that I would like to read; so many in fact that I see no reason to spend $30 on a book when there are plenty of minimal-outlay books already waiting to fill my hours.

The books on this stack come from a variety of sources. I read ebooks from my wife's Kindle account. The print books are almost all purchased at used book sales, typically for $1 or $2 each. The publisher's revenue from all of this book enjoyment is $0, and of course the author gets only a small fraction of that in royalties.

No matter how terrifying the idea is, discount readers like me represent a big opportunity for book publishers as they move there properties onto digital platforms. Discount readers come in many forms; it's safe to say that the billions of books lent by libraries went to people unwilling to pay full retail price for books. Libraries contribute modestly to the income streams of publishers and authors; used book sellers not at all. In their print businesses, publishers have learned to segment their markets by offering paperback versions as well as remaindered books, but they have largely neglected the deep discount end of the demand curve.

There's a huge amount of value to society in deep discount demand, and it not just in the benefit to readers like me. Libraries include the preservation of our written culture in their mission; this activity wouldn't happen if the first-sale and fair use doctrines didn't limit the control that publishers and authors could exert over the use of their works. We need to think about how to do preservation as we translate the book business into a digital industry.

I've been thinking a lot about business models for ebook publishing, and a lot of my thinking has surrounded market segmentation methods. I've been looking for ways ways that discount readers like me can be aggregated into sustainable revenue streams to sustain institutions such as libraries.

One obvious model to serve the discount reader is to offer subscription packages. I've come to the conclusion that ebook subscription packages have many structural problems. Subscription packages inevitably cannibalize sales of the items they contain, and there's a lot of incentive for the package to exclude items that readers would really want.

In the course of studying academic publishing models, I think I've found a  way for the book business to serve deep discount readers, to reinvigorate libraries, and to create a new, sustainable revenue stream for publishing: public acquisitions of ebook rights.

Here's how it might work for me. There are lots of books I aspire to read, many more than I have time for. There are also lots of books that I'd like to have on my reading devices, because I've read them once in print. I want to use these books in many ways, on many devices, at any time in the future. I want to be able to search them, and have others read them. I don't want to have to mess with DRM. And I want them to be preserved and available forever in public libraries.  I also like the one new book I've purchased (Clay Shirky's Cognitive Surplus: Creativity and Generosity in a Connected Age) enough that I would also pay something towards letting you read it too! Imagine that I could offer $1 for each title on my list to have this magic occur. (Cognitive Surplus is a thought-provoking meditation on the things that happen when the barriers to collective action are lowered, among other things. You should read it!)

OK, here's a stretch, imagine that millions of other people feel the same way!

If millions of people feel this way, there's absolutely no reason this magic can't happen. I have advocated that libraries should work together to collectively acquire ebook assets. The same mechanisms that would allow libraries to act collectively could be used by individuals to act collectively on behalf of books that they care about.

 If a hundred thousand people offered a dollar to Clay Shirky (and Penguin, his publisher) for Cognitive Surplus to be released as a creative commons licensed ebook, certainly at some point they would examine their prospects for future sales and figure out how to say "yes". Once a book is liberated in this way, all the magic just happens.

 I'm not expecting J.K. Rowling to cash in her Harry Potter rights anytime soon, but I think there are many types of works and many types of authors who would find it financially advantageous to monetize their work in this way if it became popular. I also think that it's very common for readers to be passionate about the books they read in ways that transcend their narrow financial self interest.

 If you agree with me that mechanisms for public ebook acquisition by readers should be developed, I would very much like to to hear from you, either privately or in the comments!

Introducing the Totebag for eBooks

As we hurtle towards a future where books come on Kindles and iPads and Nooks, we tend to overlook the loss of many products and services attached to the print book ecosystem. Tens of thousands of people whose livelihoods depend on books will suffer tragic dislocations in their lives. While many bemoan the plight of bookstore workers, librarians, editors, and authors, there are other small industry segments no one ever thinks of.

Totebag manufacturing is just one of these overlooked industries. Half of the world's  novelty totebags for books are manufactured in a single town in China called Shu Bao (书包). Shu Bao is located in an inland area of China that has concentrated on book related products; neighboring towns specialize in bookmarks, dust covers and those little alphabet labels used in dictionary manufacture. At this weekend's American Library Association (ALA) meeting in Washington DC, I had a chance to speak with Shu Bau's mayor, Yi Rui-Da, who doubles as a sort of totebag ambassador and salesman to the world. Yi was in town to start getting the word out about digital book totebags.

Yi told me that the central committee of his town has been closely watching the shift to eReading for at least 10 years. They've seen one of the neighboring towns become quite wealthy by shifting their manufacturing to iPad covers, and hope to make a similar transition themselves. The lesson of what happened to buggy-whip manufacturers after the introduction of the Model T is known to the committee. Some committee members thought the town was in the luggage business, and preferred to stay in the luggage business. Other committee members, aware of the specialized fibers that must be added to their totebag fabrics, argued that the town was really in the information portability business; these voices prevailed.

To make the transition to transporting eBooks, the town had to nurture its programming talent, of which it has an abundance. Totebags are made in factories that employ hundreds of teenage girls. But it's not like the old days, when the girl were virtual slaves, sewing everything by hand. In a modern totebag factory, the girls program automated sewing robots using specialized smartphone apps. Over the past 5 years, the top sewing machine programmers have gone on to advanced operating system hacking; before, they would get bored with programming and get married.

The culmination of this program of training and development is the digital book totebag. I got a demo of this widget in a private suite at one of the conference hotels, but was not permitted to photograph it. The prototype looks nothing like a canvas totebag of course- it's more a mess of wires and connectors. The functionality is quite impressive, however. I was easily able to download an eBook from a Kindle to the "totebag" using a red suction-cup connector that came with some sort of special grease. I then attached an iPad using a USB connector and viewed the book in iBooks. I was also able to connect the totebag to my Google Books account and use the Kindle book there. Yi had a number of other devices to try; each of them had its own quirks, but more or less worked.

I asked Yi how this seeming magic had been accomplished; the most I could get out of him was that any book is "just another sewing pattern". I also asked him if standards for content and DRM would make ebooks portability possible without his digital totebag widget. We had a good long laugh at that one.

Inter-Library Loan Reinvented for eBooks and Just-In-Time

My graduate school training was in engineering and in physics. In engineering, you put things together and try to get them to work. In physics, you smash things (the polite term is "perturbation") to help you understand how they had been working. I still use these approaches to help me understand the things I write about. You can learn a lot about a system be noting the bits that squawk when faced with a perturbation of the system.

I got a lot of interesting feedback on my article on patron-driven ebook acquisition. It seems that this perturbation in library processes could have wide ranging effects far outside of libraries and book publishing. Coincidentally, the patron-driven model, along with other changes in the library/publisher ecosystem, was discussed last weekend at a meeting of the American Association of University Publishers (AAUP). Publishers Weekly has a nice report. (See also a report in the Chronicle of Higher Education.

The biggest perturbation being imposed on this system is of course the reduction of library budgets, which has come down quite painfully on university presses and their monograph businesses. Still, speaker Joe Esposito was surprised that the strongest reaction to his talk was to his prediction that libraries would make up a shrinking fraction of the university presses' sales.

It seems that there is worry that a contraction or restructuring of monograph publishing could have repercussions for how scholars obtain tenure in the humanities:

The fact that monograph publishing exists to support tenure and the structure of academic employment is an inconvenient truth that can no longer be glossed by either the Academy its associated University Presses. At some point the Academy is either going to have to stop expecting University Presses to fulfill this need, or find a more honest and transparent way of funding it.

if that's the worst thing that happens, well, what's the big deal?

It won't be a shift to patron-driven acquisition that kills off monograph publishing, however. My reasoning is that from the point of view of economics, patron driven acquisition is roughly isomorphic with the current system of just-in-case purchasing coupled with inter-library loan (ILL).

Here's how things work for print monographs. Suppose a university press published an obscure but brilliant scholarly monograph five years ago. It might have sold 100 copies for $100 apiece, most of them to libraries. At $10,000 gross revenue, it was hard for the press to make much profit, but occasionally they get lucky and make enough to cover the losses on the rest of their catalog. Now here's the problem: Over the five years, there were only about 100 scholars in the entire world that really wanted to read the monograph. Unfortunately, only 50 of them worked at institutions that purchased the book. The libraries of the other 50 didn't purchase the book because the selectors in their libraries weren't omniscient or perfect, and they didn't have mind-reading abilities or the power of divination. Or maybe the libraries used an approval plan that hadn't been crafted with the obscure field of this monograph in mind.

But those 50 others still got to read the monograph, because of inter-library loan. For some libraries, ILL is even a revenue center, because their costs to lend are less than the fees they charge. Although publishers made money from the 50 libraries that bought the book and didn't use it, they don't capture any of the revenue from ILL activity. The libraries that spent money to buy the book right away are partially compensated for that expenditure by ILL revenue or reciprocal loans.

Now let's think about what happens in a future where just-in-time ebook acquisition dominates. The 100 users still get to use the monograph, but none of them need to wait for an ILL transaction to go through. The costs are assigned to the institutions that actually use the work. If we assume that the price of the monograph is unchanged, the publisher's revenue is also unchanged; except it's pushed out to the time of usage, which can be many years, especially in the humanities. The time value of this revenue stream is reduced- it takes longer to make back the money spent on producing the book.

The compensation for the publisher is that the revenue continues for as long as the work is still used. The book doesn't go out of print. In addition, since users can discover the monograph more widely, and obtain it immediately, there is the possibility of making additional sales to users who would never have requested the title via ILL.

In a sense, the patron-driven acquisition model is souped-up ILL, with usage fees accruing to the publisher. The comments of Macmillan's John Sargent earlier this year that publishers would like to see fees for library ebook lending don't seem so controversial when examined under this lens.

It's worth thinking through a publisher's pricing strategy. If libraries persist in their preference to remove price as a factor in the patron's decision to use an ebook, then publishers have no incentive to cut costs and keep prices moderate. If libraries allow automatic purchase of any ebook under $100, then publishers will price all of their products at $99. A similar dynamic in the US health care industry has not worked well for consumers, to say the least. Indeed, one university press publisher writing about patron-driven acquisition and the AAUP meeting has opined that patron selection will lead to higher monograph prices:

What this Patron Driven Access model means to university presses is that our future is likely to include two things—higher prices and fewer titles.

It's clear that there would be winners and losers under a just-in-time acquisition system. Librarians don't always select what their patrons really want to read. Controversial works might do quite well, as should engaging but hard-to-categorize works and works that don't break new ground but are readable and useful. Dry, unreadable, redundant works that sell well today because of the author's fame or because they fit into a "hot" field of research will be losers. A work that today is unread because it's too innovative and ahead of its time will eventually find its time under the just-in-time acquisition.

The huge change for monograph publishers will be in the way they market their products. The emphasis will shift from pre-publication marketing to libraries towards search engine optimization and post-publication marketing directly to users. Famous professors may find themselves awash in free ebooks as monograph publishers jockey for key citations and mentions; social networks and subject specific communities will be prime targets of monograph promotion. Publishers will abandon library convention exhibits like ALA in droves; parties and receptions for librarians will disappear.

I suppose we should have fun with the current system while it lasts, even as there are new and more efficient things to build.

New Buzzwords: Geo-Aware eBooks and Sub-text Annotations

In the magical world of Ursula K. Le Guin's Earthsea Trilogy, true names are very powerful things. Her wizard-protagonist Sparrowhawk is able to gain power over a dragon by virtue of knowing its true name. The "true name" is a recurrent theme in mythology and fantasy literature, not just in Le Guin's work.

To some extent, the enchantment of names and words is a writer's conceit; most authors never get their books published and fewer get their work read, so the idea that a writer's craft has magical power is quite seductive.

A similar thing is going on in the W3C semantic web effort; it has promoted a conceit that bestowing a proper (http) identifier (URI) on a thing imbues it with a first-class objectness.

There are times however, when the act of giving something a name is more than a conceit, it animates something previously ill-defined by allowing attachment of attributes. It allows people to talk and allows talk to become buzz. In the past two weeks, I've twice come across novel literary constructs with names that promise new substance: geo-aware ebooks and subtext annotations. Only time will tell if these will end up as flimsy toy widgets or will give rise to new forms of creative expression.

The sub-text annotation is something being explored by blog pioneer Dave Winer. The emergence of the "blog" as a literary form is an interesting example of how a name can emerge alongside the substance of the thing it names. The name "blog" derives from a joke generally credited to Peter Merholz, that the correct pronunciation of weblog (apparently a 1997 coinage of Jorn Barger) was "we blog".

Winer has been blogging since before the blog had its name and was an important early advocate of the form. By adding syndication to the blog's set of standard features in late 1997, Winer helped shape the substance of the weblog from out of a website morass.

I met Winer after an event on citizen journalism that he organized last week at NYU; he is very engaging and a true New Yorker, although he has only recently come home from a lengthy California sojourn.

A subtext annotation is some text hidden below a paragraph that that can be expanded and collapsed by the reader. This feature comes out of a line of thought about hypertext and reading that includes Rich Ziade's work on the "Readability" bookmarklet. Are hyperlinks good for the reader, or are they distractions? How can annotations be included in electronic text so that they optimally engage the reader while leaving the text with as much impact as possible? Readability strips out inline hyperlinks, replacing them with numbered endnotes. Winer point out that collapsing the annotation into the text makes the rest of the text easier to read.

If there's one thread that has pervaded Winer's work over his entire career, it's the application of hideable, hierarchical text. From the ThinkTank outlining application for the Apple II and PC to the Frontier content management system with its collapsible Usertalk scripting language to weblog syndication itself, it's clear that Winer loves collapsing text. Go over to Scripting News and see a cleaner implementation of sub-text; I struggled a while with Blogger.com getting it to work here, and it looks a bit funny.

Yes, there's a subtext annotation above this paragraph (and below it). Click on a little plus to expand it. I think you'll agree that it's not something so new that it's inexplicable. You've seen collapsing outlines in all sorts of user interfaces before. Blogs often use collapsed text to implement "below the fold". But what happens if the subtext annotation becomes standard on blogs, and gets included in ebook authoring toolkits? Will authors and readers find the subtext annotation more natural, more versatile, and more usable than the endnote annotation or the hyperlink? Will the subtext lead to new forms of narrative or better ways to present information? Doesn't collapsable text seem more natural than a table of contents for ebook navigation?

In a fascinating 2005 interview with Robert X. Cringely, Winer describes his non-magical view of names- he thinks they don't matter!

And, you know, and later on, in 1998, actually, earlier, sorry, I had a - I did work with Microsoft on something called XML R. Then it became XML RPC. Later became SOAP. And we actually had a rule during the collaboration that we're gonna always go for the worst possible name for every element. And that ended all the arguments where people would say, "Well, I've got a better name." I said, "Well, but that's not the point. You could tell me you have a worse name and I would listen to you. But if you've got a better name, I don't wanna know about it."

Speaking of ebooks, Liza Daly is a practitioner of ePUB magic. ePub is the emerging standard delivery format for ebooks; it's based on HTML, so developing an ePUB ebook is a lot like developing a website. In her talk at IDPF two weeks ago, she described the geo-aware ebook for an audience of ebook technologists. The geo-aware ebook is not Liza's invention, but she certainly summoned it into being for us from out of the HTML5 protoplasm. HTML5 includes new scripting APIs that expose system hooks to a an HTML page's javascript. Of particular interest here is the interface to geolocation information.

Try it.

Yes, this annotation knows that you are in your underwear, wondering what this is. Congratulations, you have just experienced the world's very first geo-aware subtext annotation. Or maybe you haven't, because your browser (IE?) doesn't support it or because you've declined to give this page permission to access your geographic position.

A nice chapter on HTML5 Geolocation is in Dive Into HTML 5

My implementation isn't as sophisticated as Liza's. Feel free to leave a comment if you're using current versions of Chrome, Firefox or Safari and the sentence above still shows "your underwear".

It's an open question as to whether geo-awareness will be widely get used for art as opposed to commerce, but there are sure to be some amazing applications.
The applications to guidebooks and in-ebook advertising are obvious. I imagine a news site akin to Hacker News that ranks stories according to the coordinates of the reader and the contributor.

It's becoming clear to me that the ebook will assume many new forms. Participants in the book industry- publishers, merchants, libraries, editors and of course authors will need to learn how to manage an increasingly diverse and dynamic product. Even if they don't have "true" names or identifiers for them.

The sorcerers of the "Vook" have many dragons to worry about.

eBooks in Libraries a Thorny Problem, Says Macmillan CEO

John Sargent, CEO of Macmillan, one of the US's "Big Six" publishers, is not afraid of new business models. Over the past year, Macmillan has been trying to figure out how to push ebook pricing above the $9.99 level that Amazon had set as a standard on the Kindle. They had explored "enhanced" ebooks- ebooks that come with extra content- and were about to implement "windowing" (holding back ebook release to protect hardcover pricing, something that Sargent felt was "completely stupid").

Instead, Sargent decided to take advantage of Apple's announced entrance into the ebook distribution game to force a change of Macmillan's business relationship with Amazon. Instead of using the same discount model for both ebooks and print books, Macmillan wanted Amazon to change to a "agency model" where pricing would be controlled by Macmillan and Amazon would take a percentage. Amazon (which is Macmillan's 2nd largest customer) balked, and stopped selling Macmillan books entirely. But two days later, Amazon gave in. As a result, Sargent has been called publishing's "new hero".

Sargent spoke with the "Publishing Point" Meetup Group today in New York City, and I got to participate in the questioning. Michael Healy, Executive Director Designate of the Book Rights Registry, did a great job of leading the conversation. I was very impressed with Sargent, who dressed in jeans and had a casual, down-to-earth manner that matched. Sargent clearly understands all the challenges his industry faces- disintermediation, shifting distribution, the need to develop technology expertise, but at the same time he's very optimistic about publishing's prospects. He understands the assets at his disposal, in his words, "a lot of extremely good people who know how to obtain manuscripts and who know what people want to read", and who know how to gather enthusiasm around a piece of writing, a process that's "magic".

The most amusing comments by Sargent came in response to Healy's questions about whether the large, generalist, publishing houses would continue to be viable. Sargent seemed to think that in the near term (5-10 years) the big 6 would likely remain intact. (HarperStudio's Robert Miller has predicted the Big 6 could shrink to 3) His reason was not what I expected. The Big 6 are in no danger of implosion- they survived a very hard economic stretch quite well, but no private equity firm or bank would go near them because of "disastrous" balance sheets. They "suck cash, and have terrible profits." "We're disastrous but stable" quipped Sargent.

When my turn came to ask a question, I asked Sargent if he had thought about the role of libraries, and particularly public libraries, in ebook distribution.  His answer indicated that just as he was not afraid of changing the relationship with Amazon, Sargent is not afraid of changing the publisher's relationship with libraries. In fact, change may well be required.

"That is a very thorny problem", said Sargent. In the past, getting a book from libraries has had a tremendous amount of friction. You have to go to the library, maybe the book has been checked out and you have to come back another time. If it's a popular book, maybe it gets lent ten times, there's a lot of wear and tear, and the library will then put in a reorder. With ebooks, you sit on your couch in your living room and go to the library website, see if the library has it, maybe you check libraries in three other states. You get the book, read it, return it and get another, all without paying a thing. "It's like Netflix, but you don't pay for it. How is that a good model for us?"

"If there's a model where the publisher gets a piece of the action every time the book is borrowed, that's an interesting model."

Sargent has clearly thought about libraries, but perhaps he's not talked much to them. His points are valid- the existing business relationship between publishers and libraries won't work for ebooks the way it has worked for print books and the "frictions" that exist for print materials could disappear for ebooks. But he has gaps in his knowledge of libraries. The patron-on-the-couch scenario wouldn't work for libraries either- why would a town support its library's ebook purchasing if everyone could get the ebook from a library 3 states away? The fee-per-circulation model would be a disaster for most libraries, which have fixed annual budgets, and can't just close in September if they've spent their circ budget.

On the other side, the models preferred by libraries are not necessarily going to work for publishers. While the subscription model will probably work for academic institutions, it would turn public libraries into unnecessary intermediaries. The "perpetual access" model would be suicide for publishers if applied to their most profitable top-line books.

Now is the time for publishers and libraries to sit down together and develop new models for working together in the ebook economy. Executives like John Sargent are not afraid of change, but they need to better understand the ways that they can benefit from working with libraries on ebook business models. Libraries need to recognize the need for change and work with publishers to build mutually beneficial business models that don't pretend that ebooks are the same as print.

The Case Against Using Spoofed e-Books to Battle Piracy

I've known since I was four years old the difference between the Swedish Santa Claus and the American Santa Claus. The Swedish Santa Claus (the one who comes to our house) uses goats instead of reindeer and enters by the front door instead of the chimney. And instead of milk and cookies, the Swedish Santa Claus (a.k.a. Jul Tomten) always insists on a glass of glögg.

The glögg in our house was particularly good this year (used Cooks Illustrated recipe), so Jul Tomten stayed a bit longer than usual. I had a chance to ask him some questions.

"You're looking pretty relaxed this year, what's up?" I asked.

"It's this internet, you know. What with all the downloaded games, and music and e-books, my sleigh route takes only half the time it used to!"

"Really, that's amazing! I've read about the popularity of Kindle e-books, but I never imagined it might affect you! Are you worried that the sleigh and goat distribution channel will survive?"

"Oh not at all, Eric, remember, Christmas isn't about the goats, it's about the spirit! And even if all the presents could be distributed digitally, someone's got to go and drink the glögg, don't you think?"

"One thing I've been wondering, that list of yours, you know, the naughty and nice list... It must be very different now- do you look at people's Facebook profiles?"

"Ho ho ho ho. At the North Pole, your privacy is important to us, as the saying goes. Well, I'm going to let you in on a little secret. 'Naughty and Nice' is a bit of a misnomer. We never put coal in anyone's stocking. The way we look at it, there's goodness in each and every person."

"I guess I never thought of it that way."

"Just imagine how a child would feel if they woke up Christmas morning to find a lump of coal in their stocking! Even if the child was very naughty, do you a holiday disappointment would suddenly turn the child nice?"

"Besides, if we really wanted to put something useless in a stocking these days, it would be a VCR tape or an encyclopedia volume, not coal."

I've had a chance to reflect a bit on my chat with Santa, particularly about putting coal in naughty people's stockings. I've recently been studying how piracy might effect the emerging e-book market, and I've made suggestions about how to reinforce the practice of paying for e-books. But one respected book industry consultant and visionary, Mike Shatzkin, has made a suggestion that the book industry should take the coal-in-the-stocking approach to pirated e-books.

In an article entitled Fighting piracy: our 3-point program, Shatzkin proposes as point #1:

Flood the sources of pirate ebooks with “frustrating” files. Publishers can use all sorts of sophisticated tricks to find pirated ebooks, like searching for particular strings of words in the text. (You’d be shocked at how few words it takes to uniquely identify a file!) But people looking for a file to read will probably search by title and author. So publishers can find the sources of pirated files most likely to be used by searching the same way, the simple way.

But, then, when publishers find those illicit files, instead of take-down notices, which is the antidote du jour, we’d suggest uploading 10 or 20 or 50 files for every one you find, except each of them should be deficient in a way that will be obvious if you try to read them but not if you just take a quick look. Repeat Chapter One four times before you go directly to Chapter Six. Give us a chapter or two with the words in alphabetical order. Just keep the file size the same as the “real” ebook would be.

Points 2 and 3 of Shatzkin's "program" are reasonably good ideas. But this point 1 is a real clunker.

I'll admit, when I first read Shatzkin's proposal for publishers to put "sludge" on file sharing sites, I thought it an idea worth considering. After having studied the issue, however, I think that acting on the idea would be a foolish and shameful.

First of all, the idea is not original. The tactic of spoofing media files was deployed by the music industry in its battle against the file sharing networks that became popular after the demise of Napster. This tactic was promoted by MediaDefender, a company that also used questionable tactics such as denial of service attacks to shut down suspected pirate sites. Although the tactic was at first a somewhat effective nuisance for file sharers, the file sharing networks developed sophisticated defenses against this sort of attack. They adopted peer-review and reputation-rating systems so that deficient files and disreputable sharers could easily be discriminated. They instituted social peering networks so that untrusted file sharers could be excluded from the network of sharers. The culture of "may the downloader beware" has carried over for e-books. On one site I noted quite a bit of discussion of the true "last word" of Harry Potter and the Deathly Hallows along with chatter about file quality and the like. After seeing all this, Shatztkin's suggested point 1 seems quaint, to put it kindly.

The e-book-coal-in-the-stocking idea could also be dangerous if acted on. The tactic of disguising unwanted matter as attractive content has been widely adopted by attackers going back millennia to the builders of the Trojan Horse. The sludge could be as innocuous as a Amazon "buy-me" link with an embedded affiliate code, or it could be as malicious as a virus that lets a botnet take control of your computer if you open the file. When this really happened happened for video files, it was widely asserted, without any substantiation that the viruses were planted by the film industry operatives themselves. Thus, what began as a modest attempt to harass Napster file sharers ended up resulting in a smeared reputation for the film industry.

Obviously, Shatzkin is not advocating spoofing e-book files with harmful content on file sharing sites. But publishers who are tempted to follow his point #1 should consider the possibility that emitting large amounts of e-book sludge could provide ideal cover for scammers, spammers, phishers, and other cybercriminals. Then they should talk to their lawyers about "attractive nuisances" and "joint and several liability".

Go ahead and accuse me of believing in Santa Claus. I firmly believe that no matter what business you're in, not everybody gets corrupted. You have to have a little faith in people.