RA21 RP does not require secure protocols. It should.

As I've written, "RA21" could be a Good Thing, or it could be a disaster. The RA21 working group has released its "Recommended Practice" draft for comments, until May 17. The draft has much to like, but also has significant flaws. I will be contributing comments to address the flaws I see, which I will also publish here so we can discuss and comment. My official comments, and many others worth reading are here.

Here's my first comment, perhaps the most predictable:

RA21 RP does not require secure protocols. It should.

RA21 envisions the creation of a widely deployed authentication and authorization for resources and tools serving the research community. In such an ecosystem, the health and security of the entire system can be degraded by a small number of weak implementations. In particular, delivering resources over insecure unencrypted channels will be harmful.

In this context it is surprising that the RA21 recommended practice (RP) fails to directly address the need for service providers and identity providers to use secure channels such as HTTPS for websites. The recommended practice makes indirect reference to this need by citing another document, "WAYF Cloud and P3W Security & Privacy Recommendations". This document fails to treat secure channels as a requirement, saying in analyzing the pilot implementations (italics added):

"All browser traffic should use secured protocols, such as https, to prevent unauthorized access and to preserve confidentiality." (WAYF cloud, page 13)

"All browser traffic should use secured protocols such as https to prevent unauthorized access and to preserve confidentiality." (P3W, page 18)

In contrast to the "should" used for secure communications, the analysis uses the stronger "must" in other places, for example,

"Therefore, applications must include strong controls to prevent user ID tampering and abuse "(Information Disclosure, page 7)

Security and privacy issues essential to the success of RA21 should not be buried in technical analyses of uncertain normativity. Secure channels should not be optional, they must be required.

Update July 2, 2019:

RA21's official response to this comment is

We agree that HTTPS everywhere is a good idea for tools and resources serving the research community. However, a specific recommendation on this would be outside of the scope of RA21.

This response strikes me as uninformed, considering that the recommendation promotes a technical solution that will likely require publishers to adopt HTTPS. Either the committee is unaware of the technical ramifications of their recommendations (very likely), or they're trying to hide from the publishing community the inconvenient fact that RA21 will require all of them to go HTTPS (I wish).

Really, all I was hoping for some bland indication that RA21 will not compromise system privacy and security to accommodate the laggards of the service provider community. Since that didn't happen, I'll do some shouting here:

ATTENTION PUBLISHERS! RA21 WON'T TELL YOU IT'S GOING TO REQUIRE HTTPS ON YOUR SITE. I JUST DID.

A New Year's Resolution for Publishers and Libraries: Switch to HTTPS

The endorsement list for the Library Digital Privacy Pledge of 2015-2016 is up and ready to add the name of your organization. We added the "-2016" part, because various things took longer than we thought.

Everything takes longer than you think it will. Web development, business, committee meetings, that blog post. Over the past few months, I've talked to all sorts of people about switching to HTTPS. Librarians, publishers, technologists. Library directors, CEOs, executive editors, engineering managers. Everyone wants to do it, but there are difficulties and complications, many of them small and some of them sticky. It's clear that we all have to work together to make this transition happen.

The list will soon get a lot longer, because a lot of people wanted to meet about it at the ALA Midwinter meeting just 1 week away OMG it's so soon! Getting it done is the perfect New Year's resolution for everyone in the world of libraries.

Here's what you can do:

If you're a Publisher...

... you probably know you need to make the switch, if for no other reason than the extra search engine ranking. By the end of the year, don't be surprised if non-secure websites look unprofessional, which is not what a publisher wants to project.

If you're a Librarian...

... you probably recognize the importance of user privacy, but you're at the mercy of your information and automation suppliers. If those publishers and suppliers haven't signed the pledge, go and ask them why not. And where you control a service, make it secure!

If you're a Library Technology Vendor...

... here's your opportunity to be a hero. You can now integrate security and privacy into your web solution without the customer paying for certificates. So what are you waiting for?

If you're a Library user...

... ask your library if their services are secure and private. Ask publishers if their services are immune to eavesdropping and corruption. If those services are delivered without encryption, the answer is NO!

Everything takes longer than you think it will. Until it happens faster than you can imagine. Kids grow up so fast!

"Passwords are stored in plain text."

Many states have "open records" laws which mandate public disclosure of business proposals submitted to state agencies. When a state library or university requests proposals for library systems or databases, the vender responses can be obtained and reviewed. When I was in the library software business, it was routine to use these laws to do "competitor intelligence". These disclosures can often reveal the inner workings of proprietary vendor software which implicate information privacy and security.

Consider for example, this request for "eResources for Minitex". Minitex is a "publicly supported network of academic, public, state government, and special libraries working cooperatively to improve library service for their users in Minnesota, North Dakota and South Dakota" and it negotiates licenses databases for libraries throughout the three states.

Question number 172 in this Request for Proposals (RFP) was: "Password storage. Indicate how passwords are stored (e.g., plain text, hash, salted hash, etc.)."

To provide context for this question, you need to know just a little bit of security and cryptography.

I'll admit to having written code 15 years ago that saved passwords as plain text. This is a dangerous thing to do, because if someone were to get unauthorized access to the computer where the passwords were stored, they would have a big list of passwords. Since people tend to use the same password on multiple systems, the breached password list could be used, not only to gain access to the service that leaked the password file, but also to other services, which might include banks, stores and other sites of potential interest to thieves.

As a result, web developers are now strongly admonished never to save the passwords as plain text. Doing so in a new system should be considered negligent, and could easily result in liability for the developer if the system security is breached. Unfortunately many businesses would rather risk paying paying lawyers a lot of money to defend themselves should something go wrong than bite the bullet and pay some engineers a little money now to patch up the older systems.

To prevent the disclosure of passwords, the current standard practice is to "salt and hash" them.

A cryptographic hash function mixes up a password so that the password cannot be reconstructed. so for example, the hash of 'my_password' is 'a865a7e0ddbf35fa6f6a232e0893bea4'. When a user enters their password, the hash of the password is recalculated and compared to the saved hash to determine whether the password is correct.

As a result of this strategy, the password can't be recovered. But it can be reset, and the fact that no one can recover the password eliminates a whole bunch of "social engineering" attacks on the security of the service.

Given a LOT of computer power, there are brute force attacks on the hash, but the easiest attack is to compute the hashes for the most common passwords. In a large file of passwords, you should be able to find some accounts that are breachable, even with the hashing. And so a "salt" is added to the password before the hash is applied. In the example above, a hash would be computed for 'SOME_CLEVER_SALTmy_password'. Which, of course, is '52b71cb6d37342afa3dd5b4cc9ab4846'.

To attack the salted password file, you'd need to know that salt. And since every application uses a different salt, each file of salted passwords is completely different. A successful attack on one hashed password file won't compromise any of the others.

Another standard practice for user-facing password management is to never send passwords unencrypted. The best way to do this is to use HTTPS, since web browser software alerts the user that their information is secure. Otherwise, any server between the user and the destination server (there might be 20-40 of these for  typical web traffic) could read and store the user's password.

The Minitex RFP covers reference databases. For this reason, only a small subset of services offered to libraries are covered here. The authentication for these sorts of systems typically don't depend on the user creating a password; user accounts are used to save the results of a search, or to provide customization features. A Minitex patron can use many of the offered databases without providing any sort of password.

So here are the verbatim responses received for the Minitex RFP:

LearningExpress, LLC
Response: "All passwords are stored using a salted hash. The salt is randomly generated and unique for each user."
My comment: This is a correct answer. However, the LearningExpress login sends passwords in the clear over HTTP.

OCLC
Response: "Passwords are md5 hashed."
My comment: MD5 is the hash algorithm I used in my examples above. It's not considered very secure (see comments). OCLC Firstsearch does not force HTTPS and can send login passwords in the clear.

Credo
Response: "N/A"
My comment: This just means that no passwords are used in the service.

Infogroup Library Division
Response: "Passwords are currently stored as plain text. This may change once we develop the customization for users within ReferenceUSA. Currently the only passwords we use are for libraries to access usage stats."
My comment: The user customization now available for ReferenceUSA appears at first glance to be done correctly.

EBSCO Information Services
Response: "EBSCOhost passwords in EBSCOadmin are stored in plain text."
My comment: Should note that EBSCOadmin is not a end-user facing system. So if the EBSCO systems were compromised only library administrator credentials would be exposed. 

Encyclopaedia Britannica, Inc.
Response: "Passwords are stored as plain text."
My comment: I wonder if EB has an article on network security?

ProQuest
Response: "We store all passwords as plain text."
My comment: The ProQuest service available through my library creates passwords over HTTP but uses some client-side encryption. I have not evaluated the security of this encryption.

Scholastic Library Publishing, Inc.
Response: "Passwords are not stored. FreedomFlix offers a digital locker feature and is the only digital product that requires a login and password. The user creates the login and password. Scholastic Library Publishing, Inc does not have access to this information.”
My comment: The "FreedomFlix" service not only sends user passwords unencrypted over HTTP, it sends them in a GET query string. This means that not only can anyone see the user passwords in transit, but log files will capture and save them for long-term perusal. Third-party sites will be sent the password in referrer headers. When used on a shared computer, subsequent users will easily see the passwords. "Scholastic Library Publishing" may not have access to user passwords, but everyone else will have them.

Cengage Learning
Response: "Passwords are stored in plain text."
My comment: Like FreedomFlix, the Gale Infotrac service from Cengage sends user passwords in the clear in a GET query string. But it asks the user to enter their library barcode in the password field, so users probably wouldn't be exposing their personal passwords.

So, to sum up, adoption of up-to-date security practices is far from complete in the world of library databases. I hope that the laggards have improved since the submission date of this RFP (roughly a year ago) or at least have plans in place to get with the program. I would welcome comments to this post that provide updates. Libraries themselves deserve a lot of the blame, because for the most part the vendors that serve them respond to their requirements and priorities.

I think libraries issuing RFPs for new systems and databases should include specific questions about security and privacy practices, and make sure that contracts properly assign liability for data breaches with the answers to these questions in mind.

Note: This post is based on information shared by concerned librarians on the LITA Patron Privacy Technologies Interest Group list. Join if you care about this.