The rest of this post is about plumbing surprises. If you're not interested in website plumbing, feel free to go watch a cat video.
I've written more than you want to read about redirection, a rather important bit of website plumbing. HTTP redirects enable things like link shortening (e.g. bit.ly), long term link maintenance (e.g. crossref.org and purl.org), and just-in-time linking (e.g. OpenURL). If you redirect to another redirector, you have what's known as a redirect chain.
You can easily imagine the kind of mischief that can go on with redirects, the redirect loop being the most obvious. The plumbing in your web software has to know how to avoid getting stuck in redirect loops or endless redirect chains, and for the most part it does.
Security issues can also arise with redirects, especially with mixed redirect chains. A mixed redirect chain is one that includes both secure (HTTPS) and non-secure (HTTP) links. Here's an example trace for a shortened ebook download link on the unglue.it website (it's the latest unglued ebook, Feeding the City, about the amazing human "plumbing" that delivers lunches to workers in Mumbai). You can try it yourself: https://bit.ly/19Ncaz7
The first thing your web browser does is it sets up a secure connection to bit.ly. While doing this it checks bit.ly's X.509 certificate with bit.ly's OCSP responder, Digicert. OCSP stands for "Online Certificate Status Protocol", and the result is that you can be reasonably sure that your connection is to bit.ly and that no one but maybe the NSA can snoop on your communication with bit.ly. In particular, no one can see what link you ask to be resolved, and no one but you can see bit.ly's answer.
(verify bit.ly, at http://ocsp.digicert.com/ )
GET /19Ncaz7 HTTP/1.1
In this example, bit.ly is redirecting to a non-secure URL, making the redirect mixed. Anyone between you and the destination can see what you're asking for if you follow the redirect. If you're in a Starbucks using wifi, Starbucks could conceivably send you a book about coffee instead. So the secure rigamarole you went through with bit.ly seems a bit wasted. But at least no one can see your bit.ly cookie and find out all the shortened links you've followed.
HTTP/1.1 301 Moved
GET /download_ebook/986/ HTTP/1.1
HTTP/1.1 302 FOUND
Since it's a different website, your web software needs to verify archive.org with their OCSP responder, GoDaddy.
(verify archive.org, at http://ocsp.godaddy.com/ )
GET /download/Feeding_the_City/9781909254039_Feeding_the_City.epub HTTP/1.1
The Internet Archive operates jillions of servers, and to save it the trouble of rebuilding its index whenever they move a file, they use a redirector to get you to the server where your ebook is living today. It's yet another server, so you have to check its certificate, too:
HTTP/1.1 302 Moved Temporarily
(verify ia801008.us.archive.org at http://ocsp.godaddy.com/ )
GET /4/items/Feeding_the_City/9781909254039_Feeding_the_City.epub HTTP/1.1
And so we get our ebook. Since it comes on a secure connection, we can be sure it's the one that Internet Archive meant to give us. Since there was an insecure link in the redirect chain, we can't also be sure that it's the one that bit.ly meant to send us to.
HTTP/1.1 200 OK
A great example of how to implement this plumbing is the Requests module for Python. (Also a great example of clear, readable source code and documentation!)
An example of a buggy implementation of this plumbing is the open-uri code in Ruby. From the source code:
At least this code errs on the side of security. If you use Ruby to try downloading something via a mixed redirect chain, open-uri will raise an exception labeled "redirection forbidden". Perhaps it would be more accurate to label this a "too dicey for Ruby" exception.
# This test is intended to forbid a redirection from http://... to
# file:///etc/passwd, file:///dev/zero, etc. CVE-2011-1521
# https to http redirect is also forbidden intentionally.
# It avoids sending secure cookie or referer by non-secure HTTP protocol.
# (RFC 2109 4.3.1, RFC 2965 3.3, RFC 2616 15.1.3)
# However this is ad hoc. It should be extensible/configurable.
- When when a links span multiple site, there's no practical way to ensure that your links don't get mixed. Even if they're not mixed now, that could change in the future.
- If you forbid https to http redirects, you're preventing sites from migrating to a more secure stance. A secure bit.ly would be impossible.
I tripped over the Ruby issue when implementing a connection to a partner that has built its site with Rails. They couldn't download some of our ebooks. Working together, we figured out what was wrong and implemented a work-around.
That's what us plumbers do for kicks.
- One thing you CAN'T do is redirect https to http if your certificate expires. To fix broken security, you need to fix the security.