On the curl | sh pattern
by Etienne Millon on December 27, 2014
Tagged as: security.
Like many, I noticed a common pattern in the past few years: software authors
publishing instructions to download and install their program from their
website, directly in the terminal, through a variant of curl URL | sh.
Since I think that it should be considered bad practice, I created a tumblr called “curl | sh” which lists occurrences of this pattern.
I would like to address some of the criticism I received about this list. Most came from Hacker News where it has been posted.
“If the URL starts with HTTPS, it is secure”
The websites I post here fall roughly into three categories:
- downloads over HTTP.
- downloads over HTTPS with certificate verification deactivated
(
curl -k/--insecure,wget --no-check-certificate). - downloads over HTTPS.
Type 1 downloads are the most insecure ones, since it is possible to change the original response from the server without the client noticing. Modifying traffic like this is very easy, for example on wifi hotspots. This is not something that happens only in hacker movies: some places use this to insert ads in web pages.
Type 2 downloads are a bit better since the content is encrypted, but encryption without authentication is mostly useless since you do not know who you are talking to. The client will connect to anything that responds to url:443, which means that it is still possible to spoof a connection and actively change the response. To the client, an encrypted connection to an attacker looks the same as an encrypted connection to the legit site.
Type 3 downloads prevent this because they require that the certificate presented by the server matches the server name and is signed by a trusted Certificate Authority (CA). This means that the certificate has been handed to the person in charge of the website.
This is not foolproof either because the client has to trust a list of root CAs. This can be a problem for example in corporate environments, where the company can include their own CA to this list of trusted roots. At every HTTPS connection, they can create on the fly a certificate that is signed by themselves and with the correct server name. In other words, they can vouch for the identity of any website. As a consequence, they are able to spoof the TLS connection, in the same way that it is possible for type 2 downloads. Having a rogue trusted root is almost the same as disabling certification checking since it is able to create correct certificates for any site.
A way to mitigate this problem is to enable certificate pinning, which alerts the user when the certificate presented by a website has changed since the last time they consulted it. But this is not a perfect solution, since there are legitimate reasons to change a certificate. For example, they are usually limited in time, and every year or so it is necessary to generate a new one. However, if a website presents a certificate with a different anchor than before, this may mean that the connection is being spoofed.
Note that the main reason people disable certificate checking (i.e., use type 2 instead of type 3) is because they use self-signed certificates. These are certificates that are not signed by a CA. They are free and simpler to use, but do not authenticate the server, so they are rejected by default by clients. In browsers, this sometimes corresponds to a big scary warning, and by a yellow or open lock instead of the green, closed lock that we have all been educated to respect. It is however possible to pin them, so it is better than plain HTTP. The effort required to spoof a self-signed certificate is also greater than to spoof plain HTTP, but both are reasonably easy.
“apt-get install $pkg does the same thing”
Not exactly. When you install a package from the Debian archive, the .deb file is retrieved along with a digital signature that authenticates the file. This signature is checked against a key that is on all Debian systems. You obtain it at install time on a CD, but you can easily get it from another trusted Debian system if you can not trust a CD from some reason. The key here (pun intended) is that this scheme authenticates files and not connections.
For example, if a HTTPS website is compromised, you will not be able to detect
that the files have been modified on the server (and thus curl | sh will work
as before). But if your local Debian mirror is compromised, the files you
download will fail to validate against the key that is on your computer.
Of course, a signature from the Debian archive signing key does not automatically ensure that the package will not delete your root filesystem. But it ensures that the software has not been tampered with since its maintainer has uploaded it. Also, since it goes through a distribution, you may expect that some quality assurance has been made there.
It is hard to distribute software in a secure manner. Protecting the connection is definitely a first step, but protecting the files themselves is better. Fortunately it is possible to take this further, by having reproducible builds for example. I highly encourage anyone to read the triangle of secure code delivery on what it takes to deliver programs in a secure manner.
In the meantime, feel free to send me more examples of this pattern so that I can publish them. Get in touch either directly on curl | sh, on twitter or by email (see the footer below). Thanks for reading!
You can see the comments about this article on Hacker News.