Background
To secure communication between a Galaxy instance and its users it is best to enable HTTPS on the Galaxy web server, to ensure that all data transmissions between Galaxy and the end user (including sensitive information such as usernames and passwords) are encrypted. This can done by obtaining and installing SSL/TLS certificates on the server.
The simplest approach in the past was to use self-signed certificates as a way to enable HTTPS while avoiding the cost of purchasing certificates from a commercial Certificate Authority (CA) (for example by using the make-dummy-certs utility found in e.g. /usr/ssl/certs). The downside of this approach is that when a user first tries to access the server their web browser will complain that the certificates are not trusted, and they would typically have to create a one-off security exception before they can access the Galaxy service.
The simplest approach in the past was to use self-signed certificates as a way to enable HTTPS while avoiding the cost of purchasing certificates from a commercial Certificate Authority (CA) (for example by using the make-dummy-certs utility found in e.g. /usr/ssl/certs). The downside of this approach is that when a user first tries to access the server their web browser will complain that the certificates are not trusted, and they would typically have to create a one-off security exception before they can access the Galaxy service.
More recently however, a free Certificate Authority called Let’s Encrypt (https://letsencrypt.org/) has been set up which issues free certificates as part of its stated mission to “secure the web”. This blog post gives an overview of how we obtained and installed certificates from Let's Encrypt to enable HTTPS for our production Galaxy instances, using their automated cert-bot client utility.
Before beginning
The procedure described below uses the 'webroot' plugin of cert-bot (see https://certbot.eff.org/docs/using.html#webroot), which is a general method recommended for obtaining certificates web servers running nginx. cert-bot also has a plugin for nginx but at the time of writing this is still at alpha-release stage so I didn't use it for our Galaxy servers (see https://certbot.eff.org/docs/using.html#nginx for more details).
For Apache-based servers you can use a dedicated plugin described at https://certbot.eff.org/docs/using.html#apache, which offers a more automated procedure than the one described here.
Also, although it targets a different operating system to ours and while many of the details are now out-of-date, DigitalOcean's how-to guide at https://www.digitalocean.com/community/tutorials/how-to-secure-nginx-with-let-s-encrypt-on-ubuntu-14-04 is still a useful resource and was immensely helpful to me for understanding the overall process.
Finally, please note that the procedure and its details are likely to change over time. Make sure you check the documentation before carrying out any of these operations on your own infrastructure!
Step 1: Install cert-bot (Let’s Encrypt client) on the server
Before beginning
The procedure described below uses the 'webroot' plugin of cert-bot (see https://certbot.eff.org/docs/using.html#webroot), which is a general method recommended for obtaining certificates web servers running nginx. cert-bot also has a plugin for nginx but at the time of writing this is still at alpha-release stage so I didn't use it for our Galaxy servers (see https://certbot.eff.org/docs/using.html#nginx for more details).
For Apache-based servers you can use a dedicated plugin described at https://certbot.eff.org/docs/using.html#apache, which offers a more automated procedure than the one described here.
Also, although it targets a different operating system to ours and while many of the details are now out-of-date, DigitalOcean's how-to guide at https://www.digitalocean.com/community/tutorials/how-to-secure-nginx-with-let-s-encrypt-on-ubuntu-14-04 is still a useful resource and was immensely helpful to me for understanding the overall process.
Finally, please note that the procedure and its details are likely to change over time. Make sure you check the documentation before carrying out any of these operations on your own infrastructure!
Step 1: Install cert-bot (Let’s Encrypt client) on the server
To begin you need to ensure the Let's Encrypt cert-bot utility (https://certbot.eff.org/) is available on the server, to perform the job of obtaining and installing the certificates.
The documentation recommends that if possible you should use the cert-bot package provided by the package manager for your system (e.g. yum, apt etc). However if one isn't available (or is unsuitable e.g. because it's out-of-date) then you can install the client using the certbot-auto wrapper script instead (see https://certbot.eff.org/docs/install.html#certbot-auto). This is the approach I used, putting certbot-auto into /usr/local/bin on the server running Galaxy and nginx.
(Note that certbot-auto takes the same arguments as the cert-bot utility, the only difference is that if necessary it will download and update itself first each time it's run.)
- Aside: there is also a cert-bot package available via the Python Package Index (PyPI). When I first performed this procedure I noted that the documentation emphasised that cert-bot should not be installed 'pip install', but now I can't find any reference to this. However I would still avoid installing from PyPI for the time being.
Step2: Get certificates using the 'webroot' method
cert-bot provides a number of different ways to obtain certificates depending on the webserver software being used. The 'webroot' protocol used here is less automated than some of the other procedures but is still quite straightforward, and works by placing a special file on your webserver which Let's Encrypt can attempt to fetch in order to verify the server name and details that are supplied when the cert-bot client is run.
First we need to set up a special directory called .well-known, where Let's Encrypt will place its file:
- Create a directory called .well-known in the document root of the server (the default for nginx is /usr/share/nginx/hmtl but the actual path can be found by looking up the value of webroot-path in the server configuration), e.g.:
mkdir /usr/share/nginx/hmtl/.well-known
Optionally also add a dummy index file to help check that the directory is visible via web browser later, e.g.:
cat >/usr/share/nginx/hmtl/.well-known/index.html <<EOF
Hello world!
EOF
- Add a new location block inside the server block in the nginx configuration file, to allow access to the .well-known directory:
location ~ /.well-known {
allow all;
}
- Restart nginx and check that the .well-known directory is visible (e.g. by pointing a web browser at it)
Then we need to run certbot-auto (or cert-bot) interactively to generate and install the certificates:
certbot will then prompt you to agree to Let's Encrypt's terms and conditions and ask you to provide an email address which will be used for notices and for lost key recovery.
If all goes well then this should produce a set of certificate files under /etc/letsencrypt/archive (with links to these from /etc/letsencrypt/live/):
To enable HTTPS we need to configure nginx to listen on port 443 with SSL enabled, and to use the certificates from Let's Encrypt. This is done by adding the following to the server block in the nginx configuration file, for example:
server {
listen 443;
ssl on;
ssl_certificate /etc/letsencrypt/live/MYDOMAIN/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/MYDOMAIN/privkey.pem;
}
(Again, your actual domain name should be substituted for MYDOMAIN above.)
It's also a good idea to block or redirect HTTP traffic, so that users don't accidentally send data via an insecure connection - for example to redirect to :
server {
listen 80 default;
server_name MYDOMAIN;
rewrite ^ https://$server_name$request_uri? permanent;
}
Once nginx is restarted you can check using your browser that HTTPS is working for your Galaxy instance; you can also use the Qualys SSL Labs website to check your server configuration:
https://www.ssllabs.com/ssltest/index.html
(NB this is useful for flagging up other issues which you might wish to address!)
Step 4: set up automated certificate renewal
Finally: since all certificates issued by Let's Encrypt expire after 90 days, they recommend that they should be renewed at least once every 3 months.
It's straightforward to automate this process by setting up a cron job on the server to run cert-bot or certbot-auto's 'renew' command (which will renew any previously-obtained certificates that are due to expire in less than 30 days) and then restart nginx (so that any renewed certificates will be loaded).
For example I have the following commands in the root crontab on our server:
- sudo certbot-auto certonly --webroot -w /usr/share/nginx/html -d MYDOMAIN
where MYDOMAIN is the domain name of your Galaxy server (e.g. "palfinder.ls.manchester.ac.uk").
- Aside: note that this bootstraps certbot, including checking for the system packages that it requires; you'll be prompted to install any that it thinks are missing via the system package manager e.g. yum.
certbot will then prompt you to agree to Let's Encrypt's terms and conditions and ask you to provide an email address which will be used for notices and for lost key recovery.
If all goes well then this should produce a set of certificate files under /etc/letsencrypt/archive (with links to these from /etc/letsencrypt/live/):
- cert.perm (your domain's certificate)
- chain.pem (the Let's Encrypt chain certificate)
- fullchain.pem (cert.pem and chain.pem combined)
- privkey.pem (your certificate's private key)
- IMPORTANT: you should ensure that the certificate files are backed up to a secure and safe location!
To enable HTTPS we need to configure nginx to listen on port 443 with SSL enabled, and to use the certificates from Let's Encrypt. This is done by adding the following to the server block in the nginx configuration file, for example:
server {
listen 443;
ssl on;
ssl_certificate /etc/letsencrypt/live/MYDOMAIN/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/MYDOMAIN/privkey.pem;
}
(Again, your actual domain name should be substituted for MYDOMAIN above.)
It's also a good idea to block or redirect HTTP traffic, so that users don't accidentally send data via an insecure connection - for example to redirect to :
server {
listen 80 default;
server_name MYDOMAIN;
rewrite ^ https://$server_name$request_uri? permanent;
}
Once nginx is restarted you can check using your browser that HTTPS is working for your Galaxy instance; you can also use the Qualys SSL Labs website to check your server configuration:
https://www.ssllabs.com/ssltest/index.html
(NB this is useful for flagging up other issues which you might wish to address!)
Step 4: set up automated certificate renewal
Finally: since all certificates issued by Let's Encrypt expire after 90 days, they recommend that they should be renewed at least once every 3 months.
It's straightforward to automate this process by setting up a cron job on the server to run cert-bot or certbot-auto's 'renew' command (which will renew any previously-obtained certificates that are due to expire in less than 30 days) and then restart nginx (so that any renewed certificates will be loaded).
# Check SSL certificate renewal from Let's Encrypt
See the documentation at https://certbot.eff.org/docs/using.html#renewing-certificates for more information on certificate renewal.
Update 22nd October 2018: the original crontab lines above didn't work for me - the certificate renewals would fail and have to be performed manually, resulting in downtime for the period when nginx would no longer have valid SSL certificates.
Since then I've replaced the original crontab lines with the following single line:
30 2 * * 1 /usr/local/bin/certbot-auto renew --deploy-hook "/sbin/service nginx reload" >> /var/log/le-renew
which uses certbot-auto's --deploy-hook option to reload the nginx configuration on successful certificate renewal via the service command. Note that the full path to service is required as cron jobs have a minimal PATH which doesn't seem to include /sbin.
