Securing sites in the cloud with TLS/SSL has become a mandatory requirement. However, certificate management – certificate issuance, revocation, etc. – is non trivial even in corporate environments. And costs along with automation of certificate management act as further impediments in its adoption. Internet Security Research Group, a non profit organization, operates a free Certificate Authority (CA) Let’s Encrypt to address these challenges. Let's Encrypt supports a standard protocol ACME to access its certificate management features. This promotes automation.
For customers requiring a free and open source certificate management solution in OCI LBaaS, the A-Team (Security) has in place a solution for automatic renewal of certificates issued by Let's Encrypt and uploading these certificates onto OCI Load Balancer. The solution leverages the free, open source tool Certbot to interact with Let's Encrypt. Certbot and Let's Encrypt use ACME protocol where Certbot is the ACME client and Let's Encrypt is the ACME server (CA). Although Certbot can interact with any standard ACME CA server, it supports Let's Encrypt by default, i.e., out-of-the-box support.
Before issuing a certificate to an ACME client, the ACME server needs to get an assurance that the client indeed controls the domain, e.g., www.some-domain.com, for which the client is requesting the certificate. The ACME protocol specifies two types of challenges – DNS and HTTP. An ACME client can use either type to prove that it controls the domain. This blog post uses the HTTP challenge in the Certbot configuration.
The architecture is as follows -
At a high level, the flow is as follows:
Once certificates are installed on the LB, users can access the domain securely using HTTPS. Most modern browsers support certificates issued by Let's Encrypt. https://letsencrypt.org/docs/certificate-compatibility/ lists browser compatibilities.
The detailed steps are as follows:
4. Deploy a Linux OCI Compute instance.
5. Install Certbot.
6. Create a Backend Set with the Certbot instance.
7. Update the Health Check for the above Backend Set with TCP/22. The rationale for this setting will be explained later.
8. Create a Path Route Set as below with the route rules directing traffic to the Backend Set of step 6.
9. Create a LB Listener with HTTP/80 along with the Backend Set from step 6. Path Route from step 8 is used here.
The net effect of steps 8 and 9 above is that the request http://www.some-domain.com/.well-known/acme-challenge will be forwarded to the Certbot instance. This is part of the ACME HTTP challenge mechanism stated earlier. Let's Encrypt implements the ACME HTTP challenge by using the path /.well-known/acme-challenge .
10. Run the following Certbot command to initiate handshake with Let’s Encrypt
sudo certbot certonly --standalone --register-unsafely-without-email
Certbot is an ACME client. It interacts with a CA for certificate management. After the CA issues the certificate, Certbot typically deploys the certificate on a web server, e.g. Apache, NGINX. Since in this case the ultimate destination is the LB, no web server is involved. However, the web server functionality is required to serve the request http://www.some-domain.com/.well-known/acme-challenge coming from Let's Encrypt. The "standalone" flag is used precisely for that: it instructs Certbot to start a standalone web server for Let's Encrypt interaction.
In step 7 above, TCP/22 is used for the health check policy. Why TCP/22? Since this "standalone" web server has limited purpose, Certbot only starts it when required for Let's Encrypt interaction and shuts it down later. If OCI LB uses HTTP/80 for health check policy, then the health check polls will fail, marking the (Certbot) instance unavailable. This will break the functionality. Since OCI Compute instances have ssh enabled by default, TCP/22 is used for health check as illustrated in step 7.
The output of the "certbot certonly" command is the following:
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator standalone, Installer None
Starting new HTTPS connection (1): acme-v02.api.letsencrypt.org
Please enter in your domain name(s) (comma and/or space separated) (Enter 'c'
to cancel): www.some-domain.com
After entering the domain name, if everything goes well, following is the output:
Obtaining a new certificate
Performing the following challenges:
http-01 challenge for www.some-domain.com
Waiting for verification...
Cleaning up challenges
IMPORTANT NOTES:
- Congratulations! Your certificate and chain have been saved at:
/etc/letsencrypt/live/www.some-domain.com/fullchain.pem
Your key file has been saved at:
/etc/letsencrypt/live/www.some-domain.com/privkey.pem
Your cert will expire on 2019-11-26. To obtain a new or tweaked
version of this certificate in the future, simply run certbot
again. To non-interactively renew *all* of your certificates, run
"certbot renew"
If everything went well, then that would confirm that the loop Certbot → Let's Encrypt → OCI LB → Certbot ( steps 1 & 2 in the above architecture diagram) is complete and working.
11. At this stage, the certificates are available to be deployed onto the LB. Use OCI CLI to add the certs.
12. Install OCI CLI on the Certbot instance.
13. Use "oci lb certificate create" command to create the <private key, public key> in the LB.
14. Next, set up the "User Traffic" segment from the architecture diagram. For this do the following:
Once complete, the Listener list will look as follows:
15. Things should be functional at this time and user traffic to https://www.some-domain.com/* should be working fine. For example:
Let's Encrypt certificates are valid for 90 days and they recommend that the certs get renewed every 60 days. The following command can be used for renewal:
sudo certbot renew --deploy-hook ./oci-lb-cert-renewal.sh
The renew command first checks if certificate renewal is required. If required, it fetches new certificate from Let's Encrypt. And then the "deploy-hook" script gets executed. The script can be any arbitrary script. Here, the script oci-lb-cert-renewal.sh does the following:
The Certbot renew command can be added as a Cron job executing it every 60 days.
In this article, a solution to automate certificate renewal for OCI LBaaS has been discussed. The solution is based on Let's Encrypt and Certbot, both of which are free and open source.
Amit is a Solutions Architect focussing on Cloud Security including Identity, Governance, Network Security and Architecture. Amit advises customers and helps them design and implement security solutions on the Oracle Cloud Infrastructure platform. Amit advises different levels of customers from executives to architects and developers. Before joining Oracle, Amit worked in software engineering as an architect and developer working on mobile, security, cloud, web, internet and wireless technologies. With a strong background in software engineering and Computer Science, Amit brings a unique perspective into solving customer security needs in the cloud.