Automatic Certificate Renewal in OCI Load Balancer

October 18, 2019 | 6 minute read
Text Size 100%:

Securing sites in the cloud with TLS/SSL has become a mandatory requirement.  However, certificate management – certificate issuance, revocation, etc. – is non trivial even in corporate environments.  And costs along with automation of certificate management act as further impediments in its adoption.  Internet Security Research Group, a non profit organization, operates a free Certificate Authority (CA) Let’s Encrypt to address these challenges.  Let's Encrypt supports a standard protocol ACME to access its certificate management features.  This promotes automation.

For customers requiring a free and open source certificate management solution in OCI LBaaS, the A-Team (Security) has in place a solution for automatic renewal of certificates issued by Let's Encrypt and uploading these certificates onto OCI Load Balancer.  The solution leverages the free, open source tool Certbot to interact with Let's Encrypt.  Certbot and Let's Encrypt use ACME protocol where Certbot is the ACME client and Let's Encrypt is the ACME server (CA).  Although Certbot can interact with any standard ACME CA server, it supports Let's Encrypt by default, i.e., out-of-the-box support.

Before issuing a certificate to an ACME client, the ACME server needs to get an assurance that the client indeed controls the domain, e.g., www.some-domain.com, for which the client is requesting the certificate. The ACME protocol specifies two types of challenges – DNS and HTTP.  An ACME client can use either type to prove that it controls the domain.  This blog post uses the HTTP challenge in the Certbot configuration.

Architecture

The architecture is as follows -

 

At a high level, the flow is as follows:

  1. Install Certbot on a Linux system.
  2. Run Certbot to fetch certificate for the domain from Let's Encrypt.
  3. Update the LBaaS with this certificate (from step 2).
  4. Repeat the above steps every 60 days to renew the certs.

Once certificates are installed on the LB, users can access the domain securely using HTTPS.  Most modern browsers support certificates issued by Let's Encrypt.  https://letsencrypt.org/docs/certificate-compatibility/ lists browser compatibilities.

OCI Configuration

The detailed steps are as follows:

  1. An org’s domain some-domain.com has been registered.
  2. Deploy a public OCI LB
  3. Add a "A Record" for “www.some-domain.com” with the LB’s public IP address as the RDATA to the registered domain's (some-domain.com) DNS Zone in OCI and publish it.

 

 

4.  Deploy a Linux OCI Compute instance.

5.  Install Certbot.

6.  Create a Backend Set with the Certbot instance.

7.  Update the Health Check for the above Backend Set with TCP/22.  The rationale for this setting will be explained later.

 

8.  Create a Path Route Set as below with the route rules directing traffic to the Backend Set of step 6.


9. Create a LB Listener with HTTP/80 along with the Backend Set from step 6.  Path Route from step 8 is used here.

The net effect of steps 8 and 9 above is that the request http://www.some-domain.com/.well-known/acme-challenge will be forwarded to the Certbot instance.  This is part of the ACME HTTP challenge mechanism stated earlier.  Let's Encrypt implements the ACME HTTP challenge by using the path /.well-known/acme-challenge .

Note: The Listener port is 80: the ACME HTTP challenge uses port 80.  If port 80 cannot be used due to corporate policies (firewall), etc., then this challenge mechanism cannot work.  Instead, DNS challenge needs to be used in that case (which is outside the scope of this blog).  However, this article can alleviate any concerns about keeping port 80 open.

 

10.  Run the following Certbot command to initiate handshake with Let’s Encrypt

sudo certbot certonly --standalone --register-unsafely-without-email

Certbot is an ACME client.  It interacts with a CA for certificate management.  After the CA issues the certificate, Certbot typically deploys the certificate on a web server, e.g. Apache, NGINX.  Since in this case the ultimate destination is the LB, no web server is involved.  However, the web server functionality is required to serve the request http://www.some-domain.com/.well-known/acme-challenge coming from Let's Encrypt.  The "standalone" flag is used precisely for that:  it instructs Certbot to start a standalone web server for Let's Encrypt interaction.

In step 7 above, TCP/22 is used for the health check policy.  Why TCP/22?  Since this "standalone" web server has limited purpose, Certbot only starts it when required for Let's Encrypt interaction and shuts it down later.  If OCI LB uses HTTP/80 for health check policy, then the health check polls will fail, marking the (Certbot) instance unavailable. This will break the functionality.  Since OCI Compute instances have ssh enabled by default, TCP/22 is used for health check as illustrated in step 7.

The output of the "certbot certonly" command is the following:

Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator standalone, Installer None
Starting new HTTPS connection (1): acme-v02.api.letsencrypt.org
Please enter in your domain name(s) (comma and/or space separated)  (Enter 'c'
to cancel): www.some-domain.com

After entering the domain name, if everything goes well, following is the output:

Obtaining a new certificate
Performing the following challenges:
http-01 challenge for www.some-domain.com
Waiting for verification...
Cleaning up challenges
IMPORTANT NOTES:
 - Congratulations! Your certificate and chain have been saved at:
   /etc/letsencrypt/live/www.some-domain.com/fullchain.pem
   Your key file has been saved at:
   /etc/letsencrypt/live/www.some-domain.com/privkey.pem
   Your cert will expire on 2019-11-26. To obtain a new or tweaked
   version of this certificate in the future, simply run certbot
   again. To non-interactively renew *all* of your certificates, run
   "certbot renew"

If everything went well, then that would confirm that the loop Certbot → Let's Encrypt → OCI LB → Certbot ( steps 1 & 2 in the above architecture diagram) is complete and working.  

11.  At this stage, the certificates are available to be deployed onto the LB.  Use OCI CLI to add the certs.

12.  Install OCI CLI on the Certbot instance.

13.  Use "oci lb certificate create" command to create the <private key, public key> in the LB.

14.  Next, set up the "User Traffic" segment from the architecture diagram.  For this do the following:

  • Instantiate a Compute instance and deploy an Apache Web Server on it
  • Create a Backend Set with the Apache instance
  • Create a new Listener with HTTP/443 and using the above Backend Set and the cert bundle (step 13)

 

Once complete, the Listener list will look as follows:

 

15.  Things should be functional at this time and user traffic to https://www.some-domain.com/* should be working fine.  For example:

 

Let's Encrypt certificates are valid for 90 days and they recommend that the certs get renewed every 60 days.  The following command can be used for renewal:

  sudo certbot renew --deploy-hook ./oci-lb-cert-renewal.sh

The renew command first checks if certificate renewal is required.  If required, it fetches new certificate from Let's Encrypt.  And then the "deploy-hook" script gets executed.  The script can be any arbitrary script.  Here, the script oci-lb-cert-renewal.sh does the following:

  1. Creates a cert bundle with the newly procured certificate from Let's Encrypt at the LB.
  2. Updates the Listener (oci lb listener update) with this bundle.
  3. Deletes (clean-up) any old certificate bundle from the LB (oci lb certificate delete).

The Certbot renew command can be added as a Cron job executing it every 60 days.

Summary

In this article, a solution to automate certificate renewal for OCI LBaaS has been discussed.  The solution is based on Let's Encrypt and Certbot, both of which are free and open source. 

Amit Chakraborty

Amit is a Solutions Architect focussing on Cloud Security including Identity, Governance, Network Security and Architecture. Amit advises customers and helps them design and implement security solutions on the Oracle Cloud Infrastructure platform. Amit advises different levels of customers from executives to architects and developers. Before joining Oracle, Amit worked in software engineering as an architect and developer working on mobile, security, cloud, web, internet and wireless technologies. With a strong background in software engineering and Computer Science, Amit brings a unique perspective into solving customer security needs in the cloud.


Previous Post

Create a VPN-RA Server on OCI with Libreswan

Catalin Andrei | 5 min read

Next Post


Installing Oracle Analytics Cloud Remote Data Gateway

Dayne Carley | 5 min read