X

Best Practices from Oracle Development's A‑Team

Oracle Cloud Infrastructure (OCI) REST call walkthrough with curl

Introduction

This post will walkthrough the elements required, and purpose of each, to make a curl request to Oracle Cloud Infrastructure (OCI) REST services.

While there are a variety of tools offered that make this task far easier than manually constructing a curl request, the purpose of this post is to provide an explanation of the required elements to make an OCI REST request and hopefully give a deeper understanding to those interested - especially if you ever want to modify/construct your own tools or use something like postman.

 

Required steps before submitting requests

There are several setup steps required to be able to make REST calls to OCI. The detailed instructions are beyond the scope of this post as they are already covered in the OCI REST documentation. The high level steps are:

  • A user in your OCI tenancy that is allowed to make API calls
  • A RSA key pair in PEM format
  • Upload your public RSA PEM key to your OCI user
  • You will need to know the OCID of your tenancy, user, and compartment you want to work in at a minimum. Other OCID's may be required depending on which REST calls you make.

Detailed instructions on these steps can be found in the OCI documentation here:

https://docs.cloud.oracle.com/iaas/Content/API/Concepts/apisigningkey.htm

 

The curl request

Note that the curl command being shown in this post is structured for Linux. If you are trying to follow this post on Windows, you will need to modify escape sequences and possibly some commands.

 

The entire request

This is the entire curl request. We will walk through different parts of the request below.

curl -v -X GET -sS https://iaas.eu-frankfurt-1.oraclecloud.com/20160918/instances?compartmentId=ocid1.compartment.oc1..aaaaaaaaxxjdxgyjxqxoiiv7yyb2zgez4pxaubsee4oguejnggwzhokqzxxx -H "date: Tue, 22 Jan 2019 19:11:56 GMT" -H "Authorization: Signature version=\"1\",keyId=\"ocid1.tenancy.oc1..aaaaaaaawpqblfemtluwxipipubxhioptheej2r32gvf7em7iftkr3vd2xxx/ocid1.user.oc1..aaaaaaaadz34xoz4dxx2ix3tobcmfqtwf65jzesp5qfqjd24hykt46nphxxx/96:71:88:02:48:ab:4a:0c:25:10:ca:f0:b3:75:1e:df\",algorithm=\"rsa-sha256\",headers=\"(request-target) date host\",signature=\"AfM67Km9eon93LQqz4HfHn9YwVUhAcxj48ZIxkFGJe0J5CSVfHJ3hZWUouPEN5Q1G7Nujz/hdMK6sIdkIg7IkTuhDajR4JqjSvseHgjEOsNdhGcZo1dZu9xf215ULTy6eezi1veViXn3vlu87XcBv9kSfHSNNzw9gh3HygjBcB8kW4MBwIVaNnkkyRs4pH469kbhYPaZRuUsP+ErYK5bKptIcWamI/8eVmbfDzkQwi9ccepCFlz5q0MtnzWNS3sEWSro6WgpJrKrZhFJbAkBLPG7NpxAbczdW5C09Xtv/tph5n+Rc6XMlyqu/1DIOvlqapF4yldlGbYdRLmL/srJxw==\""

 

curl options

curl -v -X GET -sS https://iaas.eu-frankfurt-1.oraclecloud.com......

The -v option is to set verbose output, and is for testing your request. You do not need this beyond testing purposes.

The -X GET sets the request type - http GET

The -sS tells curl to be silent, but show errors. This is useful for all normal requests so you can see if anything went wrong.

 

rest endpoint

curl -v -X GET -sS https://iaas.eu-frankfurt-1.oraclecloud.com/20160918/instances?compartmentId=ocid1.compartment.oc1..aaaaaaaaxxjdxgyjxqxoiiv7yyb2zgez4pxaubsee4oguejnggwzhokqzxxx

The first section, highlighted in yellow, is the endpoint, API version, and API you want to call. For this example, I am making a request to the core services instances API.

The second section, highlighted in green, are the required parameters for the instance API. The instances API requires me to pass in the compartmentid in my tenancy I want to list instances from.

required headers and request signature

The remainder of the request are the required headers and signature to make OCI REST calls.

OCI REST calls require request signatures, and require specific header fields. The full request signing specification can be found here:

draft-cavage-http-signatures-08

And the details of OCI signing requirements can be found in the OCI documentation here:

OCI Request Signatures

The purpose of having the RSA key pair is to allow you to sign your requests. You are signing your request with your private key before making the REST call. The corresponding public key you uploaded to your OCI user is used to to verify the request is in fact coming from you.

Header date

Whether your request is a GET, DELETE, PUT or POST - all required the date header.

curl -v -X GET -sS https://iaas.eu-frankfurt-1.oraclecloud.com/20160918/instances?.... -H "date: Tue, 22 Jan 2019 19:11:56 GMT"

The date header is highlighted in yellow.

The date must be passed in with the correct HTTP header format.

It is also important the your local system time, and the remote OCI system time be within 5 minutes of each other. This 5 minute window is to allow for clock skew. Note you are passing your time with the timezone set to GMT.

An example on linux to get the date in correct format is:

date -u "+%a, %d %h %Y %H:%M:%S GMT"

Authorization Header and Signature Version

This is the longest portion of the curl request, and will be broken into pieces for purposes of this post.

curl -v -X GET -sS https://iaas.eu-frankfurt-1.oraclecloud.com/20160918/instances?... -H "date: Tue, 22 Jan 2019 19:11:56 GMT" -H "Authorization: Signature version=\"1\",keyId=\"...

The entire header being passed in the Authorization header. This is highlighted in yellow.

The first header field is the Signature Version, highlighted in green. Currently, this is set to 1 per the OCI documentation. Note there are escaped quotes around the 1. This is because you need to pass the quotes along with your request. Whether you are using curl, or another tool, make sure all values are wrapped in double quotes in the request header.

Key Identifier (keyId)

curl -v -X GET -sS https://iaas.eu-frankfurt-1.oraclecloud.com/20160918/instances?... -H "date:..." -H "Authorization: Signature version=\"1\",keyId=\"ocid1.tenancy.oc1..aaaaaaaawpqblfemtluwxipipubxhioptheej2r32gvf7em7iftkr3vd2xxx/ocid1.user.oc1..aaaaaaaadz34xoz4dxx2ix3tobcmfqtwf65jzesp5qfqjd24hykt46nphxxx/96:71:88:02:48:ab:4a:0c:25:10:ca:f0:b3:75:1e:df\"

The keyId is required for all request types. The format is keyId="<TENANCY OCID>/<USER OCID>/<KEY FINGERPRINT>"

The tenancy OCID is in yellow. The user OCID is in green. The key fingerprint is in blue. Note that each element is sperated by a /, and the entire value is encapsulated in double quotes.

The key fingerprint is the MD5 fingerprint of your public key.

To get the fingerprint, you can use the following command (also in the OCI REST docs):

openssl rsa -pubout -outform DER -in ~/.oci/oci_api_key.pem | openssl md5 -c

Signing algorithm

The signing algorithm must be rsa-sha256

curl -v -X GET -sS https://iaas.eu-frankfurt-1.oraclecloud.com/20160918/instances?... -H "date: Tue, 22 Jan 2019 19:11:56 GMT" -H "Authorization: Signature version=\"1\",keyId=\"...\",algorithm=\"rsa-sha256\"

The algorithm is highlighted in yellow.

Headers

This sections describes the headers used for the signing string, which will be covered next.

You need to define the headers that will be included in your signing string. What headers are required depends on your request type (i.e. GET, PUT)

For a GET request, we need:

  • (request-target)
  • host
  • date

The headers required for each request type are described in the OCI REST documentation at the following link, in the required headers section:

https://docs.cloud.oracle.com/iaas/Content/API/Concepts/signingrequests.htm

curl -v -X GET -sS https://iaas.eu-frankfurt-1.oraclecloud.com/20160918/instances?... -H "date: Tue, 22 Jan 2019 19:11:56 GMT" -H "Authorization: Signature version=\"1\",keyId=\"...\",algorithm=\"rsa-sha256\",headers=\"(request-target) date host\"

The header field is highlighted in yellow. Note the required headers are encapsulated in quotes, and separated by spaces.

 

Signature

The final field is the signature, created from your signing string.

curl -v -X GET -sS https://iaas.eu-frankfurt-1.oraclecloud.com/20160918/instances?... -H "date: Tue, 22 Jan 2019 19:11:56 GMT" -H "Authorization: Signature version=\"1\",keyId=\"...",algorithm=\"rsa-sha256\",headers=\"(request-target) date host\",signature=\"AfM67Km9eon93LQqz4HfHn9YwVUhAcxj48ZIxkFGJe0J5CSVfHJ3hZWUouPEN5Q1G7Nujz/hdMK6sIdkIg7IkTuhDajR4JqjSvseHgjEOsNdhGcZo1dZu9xf215ULTy6eezi1veViXn3vlu87XcBv9kSfHSNNzw9gh3HygjBcB8kW4MBwIVaNnkkyRs4pH469kbhYPaZRuUsP+ErYK5bKptIcWamI/8eVmbfDzkQwi9ccepCFlz5q0MtnzWNS3sEWSro6WgpJrKrZhFJbAkBLPG7NpxAbczdW5C09Xtv/tph5n+Rc6XMlyqu/1DIOvlqapF4yldlGbYdRLmL/srJxw==\""

The signature is highlighted in yellow.

To create a signature, you first construct a signing string. What goes into the signing string is determined by the values required for the header field in the previous step, which is determined by request type (GET vs PUT)

For a GET request, as in this example, our signing string must contain the (request-target), date and host.

(request-target) structure is defined in the signing specification. For our GET request, it is the request type and the API call being made. It looks as follows:

(request-target): get /20160918/instances?compartmentId=ocid1.compartment.oc1..aaaaaaaaxxjdxgyjxqxoiiv7yyb2zgez4pxaubsee4oguejnggwzhokqzxxx

The date is the date in the correct format as described previously in this post.

date: Tue, 22 Jan 2019 19:11:56 GMT

The host is the endpoint you are sending your request to.

host: iaas.eu-frankfurt-1.oraclecloud.com

Per the signing specification, you include all of this in a single signing string, but each must be separated by a new line.

So the complete signing string, with new line sequences looks as follows:

(request-target): get /20160918/instances?compartmentId=ocid1.compartment.oc1..aaaaaaaaxxjdxgyjxqxoiiv7yyb2zgez4pxaubsee4oguejnggwzhokqzxxx\ndate: Tue, 22 Jan 2019 19:11:56 GMT\nhost: iaas.eu-frankfurt-1.oraclecloud.com

Note the order of the elements in the signing string matters. They must be in the same order that you specified in the header value. (request-target date host)

The next steps is to sign this string with your private RSA key, and convert it to BASE64 format. The resulting value is the string of characters you see in the signature section of our curl request.

Sign the string by passing it to the following command

openssl dgst -sha256 -sign ~/.oci/oci_api_key.pem | openssl enc -e -base64 | tr -d '\n'

This command assumes you used the key name and path used in the OCI REST documentation.

The first part of the command signs the string using sha256. This is the yellow portion.

The second part converts it to base64. This is the green section.

The third part removes new lines so the output is one continuous string. This is the blue portion.

Note that you want to sign and convert to base64 in one step if you are doing this in a shell. The output from signing is binary, and will likely not be copyable from the command line.

When passing the signing string in to be signed, be careful of how you do it. For example, using printf vs echo. Some commands will manipulate spaces, escapes and new lines in unexpected ways, which will result in an invalid signing string.

The following is a complete command that works properly.

printf "(request-target): get /20160918/instances?compartmentId=ocid1.compartment.oc1..aaaaaaaaxxjdxgyjxqxoiiv7yyb2zgez4pxaubsee4oguejnggwzhokqzxxx\ndate: Tue, 22 Jan 2019 19:11:56 GMT\nhost: iaas.eu-frankfurt-1.oraclecloud.com" | openssl dgst -sha256 -sign ~/.oci/oci_api_key.pem | openssl enc -e -base64 | tr -d '\n'

 

Debugging

The purpose of using the -v option with curl is to get some verbose output to debug with if needed.

The following are correctly formatted headers that curl -v returns with my request used in this post.

 

GET /20160918/instances?compartmentId=ocid1.compartment.oc1..aaaaaaaaxxjdxgyjxqxoiiv7yyb2zgez4pxaubsee4oguejnggwzhokqzxxx HTTP/1.1
> User-Agent: curl/7.29.0
> Host: iaas.eu-frankfurt-1.oraclecloud.com
> Accept: */*
> date: Tue, 22 Jan 2019 19:11:56 GMT
> Authorization: Signature version="1",keyId="ocid1.tenancy.oc1..aaaaaaaawpqblfemtluwxipipubxhioptheej2r32gvf7em7iftkr3vd2xxx/ocid1.user.oc1..aaaaaaaadz34xoz4dxx2ix3tobcmfqtwf65jzesp5qfqjd24hykt46nphxxx/96:71:88:02:48:ab:4a:0c:25:10:ca:f0:b3:75:1e:df",algorithm="rsa-sha256",headers="(request-target) date host",signature="AfM67Km9eon93LQqz4HfHn9YwVUhAcxj48ZIxkFGJe0J5CSVfHJ3hZWUouPEN5Q1G7Nujz/hdMK6sIdkIg7IkTuhDajR4JqjSvseHgjEOsNdhGcZo1dZu9xf215ULTy6eezi1veViXn3vlu87XcBv9kSfHSNNzw9gh3HygjBcB8kW4MBwIVaNnkkyRs4pH469kbhYPaZRuUsP+ErYK5bKptIcWamI/8eVmbfDzkQwi9ccepCFlz5q0MtnzWNS3sEWSro6WgpJrKrZhFJbAkBLPG7NpxAbczdW5C09Xtv/tph5n+Rc6XMlyqu/1DIOvlqapF4yldlGbYdRLmL/srJxw=="

With your own requests, confirm the Host, date and Authorization sections are correct.

Make sure all of the values in the Authorization section are enclosed in double quotes.

 

A further debugging test can be done using the sample keys and signing string in the OCI REST documentation.

Look at the test values section on this page: https://docs.cloud.oracle.com/iaas/Content/API/Concepts/signingrequests.htm

Here is a simple bash script using these test values. You will need to save the public and private pem keys locally, and adjust the private key path if needed.

#!/bin/bash
	privateKeyPath="/home/devuser/.oci/oci_test_key.pem";
	date_header="date: Thu, 05 Jan 2014 21:31:40 GMT"
	host_header="host: iaas.us-phoenix-1.oraclecloud.com"
	request_target="(request-target): get /20160918/instances?availabilityDomain=Pjwf%3A%20PHX-AD-1&compartmentId=ocid1.compartment.oc1..aaaaaaaam3we6vgnherjq5q2idnccdflvjsnog7mlr6rtdb25gilchfeyjxa&displayName=TeamXInstances&volumeId=ocid1.volume.oc1.phx.abyhqljrgvttnlx73nmrwfaux7kcvzfs3s66izvxf2h4lgvyndsdsnoiwr5q"
	signing_string="$request_target\n$date_header\n$host_header"
	printf '%b' "signing string is $signing_string \n"
	printf '%b' "signed request is \n"
	printf '%b' "$signing_string" | openssl dgst -sha256 -sign $privateKeyPath | openssl enc -e -base64 | tr -d '\n'
	printf "\n\n"

 

This test uses the keys and values in the OCI documentation. Your resulting signing string should be:

GBas7grhyrhSKHP6AVIj/h5/Vp8bd/peM79H9Wv8kjoaCivujVXlpbKLjMPeDUhxkFIWtTtLBj3sUzaFj34XE6YZAHc9r2DmE4pMwOAy/kiITcZxa1oHPOeRheC0jP2dqbTll8fmTZVwKZOKHYPtrLJIJQHJjNvxFWeHQjMaR7M=

You can use this same process and sample script to test your own values.

 

 

Join the discussion

Comments ( 1 )
  • Scott Wednesday, May 29, 2019
    Michael,

    Great article!

    Quick question - is there a way to sign the requests via PL/SQL so that we can call the APIs from the database?

    Thanks,

    - Scott -
Please enter your name.Please provide a valid email address.Please enter a comment.CAPTCHA challenge response provided was incorrect. Please try again.Captcha