Introduction

This post will walkthrough the elements required, and purpose of each, to make a curl request to Oracle Cloud Infrastructure (OCI) REST services.

While there are a variety of tools offered that make this task far easier than manually constructing a curl request, the purpose of this post is to provide an explanation of the required elements to make an OCI REST request and hopefully give a deeper understanding to those interested - especially if you ever want to modify/construct your own tools or use something like postman.

If you are interested in POSTMAN specific information on OCI REST calls, check out my colleague Ashish Singh's article, here - https://www.ateam-oracle.com/invoking-oci-rest-apis-using-postman

Required steps before submitting requests

There are several setup steps required to be able to make REST calls to OCI. The detailed instructions are beyond the scope of this post as they are already covered in the OCI REST documentation. The high level steps are:

• A user in your OCI tenancy that is allowed to make API calls
• A RSA key pair in PEM format
• Upload your public RSA PEM key to your OCI user
• You will need to know the OCID of your tenancy, user, and compartment you want to work in at a minimum. Other OCID's may be required depending on which REST calls you make.

Detailed instructions on these steps can be found in the OCI documentation here:

https://docs.cloud.oracle.com/iaas/Content/API/Concepts/apisigningkey.htm

The curl request

Note that the curl command being shown in this post is structured for Linux. If you are trying to follow this post on Windows, you will need to modify escape sequences and possibly some commands.

The entire request

This is the entire curl request. We will walk through different parts of the request below.

curl options

The -v option is to set verbose output, and is for testing your request. You do not need this beyond testing purposes.

The -X GET sets the request type - http GET

The -sS tells curl to be silent, but show errors. This is useful for all normal requests so you can see if anything went wrong.

rest endpoint

The first section, highlighted in yellow, is the endpoint, API version, and API you want to call. For this example, I am making a request to the core services instances API.

The second section, highlighted in green, are the required parameters for the instance API. The instances API requires me to pass in the compartmentid in my tenancy I want to list instances from.

required headers and request signature

The remainder of the request are the required headers and signature to make OCI REST calls.

OCI REST calls require request signatures, and require specific header fields. The full request signing specification can be found here:

draft-cavage-http-signatures-08

And the details of OCI signing requirements can be found in the OCI documentation here:

OCI Request Signatures

The purpose of having the RSA key pair is to allow you to sign your requests. You are signing your request with your private key before making the REST call. The corresponding public key you uploaded to your OCI user is used to to verify the request is in fact coming from you.

Header date

Whether your request is a GET, DELETE, PUT or POST - all required the date header.

The date header is highlighted in yellow.

The date must be passed in with the correct HTTP header format.

It is also important the your local system time, and the remote OCI system time be within 5 minutes of each other. This 5 minute window is to allow for clock skew. Note you are passing your time with the timezone set to GMT.

An example on linux to get the date in correct format is:

date -u "+%a, %d %h %Y %H:%M:%S GMT"

Authorization Header and Signature Version

This is the longest portion of the curl request, and will be broken into pieces for purposes of this post.

The entire header being passed in the Authorization header. This is highlighted in yellow.

The first header field is the Signature Version, highlighted in green. Currently, this is set to 1 per the OCI documentation. Note there are escaped quotes around the 1. This is because you need to pass the quotes along with your request. Whether you are using curl, or another tool, make sure all values are wrapped in double quotes in the request header.

Key Identifier (keyId)

The keyId is required for all request types. The format is keyId="<TENANCY OCID>/<USER OCID>/<KEY FINGERPRINT>"

The tenancy OCID is in yellow. The user OCID is in green. The key fingerprint is in blue. Note that each element is sperated by a /, and the entire value is encapsulated in double quotes.

The key fingerprint is the MD5 fingerprint of your public key.

To get the fingerprint, you can use the following command (also in the OCI REST docs):

openssl rsa -pubout -outform DER -in ~/.oci/oci_api_key.pem | openssl md5 -c

Signing algorithm

The signing algorithm must be rsa-sha256

The algorithm is highlighted in yellow.

Headers

This sections describes the headers used for the signing string, which will be covered next.

You need to define the headers that will be included in your signing string. What headers are required depends on your request type (i.e. GET, PUT)

For a GET request, we need:

• (request-target)
• host
• date

The headers required for each request type are described in the OCI REST documentation at the following link, in the required headers section:

https://docs.cloud.oracle.com/iaas/Content/API/Concepts/signingrequests.htm

The header field is highlighted in yellow. Note the required headers are encapsulated in quotes, and separated by spaces.

Signature

The final field is the signature, created from your signing string.

The signature is highlighted in yellow.

To create a signature, you first construct a signing string. What goes into the signing string is determined by the values required for the header field in the previous step, which is determined by request type (GET vs PUT)

For a GET request, as in this example, our signing string must contain the (request-target), date and host.

(request-target) structure is defined in the signing specification. For our GET request, it is the request type and the API call being made. It looks as follows:

The date is the date in the correct format as described previously in this post.

The host is the endpoint you are sending your request to.

Per the signing specification, you include all of this in a single signing string, but each must be separated by a new line.

So the complete signing string, with new line sequences looks as follows:

Note the order of the elements in the signing string matters. They must be in the same order that you specified in the header value. (request-target date host)

The next steps is to sign this string with your private RSA key, and convert it to BASE64 format. The resulting value is the string of characters you see in the signature section of our curl request.

Sign the string by passing it to the following command

This command assumes you used the key name and path used in the OCI REST documentation.

The first part of the command signs the string using sha256. This is the yellow portion.

The second part converts it to base64. This is the green section.

The third part removes new lines so the output is one continuous string. This is the blue portion.

Note that you want to sign and convert to base64 in one step if you are doing this in a shell. The output from signing is binary, and will likely not be copyable from the command line.

When passing the signing string in to be signed, be careful of how you do it. For example, using printf vs echo. Some commands will manipulate spaces, escapes and new lines in unexpected ways, which will result in an invalid signing string.

The following is a complete command that works properly.

Samples

GET request sample

The following is a sample GET request to retrieve all the compute instances in a compartment.

Update the fields in the fill in your values block before executing.

#!/bin/bash


########################## Fill these in with your values ##########################
#OCID of the tenancy calls are being made in to
tenancy_ocid="ocid1.tenancy.oc1.."

# OCID of the user making the rest call
user_ocid="ocid1.user.oc1.."

# path to the private PEM format key for this user
privateKeyPath="/home/sampleuser/.oci/oci_api_key.pem"

# fingerprint of the private key for this user
fingerprint="99:aa:....."

# The REST api you want to call, with any required paramters.
rest_api="/20160918/instances?compartmentId=ocid1.compartment.oc1.."

# The host you want to make the call against
host="iaas.us-ashburn-1.oraclecloud..."
####################################################################################


date=`date -u "+%a, %d %h %Y %H:%M:%S GMT"`
date_header="date: $date"
host_header="host: $host"
request_target="(request-target): get $rest_api"
# note the order of items. The order in the signing_string matches the order in the headers
signing_string="$request_target\n$date_header\n$host_header"
headers="(request-target) date host"


echo "====================================================================================================="
printf '%b' "signing string is $signing_string \n"
signature=`printf '%b' "$signing_string" | openssl dgst -sha256 -sign $privateKeyPath | openssl enc -e -base64 | tr -d '\n'`
printf '%b' "Signed Request is  \n$signature\n"

echo "====================================================================================================="
set -x
curl -v -X GET -sS https://$host$rest_api -H "date: $date" -H "Authorization: Signature version=\"1\",keyId=\"$tenancy_ocid/$user_ocid/$fingerprint\",algorithm=\"rsa-sha256\",headers=\"$headers\",signature=\"$signature\"" 

POST request sample

The following is a POST request example to create a new block volume.

First, create a JSON file containing the data you want to POST. Save the following as request.json, in the same directory as the script that calls the curl POST. Adjust the values as needed. You will need to at least put in your compartmentID.

{"availabilityDomain": "ldue:US-ASHBURN-AD-1", "compartmentId": "ocid1.compartment.oc1..unique_id", "displayName": "testvolume", "sizeInGBs": 50}

Update the fields in the fill in your values block before executing.

#!/bin/bash

########################## Fill these in with your values ##########################
#OCID of the tenancy calls are being made in to
tenancy_ocid="ocid1.tenancy.oc1.."

# OCID of the user making the rest call
user_ocid="ocid1.user.oc1.."

# path to the private PEM format key for this user
privateKeyPath="/home/sampleuser/.oci/oci_api_key.pem"

# fingerprint of the private key for this user
fingerprint="99:aa:....."

# The REST api you want to call, with any required paramters.
rest_api="/20160918/volumes"

# The host you want to make the call against
host="iaas.us-ashburn-1.oraclecloud..."

# the json file containing the data you want to POST to the rest endpoint
body="./request.json"
####################################################################################


# extra headers required for a POST/PUT request
body_arg=(--data-binary @${body})
content_sha256="$(openssl dgst -binary -sha256 < $body | openssl enc -e -base64)";
content_sha256_header="x-content-sha256: $content_sha256"
content_length="$(wc -c < $body | xargs)";
content_length_header="content-length: $content_length"
headers="(request-target) date host"
# add on the extra fields required for a POST/PUT
headers=$headers" x-content-sha256 content-type content-length"
content_type_header="content-type: application/json";

date=`date -u "+%a, %d %h %Y %H:%M:%S GMT"`
date_header="date: $date"
host_header="host: $host"
request_target="(request-target): post $rest_api"

# note the order of items. The order in the signing_string matches the order in the headers, including the extra POST fields
signing_string="$request_target\n$date_header\n$host_header"
# add on the extra fields required for a POST/PUT
signing_string="$signing_string\n$content_sha256_header\n$content_type_header\n$content_length_header"




echo "====================================================================================================="
printf '%b' "signing string is $signing_string \n"
signature=`printf '%b' "$signing_string" | openssl dgst -sha256 -sign $privateKeyPath | openssl enc -e -base64 | tr -d '\n'`
printf '%b' "Signed Request is  \n$signature\n"

echo "====================================================================================================="
set -x
curl -v -X POST --data-binary "@request.json" -sS https://$host$rest_api -H "date: $date" -H "x-content-sha256: $content_sha256" -H "content-type: application/json" -H "content-length: $content_length" -H "Authorization: Signature version=\"1\",keyId=\"$tenancy_ocid/$user_ocid/$fingerprint\",algorithm=\"rsa-sha256\",headers=\"$headers\",signature=\"$signature\"" 

Debugging

The purpose of using the -v option with curl is to get some verbose output to debug with if needed.

The following are correctly formatted headers that curl -v returns with my request used in this post.

With your own requests, confirm the Host, date and Authorization sections are correct.

Make sure all of the values in the Authorization section are enclosed in double quotes.

A further debugging test can be done using the sample keys and signing string in the OCI REST documentation.

Look at the test values section on this page: https://docs.cloud.oracle.com/iaas/Content/API/Concepts/signingrequests.htm

Here is a simple bash script using these test values. You will need to save the public and private pem keys locally, and adjust the private key path if needed.

#!/bin/bash
	privateKeyPath="/home/devuser/.oci/oci_test_key.pem";
	date_header="date: Thu, 05 Jan 2014 21:31:40 GMT"
	host_header="host: iaas.us-phoenix-1.oraclecloud.com"
	request_target="(request-target): get /20160918/instances?availabilityDomain=Pjwf%3A%20PHX-AD-1&compartmentId=ocid1.compartment.oc1..aaaaaaaam3we6vgnherjq5q2idnccdflvjsnog7mlr6rtdb25gilchfeyjxa&displayName=TeamXInstances&volumeId=ocid1.volume.oc1.phx.abyhqljrgvttnlx73nmrwfaux7kcvzfs3s66izvxf2h4lgvyndsdsnoiwr5q"
	signing_string="$request_target\n$date_header\n$host_header"
	printf '%b' "signing string is $signing_string \n"
	printf '%b' "signed request is \n"
	printf '%b' "$signing_string" | openssl dgst -sha256 -sign $privateKeyPath | openssl enc -e -base64 | tr -d '\n'
	printf "\n\n"

This test uses the keys and values in the OCI documentation. Your resulting signing string should be:

You can use this same process and sample script to test your own values.