Part 2: Using Postman Runner with Identity Cloud Service

September 14, 2018 | 11 minute read
Tim Melander
A-Team Cloud Solution Architect
Text Size 100%:

Introduction

In the last article, Part 1: Automate getting an Identity Cloud Service Access Token in Postman, I explained how using a Pre-request Script can automate getting an access token so when a request is sent to Identity Cloud Service it just works, a big time saver. In this article I am going build on using that Pre-request script and add Tests scripts in a way that we can chain requests in a collection and automate all the requests at once using Postman Collection Runner. Spoiler alert, in my final Part 3: Automate Requests against Identity Cloud Service using Newman, I will incorporate Part 1 and 2 to show how we can use everything we learned with Newman to iterate users from a CSV file and execute a bulk request from a command-line. Fantastic, let’s get started.

 

What’s Runner Collection?

Before I explain what, a Collection Runner is, let's backup a bit and make sure we are on the same page and explain what a Collection is in Postman. A Collection is a bucket if you will that has one or more requests. In the graphic below you see the Collection named Acme IDCS Collection, this is a Collection, and within that Collection are a couple requests, but it could be several.

Now going back to, "What's a Collection Runner?", it was built to automate the testing of APIs. In our case we are going to use it to automate a Collection of requests against Identity Cloud Service. When you launch Collection Runner you can run a Collection as is, or use a CSV (Comma Separated Value) file and iterate through the data.  As you will see we can grab data and dynamically apply the values from each column of each row to use in the requests within our collection. At the time of this article Identity Cloud Service has Bulk APIs https://docs.oracle.com/en/cloud/paas/identity-cloud/rest-api/api-bulk-bulk.html that only support create users, password changes, and password resets. That is why using Collection Runner can be a very powerful tool to either test REST APIs in various ways, or process bulk operations which is where I want to take you.

 

Create a New Collection and Environment

Before we get started with Collection Runner we need a Collection.  To keep things simple and since there are already a lot of articles on how to create a collection, I am going to speed things up and have you import my sample called Acme IDCS Collection, which includes our example Collection and Environment with variables. Be sure to follow all the following steps because I will be including details on what to change in the Environment variables for it to work with your tenant, if you miss that it will not work.

Import and Modify Acme IDCS Collection and Environment:

1. Get the sample Acme_IDCS_Collection.zip that includes a collection, environment, and user data by downloading it from here.

2. Unzip the Acme_IDCS_Collection.zip which should create a folder named “Acme IDCS Collection” with three files inside; i.e. Acme IDCS Collection.postman_collection.json, Acme IDCS Environment.postman_environment.json, and test_users.csv.

3. Import the sample collection into Postman by selecting File from the main menu, next Import…, next select the Choose Files button from the Import File, next navigate to the Acme IDCS Collection folder, and finally select Acme IDCS Collection.postman_collection.json.

4. Import the sample environment into Postman the same way by selecting File from the main menu, next Import…, next select the Choose Files button from the Import File, next navigate to the Acme IDCS Collection folder, and finally select Acme IDCS Environment.postman_environment.json.

5. There is no need to modify the Collection for our demo, but we do need to Modify the environment by selecting the Gear icon from the top right corner that says Managed Environments, next select the Acme IDCS Environment.

6. There are three variables HOST, CLIENT_ID, and CLIENT_SECRET, modify the values per your IDCS tenant. The CLIENT_ID and CLIENT_SECRET values should come from a registered IDCS Trusted Application that minimally has Client Credential grant types. If you are not sure what I am talking about see this article “Identity Cloud Services OAuth 2.0 and REST API”. Note that the HOST variable should have the FQDN of your IDCS tenant including the leading https:// value.

7. Finally click the Update button in the Environment.

We should now have a new collection and environment to use. Now let’s move on and look at the requests and scripts inside the collection.

Using Pre-request Scripts

As a primer, a Pre-request script is something we can add to a request or collection that will be executed first before a request is sent.  We learned about the Pre-request Script example in Part 1: Automate getting an IDCS Access Token in Postman that automated getting the Access Token. In this section I will continue to use the same Pre-request script, but expand on it to show how we can get data from a CSV file to send each request in the collection repeatedly for each data row in the CSV. In Part 1: Automate getting an IDCS Access Token in Postman the Pre-request script used to get the Access Token was placed in the main collection, but in this model I am moving it to the first request in the Acme IDCS Collection. If you look at the first request, and select the Pre-request Script tab there is already a script as follows.

Pre-request script – POST: Get userid by searching username

// Set some environment variables to use in the access token request. var client_id = pm.environment.get("CLIENT_ID"); var client_secret = pm.environment.get("CLIENT_SECRET"); var tenant = pm.environment.get("HOST") var CryptoJS = require("crypto-js") // Encrypt the Client ID and Secret in base64 var rawStr = CryptoJS.enc.Utf8.parse(client_id + ':' + client_secret) var client_cred = CryptoJS.enc.Base64.stringify(rawStr) // Now let's get a new oauth2 access_token and set it in a variable pm.sendRequest({     url: tenant + '/oauth2/v1/token',     method: 'POST',     header: {         'Authorization': 'Basic ' + client_cred,         'Cache-Control': 'no-cache',         'Content-Type': 'application/x-www-form-urlencoded'         },       body: {           mode: 'urlencoded',           urlencoded: [               {key: "grant_type", value: "client_credentials"},               {key: "scope", value: "urn:opc:idm:__myscopes__"},               ]       } },  function(err, response) {   const jsonResponse = response.json();   pm.environment.set("access_token", jsonResponse.access_token); }); // Define Single Value variables based on CSV file pm.environment.set("username", data["username"]); pm.environment.set("givenname", data["givenname"]); pm.environment.set("phonenumber", data["phonenumber"]); // Define MVA (Multi-Value Attributes) variables based on CSV file; e.g. // change From "app1,app2,app3" To "app1","app2","app3" var approles = '\"' + data["appRoles"].split(',').join('\",\"') + '\"'; pm.environment.set("approles", approles);

What has been expanded to our Pre-request script from the Part 1: Automate getting an IDCS Access Token in Postman is a section called "Define Single Value variables based on CSV file, and Define MVA (Multi-Value Attributes) variables based on CSV file.  The first new section, Define Single Value variables based on CSV file, is basically using pm.environment.set to set a variable name with data that will be extracted from a CSV file.  For example "data["username"]" maps to the column name "username" in the CSV file, along with givenname, and phonenumber.  When we get to Collection Runner, you will see how we can iterate through a CSV file and populate data to an environment variable that will be used in a request.

The second new section, Define MVA (Multi-Value Attributes) variables based on CSV file, in the Pre-request script is similar as the first, but in this case it has been designed to work with a comma delimited list of values in the CSV so that we can update a MVA or Multi-Value Attribute in the Identity Cloud Service. For example If you were to create a custom schema that uses an Array, that is a MVA, this script will look at the column “appRoles” in our example CSV. As long as the data in the CSV is formatted as “app_X,app_Y,app_Z”, the script will format the data properly so that you can add or replace a MVA attribute on a user.  If you are not going to use a MVA, then please comment this section out along with the appRole node in the JSON Body coming up in the next section.

 

Body – POST: Get userid by searching username using

In the same first request, select the Body tab. You will see the following JSON body. Note that I have added a variable “”, which will dynamically be populated by the environment variable "username" created in the Pre-request script with the data assigned to the variable that is extracted from the CSV file using “pm.environment.set("username", data["username"]);”.  I am sure this is now starting to come together and once you get the concept you can expand on this example.

{   "schemas": ["urn:ietf:params:scim:api:messages:2.0:SearchRequest"],   "attributes": ["id"],   "filter":           "userName eq \"\"",   "startIndex": 1,   "count": 10 }

In the test_users.csv file the first row with user data is username “marvel.eskridge@acme.com”, which when the Postman Collection Runner runs the first request will post a search for username “marvel.eskridge@acme.com”. The reason we are first searching for a username is because in our second request we are using a PATCH to modify that user.  In order to make a modification to a user we first need the user id. The user id is an internal value, so normally you don’t have this value, so it makes sense in order to get the ID we can search by something we know, and once we get the "id" we can use it.  As a side note in the Body request you can see “attributes: [“id”]”, this is tell our request what attributes we only want returned.  In our case we only care about getting the “id” we need to modify the user. Technically if we leave this line out of our request we will get all the attributes returned about the user including the "id", but to save on the payload response it makes sense to just ask for the "id".

 

Using Tests Scripts

Tests script, is a script that runs during the response of the request.  In my opinion the name is confusing because it is more of a Post-request script since it is being executed during the response, but I digress. Select the Tests tab in our POST Get userid by searching Username request and you will see the following script.

Tests script - POST: Get userid by searching username

// Test for a status of 200 tests["Status code is 200"] = (responseCode.code === 200); //  iteration # var n = pm.info.iteration + 1; var total = pm.info.iterationCount; if (responseCode.code === 200) {     // Get userid from JSON response and set in environment variable     var jsonData;     try {         jsonData = JSON.parse(responseBody);     } catch (err) {         console.log(err);     }     var userName = pm.environment.get("username");     // Determine if user id is found     var results = jsonData.totalResults;     if ( results > 0 ) {         // Get user data and assign to an environment variable         var jsonResources = jsonData.Resources;         var userId = jsonResources[0].id;         pm.environment.set("userid", userId);         // Log something to the console         console.log("Found Userid " + userId + " for Username " + userName);     } else if ( results === 0 ) {         // Log something to the console         pm.environment.unset("userid");         console.log("Exit workflow on ID not found for Username " + userName + " in iteration #" + n + " of #" + total);         postman.setNextRequest(null);     } } else {         console.log("POST op failed; exit the workflowfor iteration #" + n);         postman.setNextRequest(null); }

There is a lot going on in this script, but most of it has been developed to catch exceptions where for example if a username is not found, the second request in the collection will not be executed and a log will be output to the console, more on this later. The following code from the script though is worth mentioning because it is responsible for using the jsonData object set earlier in the script to get the JSON response.  With the jsonData object we can get the userid from the response and set it to an environment variable to use in the next PATCH request; remember when I mentioned we need the user "id" to modify a user?

// Get user data and assign to an environment variable         var jsonResources = jsonData.Resources;         var userId = jsonResources[0].id;         pm.environment.set("userid", userId);

Another special note are the lines that have “console.log(“blah”)”, these are used to provide helpful exception logs in the console if we can't find a username in the Identity Service Cloud, more on this in Part 3: Automating Requests against Identity Cloud Service using Newman. Remember that this a demo example and you can certainly elaborate on this to build out your own script for what ever purpose.  Let's move on.

 

Using a Pre-request Script to get an Environment Variable to PATCH

In the last section we set some environment variables in both the Pre-request and Tests scripts. In this section we are going to use the data in those variables in our PATCH request to modify each user in the CSV file.  Select the second request in our collection named Modify user Custom MVA, Phone Number. The first thing to point out is the end point for our PATCH request; see below.  Notice the variables "" and "" where the variable will be replaced with the tenant host value set in our environment setting that we updated earlier.  The is replaced with the "id" we retrieve from our environment variable that is set in the Tests script from the first request that searches for a username.

 

     /admin/v1/Users/

 

In the same second request in our collection select the Body tab, you will see the following request.  Note that there are no Pre-request Script or Tests scripts in this request though you could use those tabs and add scripts for various reasons, but in our demo we are keeping it simple.  In the body request below there are two variables, and , which again have been preset in our Pre-requests script from the first request and populated with data that is extracted from the CSV file. This is all we really need to do in order to use the variables, it's that simple.

{   "schemas": [     "urn:ietf:params:scim:api:messages:2.0:PatchOp"   ],   "Operations": [     {       "op": "replace",       "path": "urn:ietf:params:scim:schemas:oracle:idcs:extension:user:User:bypassNotification",       "value": true     },     {       "op": "replace",       "path": "urn:ietf:params:scim:schemas:idcs:extension:custom:User:appRoles" ,         "value":[  ]     },    {       "op": "replace",       "path": "phoneNumbers",       "value": [         {           "value": "",           "type": "home"         }       ]     }   ] }

 

Testing Runner Collection against Identity Service Cloud

We’re almost there! Postman Runner is a tool that lets you execute all the requests in a Collection, hence the name Runner Collection.  In this section I will give the final steps to use all our work.

STEP 1 - In Postman first select the Environment from the drop menu in the top right corner.

STEP 2 - Now in Postman move your mouse over the Acme IDCS Collection to the right of the name and you will see a little play button suddenly appear, click on the Play button.

STEP 3 - You should see a blue button named “Run”, click on it.  This will automatically add the Acme IDCS Collection to the collection and select the Acme IDCS Environment.

STEP 4 - Next select the “Select File” button next to where it says Data and browse in the “Acme IDCS Collection” folder for the test_user.csv file. At this point make sure your data has been updated to reflect actual users in your IDCS Tenant, if not no worries the requests will fail because it will not be able to find any of the users in the test_users.csv file.

STEP 5 – Now select the “Run Acme IDCS Collection button”, the BIG BLUE BUTTON on the bottom left. This will kick off Collection Runner and start iterating through the CSV file and send requests to your Identity Cloud Service tenant. As long as the environment variables are correct, and as long as the data in the CSV is correct, you should now see changes made in the Identity Cloud Service.

To confirm this all worked login to your Identity Service Cloud tenant and try looking up a couple users from the CSV file to see if the data did indeed get changed, if so, that is a success! Congrats!!  Feel free to repeat all this with different types of requests and user data on your own.

 

BONUS: I mentioned console.log(“blah”) before and if you open up the Postman Console, you will see all the requests, both successful and failures. If a user is not found, you will see the console log output that was incorporated into the Tests script.

Summary

Hopefully with this knowledge you will see the potential and use this framework to build your own collection and use Collection Runner to either test multiple requests or process some type of bulk operations. I won’t say this is perfect, but the benefit is you can easily develop the requests in Postman’s nice GUI, and with some basic script knowledge you can run several requests in a series for hundreds or thousands of users using a CSV file. In Part 3: Automate Requests against Identity Cloud Service using Newman, I am going to conclude with showing you how to install Newman and automate this same process, but from a command line. Enjoy!

Tim Melander

A-Team Cloud Solution Architect

I started with Oracle in 2005 and been a member of the Oracle A-Team since 2012 though have worked in Identity and Access Management since 1999.  My journey with security continues the cloud that heavily includes Oracle Infrastructure Cloud (OCI).  I enjoy writing articles built on real life use cases to help in areas where a standard document may not provide. I am a strong believer in learning by example to which I try to incorporate as many helpful tips, excellent diagrams, and instructional steps as I can.


Previous Post

Part 1: Automate getting an Identity Cloud Service Access Token in Postman

Tim Melander | 5 min read

Next Post


Part 3: Automate Requests against Identity Cloud Service using Newman

Tim Melander | 7 min read