Identity Cloud Services Audit Event REST API

Introduction

This article is to help expand on topics of integration with Oracle’s Cloud Identity Management service called Identity Cloud Services. Identity Cloud Services delivers core essentials around identity and access management through a multi-tenant Cloud platform. As part of the Identity Cloud Services framework, it collects audit events that capture all significant events, changes, and actions which are sent an audit table. Like any Identity and access service that revolves around security you will eventually need to access audit records for various reasons pertinent to standard security practices and corporate policy. In this article I want to cover what the Identity Cloud Services audit events provide and how to leverage them using the Identity Cloud Services REST API Audit Events endpoints.

 

Auditing Overview

The audit events can be accessed using the Identity Cloud Services SCIM 2.0 compliant REST API. SCIM (System for Cross-domain Identity Management) which is an open standard to simplify user identity management in the cloud. The following is a quick summary of what you should know from a high level.

* Audit events include login events, changes, and actions.
* Audit events are kept up to a maximum of 90 days
* Audit events are managed using REST APIs via OAuth 2.0
* Audit event REST endpoints allow query parameters and filters
* Audit event REST responses are in JSON format

Reporting is a basic feature that comes as part of the Identity Cloud Services user interface, but only provides some simple reporting. A more powerful way to retrieve Audit records from Identity Cloud Services is to use the REST API. The REST API endpoint can use optional query parameters and filters to fine tune what information you want, more on this in the next couple of sections.

 

Audit Event Endpoints

The following table covers all possible IDCS audit event endpoints. In addition, you should know that some endpoints can include query parameters that can include schema attributes, which I will cover in the next section.

 

Method

Action

Endpoint

Comment

POST

Create

/admin/v1/AuditEvents

Create a new audit record. Any parameters need to be included in a JSON body.

DELETE

Delete

/admin/v1/AuditEvents/{id}

Delete an audit record using an audit record ID. Any parameters need to be included in a JSON body.

GET

Get Event by ID

/admin/v1/AuditEvents/{id}

Retrieve a single audit record using a unique ID. Any parameters need to be included in query string.

GET

Search by GET

/admin/v1/AuditEvents

Parameters can be included using a query string. Any parameters need to be included in query string.

POST

Search by POST

/admin/v1/AuditEvents/.search

Parameters are posted in the request body using JSON. Any parameters need to be included in a JSON body.

 

Audit Event Query Parameters

The following table provides all the audit event parameters that can be used to query the records. In later sections I will get into some examples on how to use some of these parameters.

Parameter

Type

Description

filter

string

A filter using valid schema attributes to request specific resources.  The filter can include logical operators such as AND and OR. For more details see the SCIM specifications https://tools.ietf.org/html/draft-ietf-scim-api-19#section-3.4.2.2 for more information.

attributes

string

A comma delimited string of valid attributes that specify resources.  The values of the attributes should match the required SCIM schema definition.

sortBy

string

Used to sort the response by some valid audit event attribute.

sortOrder

string

Using the allowed values “ascending” or “descending” to order the sort of the results; the default if sortOrder is not used is ascending.

count

number

Can set the maximum number of records returned per page.  Excluding “count” sets the default maximum number to 50, where the maximum value allowed is 1000.  If the number of records returned is larger than the count value, you must use the startIndex to paginate through the records.

startIndex

number

This determines the first record in the page set.  The default is 1, so if the startIndex is set to 100, the 100th record will be the first in the list returned.  See the Pagination section of the SCIM specifications https://tools.ietf.org/html/draft-ietf-scim-api-19#section-3.4.2.4 for more information.

My First Audit Event Search

Now that I have covered the core endpoints and query parameters let’s get into our first search. Imagine you either work in Info Sec or work with someone who does, in either case there will been times an audit is required. Even if an audit is not something done on a regular basis, the fact that Identity Cloud Services will only keep a maximum of 90 days of records means if your corporate policy demands say records that be kept for 7 years, you must establish a process to query the Identity Cloud Services audit records on a regular basis and store them externally so that you can use tools like BICS (Business intelligence Cloud Service) to build reports when needed even if you need to go back 7 years.

There are two methods used to send searches to the Identity Cloud Services REST API Audit Events endpoints, GET or POST, and though each option provides the same results, how the query parameters are sent differ. The following basic searches should help illustrate the differences between using GET and POST.

 

GET method

     https://tenant1.mycompany.com/admin/v1/AuditEvents?filter=actorName sw “bhaas”

POST method

     https://tenant1.mycompany.com/admin/v1/AuditEvents/.search

JSON body

     JSon Body
     {
          “schemas”: [“urn:ietf:params:scim:api:messages:2.0:SearchRequest”],
          “attributes”: [“actorName”],
          “filter”: “actorName sw \”bhaas\””,
          “startIndex”: 1,
          “count”: 5
     }

Notice the GET method above send all the parameters in a URL query string, while the POST method requires the addition of “/.search” to the endpoint plus the search parameters to be sent in a JSON body. Whether you use GET or POST is up to you, it will most likely depend on your application integration requirements.

Before we jump into sending a search via the REST API, I am going to assume you already know how to get a proper OAuth 2.0 token. If not, for your convenience I have already published an article on how to do this in the article “IDCS OAuth 2.0 and REST API”, which gives easy steps and examples using cURL or Postman. So going forward I am only going to focus on the endpoint and query parameters. Let’s move on to our first audit event search example.

     /admin/v1/AuditEvents?filter=actorName sw “tim”

Let’s break out the above search to understand what we are doing using the following table.

Part

Value

Description

Endpoint

/admin/v1/AuditEvents

The endpoint used to query audit events

Query parameter

?filter=

A parameter used to filter on some SCIM attribute

Attribute

actorName

Attribute used in the filter parameter

Logical Operator

sw

The logical operator “sw” is starts with, but there are many others.

Search value

“tim”

This is the value to search for

 

Now before we finally send that search, I want to point out if you are going to send a GET request using the cURL command, the query string needs to be URL encoded.

Take the following example…

* Will NOT work with cURL:
     ?filter=actorName sw “tim”

 

* Will work with cURL:
     ?filter=actorName%20sw%20%E2%80%9Ctim%E2%80%9D

 

So the below final cURL command can be used to send our first audit event search; be sure to replace the <Your Bearer token> with a real token.

     curl \
     -X GET \
     -H “Content-Type:application/scim+json” \
     -H “Authorization: Bearer <Your Bearer token>” \
     http://tenant1.idcs.my.company.com:8990/admin/v1/AuditEvents?filter=actorName%20sw%20%E2%80%9Ctim%E2%80%9D

 

Once you send your search a response will come back if everything went successful. The format of the response is JSON, an open standard, that can then be parsed or manipulated as needed. This makes building custom interfaces relatively simple. The following is an example of what your response may look like.

JSON RESPONSE

Search by Date Range

Another useful search is to filter by date range. This example is a little more complicated, but I will walk through all the parts of the search.

     /admin/v1/AuditEvents?filter=timestamp ge “2016-06-20T00:00:00Z” and timestamp le “2016-06-22T00:00:00Z”&sortBy=timestamp&sortOrder=descending”

 

Below is a table to help understand more about the parts of the search to help learn what we are sending and why.

Part

Value

Description

Endpoint

/admin/v1/AuditEvents

The endpoint used to query audit events

Query parameter

?filter=

A parameter used to filter using a valid SCIM attribute.

Attribute

timestamp

Attribute used in the filter parameter

Logical operator

“ge” and “le”

The logical operator “ge” for greater than or equal to, and “le” for less than or equal to. 

Search value

“2016-06-20T00:00:00Z” and “2016-06-22T00:00:00Z”

These are the date values used in the search, which must be in UTC format.

sortBy

timestamp

Sorts by a valid SCIM schema attribute, “timestamp” of the audit records.

sortOrder

descending

This the sort order of the results; options are ascending or descending.

 

IMPORTANT:  When using a date range search with Identity Cloud Services, you should include the “sortBy” parameter as a habit.  The reason is if you are paging through multiple results by setting the startIndex parameter, you will get an error if the page limit goes beyond the first page set.  For example, if there are a total of 152 records returned, and you set the startIndex parameter to 51 to get the second page set of records you will get the following error unless you use the sortBy parameter.

 

     {

     “schemas”: [

     “urn:ietf:params:scim:api:messages:2.0:Error”,

     “urn:ietf:params:scim:api:oracle:idcs:extension:messages:Error

     ],

     “detail”: “Missing \”sortby\”. sortby is mandatory when startIndex is greater than 1.”,

     “status”: “400”,

     “urn:ietf:params:scim:api:oracle:idcs:extension:messages:Error”: {

     “messageId”: “error.common.common.missingSortBy”

     }

     }

 

Again, to send the query string using the cURL command, we need to URL encode our query string as follows before sending it.

     curl \
     -X GET \
     -H “Content-Type:application/scim+json” \
     -H “Authorization: Bearer <Your Bearer token>” \
     http://tenant1.idcs.my.company.com:8990/admin/v1/AuditEvents?filter=timestamp%20ge%20%222016-06-20T00%3A00%3A00Z%22%20and%20timestamp%20le%20%222016-06-22T00%3A00%3A00Z%22%26sortBy%3Dtimestamp%26sortOrder%3Ddescending%E2%80%9D

Understanding the REST API Result Limits

The Identity Cloud Services REST API has some defaults when returning a large number of records. When you send a search you will notice a couple things in the JSON response. At the top of the result you will see a parameter “totalResults”. This shows the total number of records from the query, but it does not mean that is how many results you got in your response.

       {
          “schemas”: [
          “urn:scim:api:messages:2.0:ListResponse”
          ],
          “totalResults”: 52,

At the bottom of the result there are a couple other parameters, “startIndex” and “itemsPerPage”. The startIndex is the point in the page result you are viewing where the itemsPerPage parameter lets us know there is a maximum of 50 records per page.

     ],
          “startIndex”: 1,
          “itemsPerPage”: 50
     }

 

Every page set that is returned will contain totalResults, startIndex, and itemsPerPage. If we put this together it tells us —

1. The is a total number of records in our search is 52; e.g. “totalResults”: 52.
2. Our current result set is our first; e.g. “startINdex”: 1.
3. Finally our maximum records per page set is 50; e.g. “itemsPerPage”: 50.

An important note is the “itemsPerPage” has a default value of 50, but you can override this by using the “count” parameter. If you were to include the “count” parameter in the query string or JSON body with the value of 200, the total records per page returned would be a maximum of 200 per page set. For example…

 

     /admin/v1/AuditEvents?filter=timestamp ge “2016-06-20T00:00:00Z” and timestamp le “2016-06-22T00:00:00Z”&sortBy=timestamp&sortOrder=descending&count=200”

 

An important note about the “count” parameter is there is a maximum limit of 1000. Even if you say change the count value to 2000, only a maximum of 1000 records per page set is returned. This presents a little work if say 5000 records are returned. So how do you deal with that?

 

How to Deal with Large Record Results?

To build on the previous section let’s try to understand how to deal with large record results. Let’s assume our total record result equals 152. One option is to set the count parameter we learned about earlier to a value of 1000. We certainly would get all our records returned since the total number of records is less than 1000, but if our total record result is greater than 1000 that introduces a problem. To solve this, we need to paginate through the records using the startIndex parameter.

First of all, we won’t really know how many records we are going to get returned do we. So a trick is to use the “count” parameter and set the value to “0”.

 

    /admin/v1/AuditEvents?filter=timestamp ge “2016-06-20T00:00:00Z” and timestamp le “2016-06-22T00:00:00Z”&sortBy=timestamp&sortOrder=descending&count=0”

 

This will not return any records, but it will show us the total number of records returned in your search. In the example below we have 152 records in total returned by the search.

 

     {

        “schemas”: [

        “urn:scim:api:messages:2.0:ListResponse”

        ],

        “totalResults”: 152,
   
“Resources”: [],

        “startIndex”: 1,

        “itemsPerPage”: 0

     }

 

Once we know the total records we can paginate through the records using the startIndex parameter.  Assume we don’t bother with using the count parameter and go with the default of 50 records per page.  We can then do something such as the pseudo code below using awhile loop.

     count = 0
     while count < 152
          startIndex = count +1
          Get 50 records starting at startIndex
          count = startIndex + 50
     end

 

The idea is we can get all our records by paging through them incrementing the startIndex. To continue with the previous query examples we would do something like this.

 

Records 1 – 50

     /admin/v1/AuditEvents?filter=timestamp ge “2016-06-20T00:00:00Z” and timestamp le “2016-06-22T00:00:00Z”&sortBy=timestamp&sortOrder=descending&startIndex=1”

 

Records 51 – 101

     /admin/v1/AuditEvents?filter=timestamp ge “2016-06-20T00:00:00Z” and timestamp le “2016-06-22T00:00:00Z”&sortBy=timestamp&sortOrder=descending&startIndex=51”

 

Records 102 – 152

     /admin/v1/AuditEvents?filter=timestamp ge “2016-06-20T00:00:00Z” and timestamp le “2016-06-22T00:00:00Z”&sortBy=timestamp&sortOrder=descending&startIndex=102”

Audit Events Schema

I have covered a couple common search examples to get audit events, but eventually you may want to do other things. So I already pointed out the Audit Events endpoints and the query parameters, but as you try to build more queries you may realize that is not enough. For example, you already know about a query parameter “filter”, but what attributes can I use? So to complete the full circle you will need to understand the Audit Events schema.

You could reference Oracle’s Identity Cloud Services documentation, but another tip is to simply send a REST request to return the entire Audit Events schema. Before you can do this your client will require the scope “Identity Domain Administrator”. To do this complete the following steps.

 

1. Login as an Administrator to the IDCS admin console

     2. Select the Applications tab

     3. Click your Client

     4. Select the Configuration tab

     5. Expand the Client Configuration

     6. In the Grant the client access to Identity Cloud Service Admin APIs and add the scope: Identity Domain Administrator

     7. Click the Save button.

Once the above steps are complete you will need to request a new token which will include the new authorization roles. Next you will use the following query string to request the Audit Events schema.

     /admin/v1/Schemas/urn:ietf:params:scim:schemas:oracle:idcs:AuditEvent

Below is an example cURL command you can run to return the schema.

     curl \
     -X GET \
     -H “Content-Type:application/scim+json” \
     -H “Authorization: Bearer <Your Bearer token>” \
     http://tenant1.idcs.my.company.com:8990/admin/v1/Schemas/urn:ietf:params:scim:schemas:oracle:idcs:AuditEvent

Once the REST request is sent a response a JSON format response will be returned that includes the full Audit Events schema. You can now see what attributes and objects are part of the schema in order to learn how to build new filters. The audit event schema JSON output will be fairly large so as another tip is to pipe the JSON response to a file, then copy and paste the data into the JSON field at http://json2table.com/ , and finally click the run button which will display it as a nice table. This should help make it easier to find the attributes you are looking for when building filters or other things with the query parameters.

 

Summary

I hope to have enlightened you on some knowledge to understand how to query the Identity Cloud Services audit event records. This article should be a good building block on going beyond some of my examples. The rest is up to you…no pun intended. Check out the article Loading Data from Oracle Identity Cloud Service into Oracle BI Cloud Service using REST that shows how to develop a process to extract Identity Cloud Services audit events into BICS (Business Intelligence Cloud Service) so you an leverage its powerful reporting and analytics tools. I hope this will add more excitement to Identity Cloud Services open framework. Until then enjoy!

Add Your Comment