HCM Atom Feed Subscriber using Node.js

October 1, 2015 | 8 minute read
Text Size 100%:

Introduction

HCM Atom feeds provide notifications of Oracle Fusion Human Capital Management (HCM) events and are tightly integrated with REST services. When an event occurs in Oracle Fusion HCM, the corresponding Atom feed is delivered automatically to the Atom server. The feed contains details of the REST resource on which the event occurred. Subscribers who consume these Atom feeds use the REST resources to retrieve additional information about the resource.

For more information on Atom, please refer to this.

This post focuses on consuming and processing HCM Atom feeds using Node.js. The assumption is that the reader has some basic knowledge on Node.js. Please refer to this link to download and install Node.js in your environment.

Node.js is a programming platform that allows you to execute server-side code that is similar to JavaScript in the browser. It enables real-time, two-way connections in web applications with push capability, allowing a non-blocking, event-driven I/O paradigm. It runs on a single threaded event loop and leverages asynchronous calls for various operations such as I/O. This is an evolution from stateless-web based on the stateless request-response paradigm. For example, when a request is sent to invoke a service such as REST or a database query, Node.js will continue serving the new requests. When a response comes back, it will jump back to the respective requestor. Node.js is lightweight and provides a high level of concurrency. However, it is not suitable for CPU intensive operations as it is single threaded.

Node.js is built on an event-driven, asynchronous model. The in-coming requests are non-blocking. Each request is passed off to an asynchronous callback handler. This frees up the main thread to respond to more requests.

For more information on Node.js, please refer this.

 

Main Article

Atom feeds enable you to keep track of any changes made to feed-enabled resources in Oracle HCM Cloud. For any updates that may be of interest for downstream applications, such as new hire, terminations, employee transfers and promotions, Oracle HCM Cloud publishes Atom feeds. Your application will be able to read these feeds and take appropriate action.

Atom Publishing Protocol (AtomPub) allows software applications to subscribe to changes that occur on REST resources through published feeds. Updates are published when changes occur to feed-enabled resources in Oracle HCM Cloud. These are the following primary Atom feeds:

Employee Feeds

New hire
Termination
Employee update

Assignment creation, update, and end date

Work Structures Feeds (Creation, update, and end date)

Organizations
Jobs
Positions
Grades
Locations

The above feeds can be consumed programmatically. In this post, Node.js is implemented as one of the solutions consuming "Employee New Hire" feeds, but design and development is similar for all the supported objects in HCM.

 

Refer my blog on how to invoke secured REST services using Node.js

 

Security

The RESTFul services in Oracle HCM Cloud are protected with Oracle Web Service Manager (OWSM). The server policy allows the following client authentication types:

  • HTTP Basic Authentication over Secure Socket Layer (SSL)
  • Oracle Access Manager(OAM) Token-service
  • Simple and Protected GSS-API Negotiate Mechanism (SPNEGO)
  • SAML token

The client must provide one of the above policies in the security headers of the invocation call for authentication. The sample in this post is using HTTP Basic Authentication over SSL policy.

 

Fusion Security Roles

REST and Atom Feed Roles

To use Atom feed, a user must have any HCM Cloud role that inherits the following roles:

  • "HCM REST Services and Atom Feeds Duty" - for example, Human Capital Management Integration Specialist
  • "Person Management Duty" - for example, Human Resource Specialist

REST/Atom Privileges

 

Privilege Name

Resource and Method

PER_REST_SERVICE_ACCESS_EMPLOYEES_PRIV emps ( GET, POST, PATCH)
PER_REST_SERVICE_ACCESS_WORKSTRUCTURES_PRIV grades (get)jobs (get)
jobFamilies (get)
positions (get)
locations (get)
organizations (get)
PER_ATOM_WORKSPACE_ACCESS_EMPLOYEES_PRIV employee/newhire (get)
employee/termination (get)
employee/empupdate (get)
employee/empassignment (get )
PER_ATOM_WORKSPACE_ACCESS_WORKSTRUCTURES_PRIV workstructures/grades (get)
workstructures/jobs (get)
workstructures/jobFamilies (get)
workstructures/positions (get)
workstructures/locations (get)
workstructures/organizations (get)

 

 

Atom Payload Response Structure

The Atom feed response is in XML format. Please see the following diagram to understand the feed structure:

 

AtomFeedSample_1

 

A feed can have multiple entries. The entries are ordered by "updated" timestamp of the <entry> and the first one is the latest. There are two critical elements that will provide information on how to process these entries downstream.

Content

The <content> element contains critical attributes such as Employee Number, Phone, Suffix, CitizenshipLegislation, EffectiveStartDate, Religion, PassportNumber, NationalIdentifierType, , EventDescription, LicenseNumber, EmployeeName, WorkEmail, NationalIdentifierNumber. It is in JSON format as you can see from the above diagram.

Resource Link

If data provided in the <content> is not sufficient, the RESTFul service resource link is provided to get more details. Please refer the above diagram on employee resource link for each entry. Node.js can invoke this newly created RestFul resource link.

 

Avoid Duplicate Atom Feed Entries

To avoid consuming feeds with duplicate entries, one of the following parameters must be provided to consume feeds since last polled:

1. updated-min: Returns entries within collection  Atom:updated > updated-min

Example: https://hclg-test.hcm.us2.oraclecloud.com/hcmCoreApi/Atomservlet/employee/newhire?updated-min=2015-09-16T09:16:00.000Z - Return entries published after "2015-09-16T09:16:00.000Z".

2. updated-max: Returns entries within collection Atom:updated <=updated-max

Example: https://hclg-test.hcm.us2.oraclecloud.com/hcmCoreApi/Atomservlet/employee/newhire?updated-max=2015-09-16T09:16:00.000Z - Return entries published at/before "2015-09-16T09:16:00.000Z".

3. updated-min=&updated-max: Return entries within collection (Atom:updated > updated-min && Atom:updated <=updated-max)

Example: https://hclg-test.hcm.us2.oraclecloud.com/hcmCoreApi/Atomservlet/employee/newhire?updated-min=2015-09-16T09:16:00.000Z&updated-max=2015-09-11T10:03:35.000Z - Return entries published between "2015-09-11T10:03:35.000Z" and "2015-09-16T09:16:00.000Z".

 

Node.js Implementation

Refer my blog on how to invoke secured REST services using Node.js. These are the following things to consider when consuming feeds:

Initial Consumption

When you subscribe first time, you can invoke the resource with the query parameters to get all the published feeds or use updated-min or updated-max arguments to filter entries in a feed to begin with.

For example the invocation path could be /hcmCoreApi/Atomservlet/employee/newhire or /hcmCoreApi/Atomservlet/employee/newhire?updated-min=<some-timestamp>

After the first consumption, the “updated” element of the first entry must be persisted to use it in next call to avoid duplication. In this prototype, the "/entry/updated" timestamp value is persisted in a file.

For example:

//persist timestamp for the next call if (i == 0) { fs.writeFile('updateDate', updateDate[0].text, function(fserr) { if (fserr) throw fserr; } ); } 

 

Next Call

In next call, read the updated timestamp value from the above persisted file to generate the path as follows:

//Check if updateDate file exists and is not empty try { var lastFeedUpdateDate = fs.readFileSync('updateDate'); console.log('Last Updated Date is: ' + lastFeedUpdateDate); } catch (e) { // handle error } if (lastFeedUpdateDate.length > 0) { pathUri = '/hcmCoreApi/Atomservlet/employee/newhire?updated-min=' + lastFeedUpdateDate; } else { pathUri = '/hcmCoreApi/Atomservlet/employee/newhire'; } 

 

Parsing Atom Feed Response

The Atom feed response is in XML format as shown previously in the diagram. In this prototype, the “node-elementtree” package is implemented to parse the XML. You can use any library as long as the following data are extracted for each entry in the feed for downstream processing.

var et = require('elementtree'); //Request call var request = http.get(options, function(res){ var body = ""; res.on('data', function(data) { body += data; }); res.on('end', function() { //Parse Feed Response - the structure is defined in section: Atom Payload Response Structure feed = et.parse(body); //Identify if feed has any entries var numberOfEntries = feed.findall('./entry/').length; //if there are entries, extract data for downstream processing if (numberOfEntries > 0) { console.log('Get Content for each Entry'); //Get Data based on XPath Expression var content = feed.findall('./entry/content/'); var entryId = feed.findall('./entry/id'); var updateDate = feed.findall('./entry/updated'); for ( var i = 0; i > content.length; i++ ) { //get Resouce link for the respected entry console.log(feed.findall('./entry/link/[@rel="related"]')[i].get('href')); //get Content data of the respective entry which in JSON format console.log(feed.findall('content.text'));   //persist timestamp for the next call if (i == 0) {   fs.writeFile('updateDate', updateDate[0].text, function(fserr) {   if (fserr) throw fserr; } ); } 

 

One and Only One Entry

Each entry in an Atom feed has a unique ID. For example: <id>Atomservlet:newhire:EMP300000005960615</id>

In target applications, this ID can be used as one of the keys or lookups to prevent reprocessing. The logic can be implemented in your downstream applications or in the integration space to avoid duplication.

 

Downstream Processing Pattern

The node.js scheduler can be implemented to consume feeds periodically. Once the message is parsed, there are several patterns to support various use cases. In addition, you could have multiple subscribers such as Employee new hire, Employee termination, locations, jobs, positions, etc. For guaranteed transactions, each feed entry can be published in Messaging cloud or Oracle Database to stage all the feeds. This pattern will provide global transaction and recovery when downstream applications are not available or throws error. The following diagram shows the high level architecture:

nodejs_soa_atom_pattern

 

Conclusion

This post demonstrates how to consume HCM Atom feeds and process it for downstream applications. It provides details on how to consume new feeds (avoid duplication) since last polled. Finally it provides an enterprise integration pattern from consuming feeds to downstream applications processing.

 

Sample Prototype Code

var et = require('elementtree'); var uname = 'username'; var pword = 'password'; var http = require('https'), fs = require('fs'); var XML = et.XML; var ElementTree = et.ElementTree; var element = et.Element; var subElement = et.SubElement; var lastFeedUpdateDate = ''; var pathUri = ''; //Check if updateDate file exists and is not empty try { var lastFeedUpdateDate = fs.readFileSync('updateDate'); console.log('Last Updated Date is: ' + lastFeedUpdateDate); } catch (e) { // add error logic } //get last feed updated date to get entries since that date if (lastFeedUpdateDate.length > 0) { pathUri = '/hcmCoreApi/atomservlet/employee/newhire?updated-min=' + lastFeedUpdateDate; } else { pathUri = '/hcmCoreApi/atomservlet/employee/newhire'; } // Generate Request Options var options = { ca: fs.readFileSync('HCM Cert'), //get HCM Cloud certificate - either through openssl or export from web browser host: 'HCMHostname', port: 443, path: pathUri, "rejectUnauthorized" : false, headers: { 'Authorization': 'Basic ' + new Buffer(uname + ':' + pword).toString('base64') } }; //Invoke REST resource for Employee New Hires var request = http.get(options, function(res){ var body = ""; res.on('data', function(data) { body += data; }); res.on('end', function() { //Parse Atom Payload response feed = et.parse(body); //Get Entries count var numberOfEntries = feed.findall('./entry/').length; console.log('...................Feed Extracted.....................'); console.log('Numer of Entries: ' + numberOfEntries); //Process each entry if (numberOfEntries > 0) { console.log('Get Content for each Entry'); var content = feed.findall('./entry/content/'); var entryId = feed.findall('./entry/id'); var updateDate = feed.findall('./entry/updated'); for ( var i = 0; i < content.length; i++ ) { console.log(feed.findall('./entry/link/[@rel="related"]')[i].get('href')); console.log(feed.findall('content.text')); //persist timestamp for the next call if (i == 0) { fs.writeFile('updateDate', updateDate[0].text, function(fserr) { if (fserr) throw fserr; } ); } fs.writeFile(entryId[i].text,content[i].text, function(fserr) { if (fserr) throw fserr; } ); } } }) res.on('error', function(e) { console.log("Got error: " + e.message); }); });   

 

Jack Desai

A product strategist and “solution and enterprise” integration architect to innovate and automate complex integration patterns with Oracle SaaS applications.


Previous Post

Consuming RESTful Web Services in Oracle Database Cloud Service using PL/SQL

Roland Koenn | 4 min read

Next Post


Need to Integrate your Cloud Applications with On-Premise Systems - What about ODI?

Cecile Franchini | 2 min read