In this blog we will look into a pattern to connect Oracle Fusion SCM Product Development (Product Data Hub - PDH) to downstream applications in realtime using an event based streaming integration.
Here is the use case and an overview of the integration solution.
Items are loaded into Fusion SCM Product Development (PDH) using regularly scheduled Item Bulk load batch job. When the batch completes, a bulk item load event is generated which encapsulates the ESS Request ID and status information of the batch job. An Oracle integration (OIC) flow which subscribes to the SCM Item bulk load event is triggered on completion of the batch job. The Integration flow then queries Fusion SCM for list of Items loaded in the batch, iterates through each Item, enriching by additionally querying required Item Info attributes and child structures as needed. The integration then publishes every item info to an OCI stream partition (OSS) for further downstream processing by downstream applications.
The pattern involves a event driven integration that is triggered by an event of interest occuring on Fusion SCM. The processing further handed down to an asynchronous integration to help deal with high volume item load synchronization downstream in a performant manner.
The further section will detail the integration solution. Note that the solution is similar on both OIC Gen3 as well as OIC Gen2 and screenshots from both flavours of OIC are shown to remind of this fact.
Below are the connections used in the integration
Below screenshots show in brief the overall integrations, configurations used , rough mappings between payloads and scope level fault handler
Here are some of the salient steps from the above Event listener integrtaion.
Firstly the Trigger which is configured to listen to Item Bulk Load event as shown below
The received event payload consists of the Request ID of the ESS Job that performed the bulk load of items into Fusion SCM. Shown below are the mapping task and invoke operation on SCM connection using findItem operation on ItemService. They are responsible to fetch all the individual Item Numbers from the Bulk Item process
Note that the query of Items is limited to 500 items per fetch to avoid the below Fusion SaaS error when invoking the FindItem Service
For requirements where there are more than 500 items per batch, the below approaches can be adopted
1) Break the ESS batches into smaller chunks so that each batch processes less than 500 item load into PDH
2) Use an outer while loop in BulkItemsLoad EventListener which passes 500 items in every turn to the ForEach Iterator, until there are more items to process.
Next, here is the For-Each iterator that invokes the Stream Dispatcher asynchronously for every item from the ESS batch
Note the scope task within the For-Each iterator. The scope handler allows us to attach a scope level fault handler to handle any errors while processing individual items without interrupting the entire batch
Depicted above is a simple scope handler that logs the fault information and proceeds iterating to other items in the batch. But practically, the scope handler could be used to notify operations about the individual failures so they can be recovered manually after correcting any errors.
The streams dispatcher is a simple asynchronous integration that gets an Item id as input. It fetches the required items information using Items SOAP or REST services. This enriched Items information is then sent as payload content for OCI streaming service OSS. Further downstream, we clould have multiple subcribers that consume the Item details and process them in various end systems. The subscriber flows are not deplicted in this blog.
Below picture shows a sample mapping to fetch Item details from Fusion SCM using the ItemService
The retrieved Item details are then mapped to streams payload. Streaming supports various message structure options like Avro, JSON and XML. Shown here is message publish without message structure, where the entire payload is mapped to streaming message's content field.
Once the enriched Items information are published to OCI streams, further downstream integrations using OCI Streams connection can be used to consume and process them to downstream applications.
We have seen how to build event driven integrations to synchronize business objects from Fusion SCM with external applications asynchronously using OCI Streaming sevice
As a member of Cloud Solution Architects A-Team, my area of work involves SaaS, PaaS4SaaS and IaaS. My work focuses on Solution architecture involving Fusion Supply Chain Management (SCM), Integration, Extensions, High Availability, Security, Infrastructure, Hybrid Cloud and Networking aspects of Oracle cloud solutions.