Performance implications of Synchronous and Asynchronous patterns in large enterprise Oracle Integration Cloud implementations with SaaS Applications

Executive Summary

Oracle Integration Cloud (OIC) platform Is commonly used in many large enterprise implementations with SaaS Applications and is a key component in time-critical integrations. Often, these applications have a need to orchestrate interactions with multiple end points quickly to meet stringent SLAs.  A common OIC pattern leveraged in such situations uses asynchronous integrations which fires a flow and returns a successful completion response immediately. However, it does not guarantee the completion of the entire process at that time as the integration might still be running in the background. Consequently, with several asynchronous integrations running, we may see some asynchronous integrations, which are normally short-lived, may take an unusually long time to complete during these peak load intervals. In contrast, synchronous integrations provide a much quicker turnaround time during such overload of asynchronous integrations and its performance remain virtually unaffected by the large volume of asynchronous integrations. This blog attempts to highlight the performance differences between asynchronous and synchronous integrations by simulating a peak load scenario and observing their completion times.

Integration Pattern Architecture

We start with building an asynchronous and synchronous version of a simple integration to accept a JSON payload as input and write the payload contents to a record in a database table using the DB adapter. Fig.1 shows the 2 integrations.

1. Each of the integrations accepts a JSON payload as input
2. The payload contents are written as a single record to a database table via DB Adapter with the creation timestamp as one of the fields in the record.

Fig. 1 Synchronous and Asynchronous Integrations for Test Cases

Implementation of the Test integrations

The implementation of the asynchronous integration flow is shown in Fig. 2.

Fig. 2 Asynchronous Integration

Similarly, the details of the synchronous integration flow is shown in Fig. 3.

Fig. 3 Synchronous Integration

Setup

To demonstrate the performance implications of the asynchronous and synchronous patterns, 2 shell scripts are built to invoke both the integrations in parallel. The pseudocode with overall structure of the shell script is described below.

• Initialize header parameters
• In a loop that repeats 50 times, do the following:
  • Build the JSON payload at runtime with record number incremented with loop counter.
  • Call the REST endpoint to invoke the integration with the JSON payload passed as the body.
  • Sleep for 500 milliseconds (optionally inserted to simulate the real-life behavior).

Baseline

For setting baseline metrics, both the integrations were run in parallel in an idle environment with no other integrations running. The start and end times were noted down for the first and last of the 50 records inserted by the asynchronous and synchronous integrations.

Results were in alignment with our expectations, as it was seen that both the integrations were able to insert 50 records within 50 seconds, accounting for an insert of approximately 1 record/sec.

Test with System Under Load

Next, we ran a few integrations that essentially generated a flurry of asynchronous integration instances and mostly remained in processing state during our entire test window, which was around 3 minutes.

There is no specific method to achieve this goal and it is left to the user’s discretion to create their own way of spawning several asynchronous integrations. The implementation of such integrations is beyond the scope of this blog and hence not discussed in detail. Any follow-up queries on methods to simulate a peak load of asynchronous integrations can be addressed offline.

A good indicator of a busy system with asynchronous integrations can be determined from the OIC monitoring dashboard. If we see many asynchronous integrations are stuck in running or processing state without completing successfully during any certain interval, we can assume that the system is backed up with asynchronous processing. After the system was setup with such a large volume of asynchronous integrations stuck in running state, we ran the test shell scripts again to insert 50 records in the database table via the synchronous and asynchronous integrations.

The next section describes the results obtained by running these shell scripts the second time and comparing them with the baseline results found earlier.

Test Results

The test results obtained with the 2 integration patterns are shown in Fig. 4. At peak loads, 50 records were inserted into the database table around the same time via synchronous integration (~51 seconds) but the time taken by asynchronous integrations was much higher (2+ minutes).

Fig. 4 Test results for Synchronous and Asynchronous Integrations

The data shown above clearly demonstrates a significant degradation of performance of the asynchronous test candidate at peak loads when several other asynchronous integrations were running in the system. In contrast, the performance of synchronous integrations remained about the same regardless of the asynchronous load on the environment.

Interpretation of Results

The slowdown of asynchronous integrations at peak loads can also be observed by looking at the activity stream of any asynchronous integrations that took a long time to complete. Fig. 5 below shows the activty stream of such an instance.

Fig. 5 Activity Stream of Asynchronous Integration instance under peak load

The important timings to look at the activity stream are the first 2 timestamps, namely,

• Message received -  4:17:33.911 PM
• Message dequeued for processing – 4:20:05.970 PM
• Processing completed – 4:20:06.490 PM

We notice here that the message was dequeued for processing after ~3 minutes from its receipt. However, after the processing started, it did not take any time to complete rest of the processing. In summary, the bottleneck during the beginning of the asynchronous integrations becomes significant at peak loads.

On the other hand, the synchronous integrations did not experience any delays as a result of the large volume of asynchronous integrations in the system. This concludes the testing of the synchronous and asynchronous patterns.  

Summary

To summarize, this exercise is carried out to demostrate the fact that asynchronous integrations may experience a significant delay between the time of enqueing and dequeing during peak loads in the system.

Consequently, if a quick response is desirable at peak loads with several asynchronous integrations running, it might be worthwhile to look at synchronous processes to model the integration flow. This does not mean that a synchronous integration will always guarantee a quick response time at peak loads. The exercise here only demonstratres that the performance of synchronous integrations is unaffected by the peak load of asynchronous integrations in the system.

In large enteprise implementations of OIC, design decisions for synchronous and asynchronous modeling should be carefully assessed specially when complex integrations with SaaS applications are involved. The proper mix of diferent patterns should be optimized to ensure good performance of the overall implementation.

For more details on integration of OIC with SaaS and other external endpoints, please refer to the OIC Product Documentation portal ^[1].

Acknowledgements

I would like to acknowledge the assistance I received from my teammate, Greg Mally for the compilation of this blog. I would also like to thank Tulasiram Kodali of Oracle PaaS Performance Team for his help at different times.

References

1. OIC Documentation Portal – Product Documentation for Oracle Integration