Dealing with high volume of transactions in Oracle Business Activity Monitoring (BAM) 11g can be challenging. Making the right design decision is critical for the stability and performance of your system. This post covers the common issues and general design recommendations for Oracle BAM 11g in the context of high volume of transactions.
The common issue in the context of high volume of transactions, is that Active Data processing at the BAM Server Side may not catch up the speed of data received in BAM Active Data Cache (ADC), which in turn may cause slight or serious delays in report refreshing with real-time data. When there are a huge backlog of unprocessed data in BAM, the performance and functionality of the whole system will be significantly impacted. Now, let's take a look at the following report design aspects which are the key factors impacting the scalability and performance, and our recommendations
Star Schema with High Volume of Data
It is common to use Star Schema as the data model in BAM to store business data. Whenever a message is received by BAM, it will be stored in the schema, and triggers Active Data processing within BAM Server, Under small or median load, say less than 10 - 20 messages per second (the rate of incoming messages received by BAM ADC), Active Data processing can keep up with the incoming data rate, and report should be working as expected. However, if the load goes higher, say over 100 messages per second, Active Data processing will consume higher resources (e.g. CPU, Memory, IO, etc), and the chance of performance degradation will be higher as well. Thus, our main design consideration here is to find out a way to shrink the size and data volumes of the star schema.
1. If you have one single Star Schema that contains a lot of look-up and calculated fields, consider breaking the one big Star Schema into multiple smaller schemas. The benefit of having multiple schemas is that the size and data volumes for each schema is reduced, therefore reducing server side contention and Active Data processing time.
2. If the transaction volume is high, say more than 60 transactions per second, consider adding additional pre-processing for the business data before sending them to BAM. For example, you can use Oracle Event Processing (OEP) to apply filters or aggregation for the business data before sending them to BAM. Using this approach, the data volume in BAM will be dramatically reduced.
3. Another approach for dealing with the high volume of data is to use Poll Mode versus Push Mode for BAM dashboards. BAM Dashboards in Poll Mode bypass Active Data Processing and reload itself at a time interval. Under heavy load, this approach consumes less system resources (CPU, Memory, Threads, etc.), and therefore performs better in terms of throughput and response time. The charts below shows the difference between Push and Poll Mode in terms of CPU usage and thread count.
CPU and Thread Count - Push Mode
CPU and Thread Count - Poll Mode
As you can see, when a report is running in Push Mode, CPU usage is consistently around 20%, and the system creates more threads for handling active data. From the performance perspective, a report in Poll Mode uses less resources and perform better under heavy load.
4. Consider normalizing your current data model to reduce the data volume of a single Data Object. If the main fact table (main Data Object of Star Schema) contains redundant fields that cause the data volume growing dramatically, consider normalizing the data model by moving these fields to separate Data Objects or External Data Objects. If you need to drill down the current view to show the values of these redundant fields, create a new report displaying these fields in Poll Mode and use the drill across feature to link the original view to the new view.
Manipulating Data in Views
BAM reports can include data manipulation functions, such as filters, calculations, drill down, drill across, driving, etc. Data manipulation is expensive in BAM, and can seriously impact performance if overused in BAM reports under heavy load. Applying filters, drill down, or drill across in a BAM report will get all report views reloaded. Reloading a report is expensive as it requires querying database, re-compiling internal classes built for evaluating calculated fields, and establishing persistent HTTP connections between client and server. Frequent reloading will increase the changes of getting contentions for accessing server resources.
1. Minimize the usage of data manipulating functions if possible. If these functions have to be used, ensure the underling Data Object does not contain high volume of data. In the context of high transaction rate, consider normalizing Data Objects instead of using single Star Schema or using Poll Mode for dashboards.
2. If drill down or drill across function is used in the report design, we recommend that you use separate Data Objects for the main and target view. Using separate Data Objects can help to reduce the data volume, which is the key factor for improving performance under heavy load.