ODI Agents: Standalone, JEE and Colocated

 Introduction

 With its version 12, Oracle Data Integrator offers three flavors of agents: JEE, Colocated and Standalone. The purpose of this post is to compare and contrast the benefits of each one of these types of agents and to help developers and architects select the best option(s) for their implementation.

 ODI Agents: Standalone, JEE and Colocated

In this comparison between the three agent flavors, we only refer to the 12c version of ODI unless otherwise noted.

Agents flavors and installation types

To provide some context, let us review the different agents available in ODI 11g and ODI 12c, as well as the type of installation required for each agent flavor.

Agents Install Compare

Low footprint: the standalone agent

Historically, this is the original ODI agent, and it is still available today. The number one benefit of this agent is its very light footprint: it does not require an application server since it is a standalone Java program. This makes it ideal for installations on busy and already active systems, including database servers. It provides all the required features to orchestrate data integration processes, including exposing these processes as web services. The lightweight nature of the agent brings many benefits:

  • In an ELT architecture, the agent can reside on the same server as the target database, hence reducing the number of network hops when transferring data through JDBC (see Understanding Where to Install the ODI Standalone Agent for more details on this);
  • For integrations that require access to flat files, scripts and utilities that are not accessible from the network, installing the agent on the file server or the machine that hosts scripts and utilities makes for a very flexible infrastructure.
  • Lifecycle and availability of a Standalone Agent is managed by Node Manager monitoring and restarting the agent.

Enterprise deployments:  the JEE agent

There are a number of limitations with the standalone agent when we are looking at an enterprise deployment of ODI. These limitations are addressed with the JEE agent (introduced in version 11g of ODI) that can be installed on WebLogic Server (WLS). The following are benefits of the JEE agent which are inherited directly from Weblogic server:

 

  • High availability: since the execution of the scenarios depends entirely on the agents, it is important that the agent be up and running continuously. This is even more important if the agent is used as a scheduler: if the agent is down when a scenario is scheduled to start, the execution of said scenario will be skipped. Since the JEE agent runs on WebLogic Server, agents can now be deployed on several nodes of the cluster. The schedules can be stored in a Coherence cache to guarantee that even if a node is down, the schedules are always executed. A very good description of the mechanisms involved is available in the Fusion Middleware documentation here: Oracle Fusion Middleware High Availability Guide. Another very good introduction to the concepts from an ODI perspective is available from Sachin Thatte here: https://blogs.oracle.com/dataintegration/entry/setup_of_odi_11g_agents_for_hi
  • Configurable connection pooling: The Standalone agent also uses a UCP (Universal Connection Pool), but its pool parameters cannot be configured. This configuration is possible with WLS for the JEE agent. Creating a DBMS connection is a slow operation. With WLS connection pools, connections are already established and available to the ODI JEE agent. The agents are more efficient since they do not have to keep connecting and disconnecting from the databases. In addition, DBMSs run faster with dedicated connections than if they have to handle incoming connection attempts at run time. Connection pooling also makes it easier to manage and optimize the number of concurrent connections to a system. A very good overview of the mechanisms and benefits of connection pooling is available here: http://docs.oracle.com/cd/E16655_01/java.121/e17659/intro.htm#JJUCP8109
  • Management, Monitoring, Alerting: Oracle Enterprise Manager Cloud Control (OEM) can be used to manage the ODI JEE agents:
  • Agents discovery
  • Configuration management (components version, connection parameters)
  • Monitoring of agents’ health and performance
  • Alerting
  • Trending

Obviously the JEE agent would not be installed on a database server, and since it is running on top of an application server it is constrained to access the resources accessible to that web server. To access files and utilities that would not be accessible from the application server, the JEE agent can be combined with Standalone agents that would provide local access to remote resources.

Lightweight and centrally manageable: the Colocated agent

As specified here: http://docs.oracle.com/middleware/1212/wls/WLDCW/intro.htmA WebLogic domain is the basic administrative unit of WebLogic Server. It consists of one or more WebLogic Server instances, and logically related resources and services that are managed collectively as one unit.” For implementations that require neither high availability for the agents nor connection pooling, but where there is an interest in all other benefits offered by the JEE agent, the Colocated agent is the solution. The only component that is required to run on the machine that hosts the Colocated agent is the WLS Node Manager. Thanks to the Node Manager, the Colocated agent is part of the WLS domain, and as such it benefits from all the features available to elements of the domain, except for those that would require the presence of a Managed Server on the machine. In other words, the Colocated agent is a Standalone agent that can be centrally managed and monitored.

Summary

The following rule-of-thumb can be used to decide when to use the different agents. Keep in mind that you can mix and match the different types of agents within the same environment.

Standalone agent

  • Light install and runtime footprint
  • Requirement to access local files or utilities
  • Optimized network traffic for JDBC connections

Java EE agent

  • High Availability Requirement
  • Configurable connection pooling requirement
  • Colocation with others Fusion middleware components such as SOA or BI
  • Management, monitoring and alerting requirement

Colocated agent

  • Light runtime footprint (The Enterprise Install lays out WLS and the Java Required Files. See Java Required Files Custom WLST Commands for more details on JRF)
  • Need to access local files or utilities
  • Optimized network traffic for JDBC connections
  • Management, monitoring and alerting requirement

The following chart summarizes the features of the different agent types

Agents_Compare

(*): Only if the node manager is installed and running

(#): An Administration Server must be installed in the domain, but can reside on a different machine.

(x): This feature is a function of Oracle Enterprise Manager, not of the agent itself.

(&):Learn about Fusion Middleware Control

References

If you are not familiar with WLS components leveraged by ODI (a lot of the content above refers to the Oracle WebLogic Server documentation) here are some references that will help you with a better understanding of the architecture and benefits of WLS:

  • All WLS Domains must have an Administration Server, which is a central point for managing the domain and to provide access to the WLS administration tools. All changes to configuration and deployment of the applications are done through the Administration Server. A very good introduction is available here.

More Web Logic Server references

ODI agent references

Special thanks to Sachin Thatte, Julien Testut, Benjamin Perez-Goytia and Sandrine Riley for their help on this subject.

For more ODI best practices, tips, tricks, and guidance that the A-Team members gain from real-world experiences working with customers and partners, visit Oracle A-Team Chronicles for ODI.

Add Your Comment