Content validated on 12/18/2020 with
ODI Version 12.2.1.4.200304.2238
ATP Version Oracle Database 19c Enterprise Edition Release - Production Version 19.5.0.0.0
This blog walks though how to configure Oracle Data Integrator (ODI) on Oracle Cloud Marketplace Agents in High Availability (HA) Mode with the Autonomous Transaction Processing (ATP) Database to hold the ODI repository.
To allow for a true application HA, the agents configured throughout this blog will be J2EE agents deployed in a WebLogic domain cluster of managed servers . For a reminder on the different types of agents that can be configure with ODI, please see this blog.
Suggested Prerequisite Reading Material:
* Using Oracle Cloud Marketplace
* Configuring High Availability for ODI
Overall prerequisites
* Follow the steps in the first blog for instructions on:
Policies Assignment (Step 1)
SSH key-pair Generation (Step 4-b and 4-c)
ODI Marketplace Provisioning (Step 2)
User/Policy/Key Requirements:
* It is strongly recommend to configure OCI and ODI with a local (non-Federated) user.
The following diagram describes the different components of the ODI Marketplace HA architecture :
Figure 1: ODI Agents HA Architecture
* The OCI foundations, including the tenancy; a compartment within the tenancy, the network elements such as a Virtual Cloud Network (VNC), a subnet with the proper security lists (Egress/Ingress rules) and routing table, and finally the public load balancer (primary and standby) with its listener, back end server set and health check rules;
* The ATP database to support the ODI Repository and the WebLogic domain schemas : the Metadata Services schema (MDS), the Audit Services schemas (IAU), the Oracle Platform Security Services schema (OPSS), the WebLogic Services schema (WLS), the ODI Repository schemas (ODI Master and Work) and the Common Infrastructure Services (STB) schema . For more details on the WebLogic domain schema, please refer to Understanding Repository Creation Utility Schemas, IDs and TableSpace;
* The two ODI Marketplace compute instances, provisioned in two different availability domains and that will host the two J2EE agents. Both ODI instances are provisioned within a public subnet with both private and public IPs. By default, the ODI Marketplace instance firewall is enabled and configured to block all traffic. Therefore additional firewall configuration is required;
* The WebLogic domain that consists of two WebLogic Managed Servers instances, one on each of the ODI Marketplace compute instances. The Managed Servers are grouped in a WebLogic cluster, in which the ODI J2EE agents application are deployed. They are managed with a single Administration server running on the first ODI Marketplace compute instance. The WebLogic domain here will also host the Coherence cluster. To avoid duplicate ODI schedule processing, only one of the ODI agents behaves as a scheduler. A Coherence cache is used to handle scheduler service uniqueness and migration;
* The two Node Managers java utilities running on each ODI Marketplace compute instance as a separate process from the Managed or Admin Servers. The Node Managers allows to perform common operations tasks for a Managed Server (start, stop,…) regardless of its location with respect to its Admin server. These utilities are not represented explicitly on the diagram though.
Setting up HA ODI Agents on a Marketplace context can be summarized in twelves high level steps:
For a successful set up, some of the twelves steps above need some more granular level of description. There are indeed some common blockers that one can face during the process. This section will go over five of these main common potholes:
When starting the Managed Servers, one may hit the waring below, pertaining to Coherence Death detect port being closed.
#### <> <> <64568a5f-5798-46f2-9b2e-bf961a395ccd-00000007> <1586895917456> <[severity-value: 16] [rid: 0] [partition-id: 0] [partition-name: DOMAIN] > <2020-04-14 20:25:17.456/121.718 Oracle Coherence GE 12.2.1.4.0 (thread=Cluster, member=n/a): Delaying formation of a new cluster; IpMonitor failed to verify the reachability of senior Member(Id=1, Timestamp=2020-04-13 18:54:41.545, Address=10.0.17.7:32783, MachineId=7318, Location=site:app.odiu6pmpcnx.oraclevcn.com,machine:odi-node1,process:19998,member:ODI_server1, Role=ODI_cluster1) via /10.0.17.8; if this persists it is likely the result of a local or remote firewall rule blocking either ICMP pings, or connections to TCP port 7>
Having this port closed may create subsequent issues, specifically it may prevent the Coherence cache to handle scheduler service uniqueness and migration correctly.
Therefore, TCP Port 7 needs to be open in the security list (Ingress rules) as described below
Stateless |
Source |
IP Protocol |
Source Port Range |
Destination Port Range |
Type and Code |
Allows |
Comments |
No |
0.0.0.0/0 |
TCP |
All |
7 |
Nil |
TCP traffic for ports : 7 ECHO |
Coherence TcpRing/IpMonitor death detection feature. |
Table 1: TCP port 7 ingress rule
Also, the firewall rules needs to be amended with the rule below, on each compute instance hosting the ODI Agent Managed Servers.
sudo firewall-cmd --permanent --new-service=odicoherencedeathdetect
sudo firewall-cmd --permanent --service=odicoherencedeathdetect --set-description="ODI Coherence Cluster TCP Ring"
sudo firewall-cmd --permanent --service=odicoherencedeathdetect --add-port=7/tcp
sudo firewall-cmd --permanent --add-service=odicoherencedeathdetect
sudo firewall-cmd --reload
sudo firewall-cmd --list-all
To allow the load balancing to work properly, HTTP Traffic needs to be enabled between the two ODI Marketplace compute instances. This translate into updating the security list of the VCN (Ingress rules) and the firewall rules on each of ODI Marketplace compute instances, as described respectively in the table and command below.
Stateless |
Source |
IP Protocol |
Source Port Range |
Destination Port Range |
Type and Code |
Allows |
Comments |
No |
0.0.0.0/24 |
TCP |
All |
80 |
Nil |
TCP traffic for ports : 80 |
OCI Load balancer |
No |
|
TCP |
All |
80 |
Nil |
TCP traffic for ports : 80 |
OCI Load balancer |
Table 2: TCP port 80 ingress rule
sudo firewall-cmd --permanent --zone=public --add-service=http
sudo firewall-cmd --reload
When running the configuration script to create the ODI domain, special care needs to be applied when entering the JDBC connection properties for the different RCU schemas.
When filling the STB Schema details, make sure to only check the LocalSvcTBl Schema. Then Connection Properties. The Connection Properties screen displays the following details:
oracle.net.authentication_service TCPS
oracle.net.ssl_server_dn_match false
javax.net.ssl.trustStore /u01/oracle/mwh/wallet/cwallet.sso (Path of the wallet location)
javax.net.ssl.keyStoreType SSO
javax.net.ssl.keyStore /u01/oracle/mwh/wallet/cwallet.sso(Path of the wallet location)
javax.net.ssl.trustStoreType SSO
Figure 2: ODI domain LocalSvcTBl Schema JDBC connection properties
Then all the other schemas can be selected before clicking Connection Properties. Then the connection properties described above can be filled it at once for all schemas but the LocalSvcTBl Schema.
Figure 3: ODI domain other schemas JDBC connection properties
At the time of writing this blog, these properties are described in a different section than the HA section in the ODI Marketplace documentation. For further details, please see here.
A common mistake when creating the health check rule at the load balancer level is to forget the forward slash “/” at the end of the URL.
Figure 4: Load balancer Health Check URL Path missing “/”
Doing so will result in getting 502 Bad Gateway Errors when accessing the load balanced ODI Agent URL.
Figure 5: Bad Gateway error
So when updating the load balancer health check, please make sure to add the forward slash “/” at the end of the URL Path, as depicted in the below figure.
Figure 6: Load balancer Health Check URL Path with “/”
The OCI Threat and Vulnerability Management Team (TVM) was notified that there have been successful exploits against Oracle WebLogic Server leveraging Java Deserialization issue (CVE-2020-2883). TVM will identify the IPs in a tenancy using T3 protocol that are vulnerable to this CVE.
To address this issue, TVM recommends to identify all instances within your organization which may be using the T3 protocol and close the affected ports (Default ports 7001 and 7002).
If one would like to continue using T3 protocol, the recommendation is to block ports 7001, 7002 and any other ports using T3 protocol from external access and make sure they are accessible via Oracle VPN only using T3 over SSL (T3S).
In a next blog, ATP Roadmap for HA and Disaster recovery will be discussed, as well as configuring HTTPS and SSL configuration with a private load balancers, ODI Marketplace instances in a private network and ATP with Private Endpoint, to fulfill ODI Marketplace HA Enterprise Deployments.
Click here for more A-Team Oracle Data Integrator (ODI) Blogs
Click here to sign up to the RSS feed to receive notifications for when new A-team blogs are published
Click here to access the ODI Marketplace documentation library
This article walked through the steps to configure Oracle Data Integrator (ODI) on Oracle Cloud Marketplace Agents in High Availability (HA) Mode with the Autonomous Transaction Processing (ATP) Database to hold the ODI repository.
Stay tuned for more to come!
Bookmark this post to stay up-to-date on changes made to this blog as our products evolve.
The Oracle A-Team is a central, outbound, highly technical team of enterprise architects, solution specialists, and software engineers.
The Oracle A-Team works with external customers and Oracle partners around the globe to provide guidance on implementation best practices, architecture design reviews, troubleshooting, and how to use Oracle products to solve customer business challenges.
I focus on data integration, data warehousing, Big Data, cloud services, and analytics (BI) products. My role included acting as the subject-matter expert on Oracle Data Integration and Analytics products and cloud services such as Oracle Data Integrator (ODI), and Oracle Analytics Cloud (OAC, OA For Fusion Apps, OAX).
Previous Post