This article discusses how to configure Oracle Data Integrator (ODI) for Oracle Big Data Cloud (BDC) using the ODI Standalone installation. ODI offers out of the box integration with Big Data technologies such as Apache Hadoop, Apache Spark, Apache Hive, Apache Pig, and Apache Kafka, among others. ODI supports both distributions of Hadoop: Hortonworks Data Platform (HDP), and Cloudera Enterprise Data Hub (CDH). Additionally, ODI can also be used on other distributions of Hadoop such as Amazon Elastic MapReduce (EMR).
For additional information on how to use ODI with BDC, go to “Using Oracle Data Integrator with Oracle Big Data Cloud.” A pre-recorded live demonstration that supports this discussion can be found at the following Oracle Data Integration webcast: “Mastering Oracle Data Integrator with Big Data Cloud.”
In order to use ODI with BDC, users can install and configure ODI in one of two ways: ODI Standalone or ODI with High-Availability. The ODI Standalone configuration requires the installation and configuration of the ODI Standalone agent in an instance of BDC. The ODI with High-Availability configuration is an extension of the ODI Standalone configuration, but it uses the ODI J2EE agent as an orchestrator for Big Data workloads. The following sections of this article provide a guideline for installing and configuring ODI on BDC using the ODI Standalone configuration.
For additional information on how to install and configure ODI for BDC using the ODI with High-Availability configuration, go to “Configuring Oracle Data Integrator for Big Data Cloud: High-Availability Configuration.”
The ODI Standalone Configuration for Big Data requires that the ODI Standalone agent be hosted on the BDC cluster. The ODI Standalone agent is installed and configured as a standalone lightweight Java application and it is hosted on the master node of the DBC cluster. The ODI Standalone agent uses an ODI repository installed on an instance of the Oracle Database Cloud Service (DBCS). Figure 1, below, illustrates this configuration:
Figure 1 – Configuring ODI Standalone for Big Data Cloud
To install and configure ODI for BDC, using the ODI Standalone configuration, follow these steps:
To host an ODI repository, users must provision an instance of a SQL database. Oracle Cloud offers MySQL Cloud Service and Oracle Database Cloud Service. The following instructions use the Oracle Database Cloud Service to host the ODI repository:
The ODI Studio is the user interface that ODI offers to perform the ETL development. It is recommended to perform this development in a compute resource such as Oracle Compute Classic. By installing ODI Studio on Oracle Compute Classic, ODI users have the flexibility of having an ODI Studio installation that is independent of the ODI agent installation, and it provides more scalability when more developers are added into the ETL project. Use the following instructions to provision an instance of Oracle Compute Classic and install the ODI Studio on this instance:
Use the following instructions to provision an instance of BDC and install the ODI Standalone agent:
The ODI High-Availability configuration requires additional access rules between cloud instances, so the instances can communicate with each other. For instance, the ODI Standalone agent, on BDC, must access the ODI repository on DBCS. Thus, follow these instructions in order to configure access rules between cloud instances:
Once the access rules have been configured, users can launch the ODI Studio on Compute Classic, and start their ETL development work.
ODI offers out of the box integration with Big Data technologies such as Apache Hadoop, Apache Spark, Apache Hive, and Apache Pig, among others. ODI supports both distributions of Hadoop: Hortonworks Data Platform (HDP), and Cloudera Enterprise Data Hub (CDH). Additionally, ODI can also be used on other distributions of Hadoop such as Amazon Elastic MapReduce (EMR). This article discussed how to configure ODI with BDC using the ODI Standalone configuration.
For more Oracle Data Integrator best practices, tips, tricks, and guidance that the A-Team members gain from real-world experiences working with customers and partners, visit “Oracle A-team Chronicles for Oracle Data Integrator (ODI).”