Oracle GoldenGate: How to Force OGG Monitor Server to Use a Specific Network Address to Communicate to a Multihomed Monitor Agent

Introduction

A multihomed machine is a type of host that uses multiple interfaces and contains two or more network addresses that can be attached to the same LAN segment or a separate LAN segment. Multihoming is a process or a technique widely used to eliminate single point of failure on the network, and provides alternate data transmission paths in the event of a path failure.

Having a multihomed host, sometimes can present an issue with an application that uses or expects to communicate with a specific network interface or network address.

In this article we shall present a way to force OGG Monitor Server to use a specific network address to communicate to a Monitor Agent running on a multihomed machine.

Main Article

In a multihomed server environment where the OGG Monitor agent runs, when each interface contains a separate IP addresses it can sometimes cause connectivity issues with the OGG Monitor server.

Here’s an architecture diagram of Oracle GoldenGate Monitor:

OGG Monitor Architecture

 

As you can see in the above diagram, there’s a 2-way communication between the Monitor Agent and the Monitor Server.

The communication between the Monitor agent and the Monitor server is initially started by the agent.  When Oracle GoldenGate (OGG) processes get started, the agent registers with the OGG Monitor server. The server uses the information provided by the agent to communicate back to the agent host to get its updates on the OGG processes.

Here’s the sequence of OGG Monitor Agent/Server’s communication flow:

1. Monitor agent initially communicates to Monitor server and registers its OGG instance and components to be monitored. One of the pieces of information that gets passed to the Monitor server during this initial communication is the host name of the machine where Monitor agent is running

2. Monitor server registers this information in the Monitor repository database and communicates back to the agent via the host name that was provided. This communication instructs the agent to create a new connection to be used by the server to get its updates.

3. Monitor agent creates a new socket connection and sends a message to the Monitor server containing the connectivity information. Included in this message is the host IP address to be used by Monitor server to connect and start getting updates.

4. Monitor server connects to that new connection using the connectivity information that was passed and starts getting updates from the Monitor agent.

 

Here’s what a working OGG Monitor Environment would look like on the Web UI when the server and agent are communicating with each other:

working_mon

Multihomed Host

If you have multiple interfaces with different IP addresses which is the case for a multihomed machine, there’s a chance that the agent might give the IP address that you don’t want to be used by the Monitor server process. This can cause a possible connectivity issue. Usually the indication of this type of connectivity issue will manifest itself on the Monitor Web UI by having the OGG components grayed out and not being updated.

Here’s what an OGG Monitor Environment with connectivity issue would look like on the Web UI when there’s a connectivity issue between Server and Agent:

non_working_mon

To avoid this possible issue, we need to force the IP address that we want Monitor server to use to connect to the agent process.

There are two ways of achieving this objective. The first one is a little bit intrusive as you will need an Administrator/root privileges to modify the host configuration file (e.g. /etc/hosts) to point to the right IP address. The second approach which is less intrusive and the recommended approach, is to use a little less known parameter that you can add to the Java command when it starts the Monitor Java Agent.

Sample Scenario

Let’s use a host that has two interfaces to illustrate the steps involve for each approach. For this example we will use slc08ggk as the host machine.

Here’s the output of an ifconfig command that shows the two separate interfaces on slc08ggk:

$ (slc08ggk)[a11204s] /ogg_ateam/mpapio\> ifconfig
eth0 Link encap:Ethernet HWaddr 00:21:F6:00:00:02
inet addr:10.245.109.154 Bcast:10.245.111.255 Mask:255.255.248.0
inet6 addr: 2606:b400:2010:6852:221:f6ff:fe00:2/64 Scope:Global
inet6 addr: fe80::221:f6ff:fe00:2/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:585782 errors:0 dropped:0 overruns:0 frame:0
TX packets:15723 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:50479129 (48.1 MiB) TX bytes:14862139 (14.1 MiB)
Interrupt:37

eth1 Link encap:Ethernet HWaddr 00:21:F6:00:00:15
inet addr:192.168.120.154 Bcast:192.168.120.255 Mask:255.255.255.0
inet6 addr: 2606:b400:2010:6852:221:f6ff:fe00:15/64 Scope:Global
inet6 addr: fe80::221:f6ff:fe00:15/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:564256 errors:0 dropped:0 overruns:0 frame:0
TX packets:141 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:48949154 (46.6 MiB) TX bytes:11558 (11.2 KiB)
Interrupt:38

The first interface is eth0 which has the IP address of 10.245.109.154. The second interface is eth1 which has the IP address of 192.168.120.154. In this sample host configuration, eth1 is being used as the management interface which means that it can’t communicate to the rest of the world. The interface to use for public usage is eth0.

Here’s the content of the host configuration (/etc/hosts) file for slc08ggk:

# -- SLC08GGK HOST File 
# ::1 localhost localhost.localdomain localhost6
# 127.0.0.1 slc08ggk localhost localhost.localdomain localhost4a
127.0.0.1 localhost localhost.localdomain localhost4a
192.168.120.154 slc08ggk slc08ggk.us.oracle.com
10.245.109.154 slc08ggk slc08ggk.us.oracle.com

Now in the /etc/hosts file, the first entry that contains the hostname slc08ggk contains the IP address (192.168.120.154) for eth1, and by default this will be the IP address that will be returned by the agent to the Monitor server.

Force IP Address

To use the 1st approach to force the IP address to be used by Monitor server, we need to modify the host configuration file. We would need to do the following steps:

1. Stop Monitor Agent

2. Modify the /etc/hosts file and move the IP address of eth0 as the first entry that contains the slc08ggk host.

 

Here’s the content of the modified /etc/host file:

# -- SLC08GGK HOST File
 # ::1 localhost localhost.localdomain localhost6
 # 127.0.0.1 slc08ggk localhost localhost.localdomain localhost4a
 127.0.0.1 localhost localhost.localdomain localhost4a
 10.245.109.154 slc08ggk slc08ggk.us.oracle.com
 192.168.120.154 slc08ggk slc08ggk.us.oracle.com

 

3. Start Monitor Agent

 

To use the 2nd approach which is less intrusive and the preferred way, we need to modify the parameter file for the Monitor Agent (jagent.prm) to include the optional parameter “-Djava.rmi.server.hostname“. It needs to be set to the correct hostname or IP address that you want to use for the Monitor Server & Agent communication.

This parameter overrides the local server IP address that’s being passed by the Monitor agent to the Monitor server. With this parameter the host resolution or host lookup now takes place on the Monitor server not the Monitor agent anymore.

You would need to add the parameter anywhere after the “java” command in the file.

Here are the steps to implementing the 2nd approach:

1. Stop Monitor Agent

2. Modify the Monitor Agent paramter file (jagent.prm) and add the -Djava.rmi.server.hostname parameter with the correct hostname or IP address.

 

Here’s a sample of the original Monitor Agent parameter file – jagent.prm (Note: The content is all in one line):

COMMAND java -Dconfig.dir=/ogg_ateam/mpapio/ogg/core/v1121025_ora11g_A/mon_agent/cfg -Djava.util.logging.config.class=oracle.core.ojdl.logging.LoggingConfiguration -Doracle.core.ojdl.logging.config.file=/ogg_ateam/mpapio/ogg/core/v1121025_ora11g_A/mon_agent/cfg/logging-config.xml -Doracle.core.ojdl.logging.componentId=JAGENT -jar -Xms512m -Xmx1024m /u02/app/oracle/product/wls/oggmon/ogg_agent/dirjar/jagent.jar

Here’s what the modified Monitor Agent parameter file – jagent.prm (Note: The content is all in one line):

COMMAND java -Djava.rmi.server.hostname=slc08ggk -Dconfig.dir=/ogg_ateam/mpapio/ogg/core/v1121025_ora11g_A/mon_agent/cfg -Djava.util.logging.config.class=oracle.core.ojdl.logging.LoggingConfiguration -Doracle.core.ojdl.logging.config.file=/ogg_ateam/mpapio/ogg/core/v1121025_ora11g_A/mon_agent/cfg/logging-config.xml -Doracle.core.ojdl.logging.componentId=JAGENT -jar -Xms512m -Xmx1024m /u02/app/oracle/product/wls/oggmon/ogg_agent/dirjar/jagent.jar

 

3. Start Monitor Agent

 

Summary

In this article we showed two possible approaches to use to force OGG Monitor Server to use a specific IP address to communicate to Monitor Agent. It is recommended to use the less intrusive approach which is via the use of an additional java parameter in the Monitor agent parameter file.

Add Your Comment