Avoiding Memory Leaks in Exalogic when using the SDP Support from JDK 1.7

Introduction

Although Exalogic is widely used as a consolidation platform due its concept of data center in-a-box, most customers look after it as a technical solution for improving application throughput. Counting with a powerful 40GB/s bandwidth InfiniBand fabric, applications running on top of Exalogic can benefit from the high throughput, low latency and efficient network layer while exchanging messages across different processes.

One of the main benefits of using the InfiniBand technology is providing a way to applications to move data directly from the memory of one server to another, bypassing the operating system of both servers, resulting in significant CPU usage reduction plus latency improvement. But to leverage this capability, all applications must use a specific protocol. The most well-known protocol for this is called SDP (Sockets Direct Protocol) and it provides a transport-agnostic way to support streams sockets over RDMA (Remote Direct Memory Access) network fabrics.

The most common case when SDP need to be leveraged is when applications running on Exalogic need high performance access to Oracle databases running on Exadata. In the Exadata machine, most DBAs only need to set up a new listener based on SDP instead of TCP and that’s it. But in the Exalogic machine things can get a little complicated, especially if you are trying to leverage SDP from the support introduced in JDK 1.7. This article will discuss the issues that can happen when Java applications access SDP-enabled databases in Exadata.

It is important to state that the issues mentioned in this article does not happen when SDP connections between Exalogic and Exadata are established using the built-in stack available in Exalogic, which is essentially the EECS (Exalogic Elastic Cloud Software). Applications intended to run on top of Exalogic must leverage the EECS, and Oracle supports ISVs interested in doing this with the Exastack program. The SDP support available in JDK 1.7 must be used for ISVs interested in building applications for both Exalogic and other InfiniBand-based machines.

The Problem

As you can follow in this article, to leverage SDP in Java applications you need to provide to the JVM a SDP configuration file. In the content of this file, you need to define a set of rules about which endpoints to connect (outbound) or bind (inbound). In case of accessing Oracle databases through SDP-enabled listeners running on the Exadata machine, you need to provide a connect rule just as shown in the listing 1. The rule assumes that the SDP listener is running on port 1522.

connect 192.168.1.102 1522

Listing 1: Connect rule for the SDP configuration file with specific port.

To avoid setting too many rules in the SDP configuration file, some people use wildcards to generalize IP ranges and/or ports. For instance, listing 2 shows a variation of the rule shown in listing 1, in which SDP can be established to any port of the given IP address.

connect 192.168.1.102 *

Listing 2: Connect rule for the SDP configuration file, using wildcards.

Using wildcards in rules of the SDP configuration file would be fine if wasn’t for the fact that multiple protocols can be activated at the same time for a given IP address. For instance, imagine an Exadata machine having database listeners exposed through SDP on port 1522, but also having FAN events listened through TCP on port 6200. This mix of protocols can be a big source of problems in the Exalogic machine if the SDP configuration file uses wildcards. In runtime, for every IP address set in the SDP configuration file, the JVM will try to establish socket connections through SDP, even if the target port does not support the SDP protocol. Due the mismatch of protocols, a continuous set of connection retries will occur, potentially leading to serious memory leak issues.

The Solution

The easiest way to avoid the memory leak issues is avoiding using wildcards in the SDP configuration file. As much as possible, try to set specific ports for each IP address, just as shown in listing 1. There are several tools that you can use in both Exalogic and Exadata to make sure if a specific port supports SDP, as you can see in the “Monitoring SDP Sockets using sdpnetstat on Oracle Linux” section of the following link.

Add Your Comment