IDCS and Weblogic Federation with Virtual Users and Groups

Introduction Federation is a well-known pattern and has been discussed at length on this blog. Almost every vendor or cloud provider out there supports Federation and it’s been around for quite some time now. In this blog post, I will talk about Federation again, but this time in combination with Weblogic’s Virtual Users and Groups. […]

Transport Level Security (TLS) and Java

Know Which Versions of TLS are Supported in Recent Java Versions In the twenty-plus years of the Internet’s interaction with the Secure Sockets Layer (SSL) and Transport Level Security (TLS) protocols, there have been some rough patches.  Over the years, various vulnerabilities, some of them exposed in a laboratory setting and others discovered and exploited […]

Understanding listen ports and addresses to effectively troubleshoot Fusion Applications

Introduction: To communicate with any process, you need to know at least 3 things about it – IP address, port number and protocol. Fusion Applications comprises many running processes. End users communicate directly with only a handful of them, but all processes communicate with other processes to provide necessary services. Understanding various IP addresses and listen ports […]

WebLogic Server: Saving Disk Space in /tmp

Introduction Many WebLogic Server (WLS) implementations use JRockit 28 as the JVM implementation. JRockit 28 comes with the very useful JRockit Flight Recorder which helps in many troubleshooting situations. Problem In high volume WLS implementations with many domains and many managed servers the Flight Recorder could fill up the disk of the temporary file storage […]

Addition by Subtraction – A Quick Win in Performance

Recently, I worked with a customer where we would notice severe performance degradation over time within JCS (Java Cloud Service). The product used was Oracle Service Bus (OSB) where it would take around 2 seconds to process a request and send a response.  Over time, we would see the same request/response take over 1 minute!  If we […]

Avoiding Memory Leaks in Exalogic when using the SDP Support from JDK 1.7

Introduction Although Exalogic is widely used as a consolidation platform due its concept of data center in-a-box, most customers look after it as a technical solution for improving application throughput. Counting with a powerful 40GB/s bandwidth InfiniBand fabric, applications running on top of Exalogic can benefit from the high throughput, low latency and efficient network layer […]

Caching in OSB 12c without Out-Of-Process Coherence Servers

Introduction One of the most popular use cases for Oracle Service Bus (OSB) is the mediation of synchronous service interactions. In this scenario, a client invokes the service through a proxy instead of the actual service endpoint, guaranteeing that the consumer is decoupled from the producer. This type of architecture allows producers to be changed […]

Understanding the use of “WebLogic Plugin Enabled”

Introduction: Requests to a WebLogic Server (WLS) usually go through a web server or a load balancer which serve as a proxy for the client requests. When the WLS requests are “front-ended” by either a web server or a load-balancer, the requests are handled via a plugin. It is important for WLS to be aware of the […]

Interoperability between Microsoft and SOA Suite 12c

Introduction During the design of SOA applications it is inevitable that from time to time you will need to interface with Microsoft-based applications. While technologies like SOAP and REST do a great job when request-reply communication is needed, most people struggle when a messaging-based communication is required. This blog will present two approaches to get messaging working between Microsoft […]

Where did my MDB concurrency go?

Problem Description Recently, there have been numerous customers reporting a similar symptom with regards to MDB (Message Driven Bean) concurrency within WebLogic Server.  The complaint is that not all JMS messages are being consumed in a timely manner and in fact some messages look to be consumed in a serial fashion instead of concurrently. Here is […]

Diagnosing High CPU problems with Java Mission Control

How to diagnose high CPU problems with Java Mission Control? Recently we worked on couple of customer escalations where we have extensively used Java Mission Control to resolve high CPU problems. That prompted me to write an article on Java Flight Recording analysis using Java Mission Control (JMC) In this posting, I’m going to teach you […]

A crude JMS performance comparison between different WebLogic server persistence stores

When it comes to persisting service and subsystems data (i.e. JMS), WebLogic server offers customers a choice: filesystem or relational database using JDBC.  Persistence store has implications on WLS performance and systems management.  In this post, I will provide an unofficial JMS performance comparison using different persistence stores. PLEASE NOTE: This is not an official Oracle performance benchmark.  […]

WebLogic Server 12.1.2 and Coherence 12.1.2: New Technical Whitepapers

Several new technical whitepapers on WebLogic Server 12.1.2 and Coherence 12.1.2 have recently been released. In case you missed them, this post provides links to some of the more interesting ones.  You’ll need to log into Oracle.com to download these.   Oracle WebLogic: 12.1.1 Business Overview – PDF download This whitepaper provides an overview of […]

WebLogic Server Cluster Messaging Protocols

by Robert Patrick and Sabha Parameswaran WebLogic Server clusters form a loosely-federated group of managed servers that provide a model for applications to leverage for achieving scalability, load balancing, and failover. Each managed server manages its view of what servers are in the cluster and its own cluster-wide view of the JNDI tree. The cluster […]

Oracle Dynamic Monitoring Service – metricdump Log

Introduction Have you ever wondered how to control/configure the metricdump log files that accumulate in your WebLogic Server domains configured with Fusion Middleware?  This blog will help to answer that question. Main Article When running WebLogic Server instances configured with Fusion Middleware (e.g., SOA Suite), there are Dynamic Monitoring Service (DMS) metrics that are gathered every 3 […]

Log rotation for WebLogic Server (and friends)

If you have a number of WebLogic Server instances running, or applications that write a lot of information into the logs, you might find that your log files, and your stdout files start to eat up a lot of space … Continue reading

A Universal JMX Client for Weblogic –Part 1: Monitoring BPEL Thread Pools in SOA 11g

Monitoring and optimizing BPEL Thread Pool utilization (and other metrics) is one of the key activities in performance tuning of BPEL/SOA based integrations. Although EM console provides some basic monitoring of the BPEL engine statistics, it is limited regarding the update interval, detail and the recording interval and cannot display historic data. Of course you can setup Grid Control 11g with its Repository, but this is in most cases too complex to setup just for monitoring during performance and load testing.

So, the idea came to create a tool which can easily record these statistics and export them to MS Excel or OpenOffice to create charts for the thread pool utilization over a time period (for example a whole load test execution).

All values of WLS or the SOA engine can be queried using the JMX MBean framework. I have designed the JMXClient to be able configure which MBeans should be queried by using a property file (beans.properties). I decided to connect in this first release to only one Managed Server of WLS to record / export data. This means that if you have a WLS cluster, you need to start multiple JMX Clients to record the values of each node. (In a later release the JMX Client could be optimized to query all nodes automatically).

JMXClient can be used by downloading from the project page at Sourceforge.  (including JDeveloper 11.1.1.6 project and sources)

After that you need to configure

  1. your connection properties, JAVA_HOME and WLS_HOME of  your WLS managed server of SOA in  jmxclient.bat (or jmxclient.sh)
  2. the MBeans names, WLS Server name  and the attributes to record in classes/beans.properties  (you can find the MBean names form the System MBean Browser in EM)

The syntax in jmxclient.bat is

java -cp classes;%INCLUDE_LIBS% jmxclient.JMXClient <server> <user> <password> -monitor 1000

For example

java -cp classes;%INCLUDE_LIBS% jmxclient.JMXClient 192.168.56.101 7001 weblogic welcome1 -monitor 1000

“1000” specifies the interval in milliseconds between the recording.

Then you can run it with

jmxclient > out.txt

Then simple import this text file using Excel or OpenOffice and a comma “,” as delimiter and create a line chart using line 2 as titles and lines 3 to end as data.

Let me first show a couple of results using JMXClient using the properties to record the BPEL thread pool statistics:

The following chart shows a scenario where the invoke thread pool is much too low (20) so that the queue of scheduled invocations waiting for a free thread is growing rapidly:

image

The second example shows a scenario where invoke and callback threads are within normal limits:

image

In the next posts I will show how to use JMXClient to record the BPEL process execution times or the number of messages in the AIA 11g JMS queues by simply exchanging the beans.properties file…..!

Have fun,
Stefan

DISCLAIMER: JMXClient is provided for free use “as is” without any support or warranty. Please provide enhancements or modifications you make yourself. Feedback is welcome using the comments feature of this blog.

PS: for the experts: the format of the beans.properties file:

Every line contains 3 items separated by semicolon:

  1. the name of the MBean to query
  2. the attribute to query
  3. the title string which should be displayed for the column

Example for the bpel thread pools:

oracle.dms:Location=AdminServer,name=/soainfra/engines/bpel/requests/invoke,type=soainfra_bpel_requests;active_maxValue;Invoke Active Max
oracle.dms:Location=AdminServer,name=/soainfra/engines/bpel/requests/invoke,type=soainfra_bpel_requests;scheduled_maxValue;Invoke Scheduled Max
oracle.dms:Location=AdminServer,name=/soainfra/engines/bpel/requests/invoke,type=soainfra_bpel_requests;scheduled_value;Invoke Scheduled Current
oracle.dms:Location=AdminServer,name=/soainfra/engines/bpel/requests/invoke,type=soainfra_bpel_requests;active_value;Invoke Active Value
oracle.dms:Location=AdminServer,name=/soainfra/engines/bpel/requests/invoke,type=soainfra_bpel_requests;threadCount_value;Invoke Threads Value
oracle.dms:Location=AdminServer,name=/soainfra/engines/bpel/requests/invoke,type=soainfra_bpel_requests;threadCount_maxValue;Invoke Threads Max
oracle.dms:Location=AdminServer,name=/soainfra/engines/bpel/requests/system,type=soainfra_bpel_requests;active_maxValue;System Active Max
oracle.dms:Location=AdminServer,name=/soainfra/engines/bpel/requests/system,type=soainfra_bpel_requests;scheduled_maxValue;System Scheduled Max
oracle.dms:Location=AdminServer,name=/soainfra/engines/bpel/requests/system,type=soainfra_bpel_requests;scheduled_value;System Scheduled Current
oracle.dms:Location=AdminServer,name=/soainfra/engines/bpel/requests/system,type=soainfra_bpel_requests;active_value;System Active value
oracle.dms:Location=AdminServer,name=/soainfra/engines/bpel/requests/system,type=soainfra_bpel_requests;threadCount_value;Invoke Threads Value
oracle.dms:Location=AdminServer,name=/soainfra/engines/bpel/requests/system,type=soainfra_bpel_requests;threadCount_maxValue;Invoke Threads Max
oracle.dms:Location=AdminServer,name=/soainfra/engines/bpel/requests/engine,type=soainfra_bpel_requests;active_maxValue;Engine Active Max
oracle.dms:Location=AdminServer,name=/soainfra/engines/bpel/requests/engine,type=soainfra_bpel_requests;scheduled_maxValue;Engine Scheduled Max
oracle.dms:Location=AdminServer,name=/soainfra/engines/bpel/requests/engine,type=soainfra_bpel_requests;scheduled_value;Engine Scheduled Current
oracle.dms:Location=AdminServer,name=/soainfra/engines/bpel/requests/engine,type=soainfra_bpel_requests;active_value;Engine Active value
oracle.dms:Location=AdminServer,name=/soainfra/engines/bpel/requests/engine,type=soainfra_bpel_requests;threadCount_value;Engine Threads Value
oracle.dms:Location=AdminServer,name=/soainfra/engines/bpel/requests/engine,type=soainfra_bpel_requests;threadCount_maxValue;Engine Threads Max
oracle.dms:Location=AdminServer,name=/soainfra/engines/bpel/requests/audit,type=soainfra_bpel_requests;active_maxValue;Audit Active Max
oracle.dms:Location=AdminServer,name=/soainfra/engines/bpel/requests/audit,type=soainfra_bpel_requests;scheduled_maxValue;Audit Scheduled Max
oracle.dms:Location=AdminServer,name=/soainfra/engines/bpel/requests/audit,type=soainfra_bpel_requests;scheduled_value;Audit Scheduled Max
oracle.dms:Location=AdminServer,name=/soainfra/engines/bpel/requests/audit,type=soainfra_bpel_requests;active_value;Audit Active value
oracle.dms:Location=AdminServer,name=/soainfra/engines/bpel/requests/audit,type=soainfra_bpel_requests;threadCount_value;Audit Threads Value
oracle.dms:Location=AdminServer,name=/soainfra/engines/bpel/requests/audit,type=soainfra_bpel_requests;threadCount_maxValue;Audit Threads Max

Starting a cluster

Recently, I have been involved in a number of discussions with people who are setting up clusters of various Fusion Middleware products, often on an Exalogic machine.  These discussions have led me to feel that it would be worth sharing … Continue reading

ThreadLogic version 0.95 available

  ThreadLogic version 0.95 is available for public download now. Additional Features Biggest change is support for externalizing the Advisories and group definitions. Users can use the pre-defined AdvisoryMap.xml to come up with custom …

Introducing ThreadLogic

Eric Gross and Sabha Parameswaran, from Oracle FMW Architects Team (The A-team) are happy to introduce ThreadLogic, an open source Thread Dump Analyzer tool, to Middleware community.Motivation behind ThreadLogic  The current set of TDA tools (Sam…