Patching Fusion Applications – What Types of Patches Exist and How Often Should They Be Applied

Introduction When deploying Fusion Applications, or any suite of software for that matter, careful thought and planning must go into understanding the life cycle management of that product, and in particular, choosing the right cadence for patching.  In this article we will discuss the types of patching required for Fusion Applications, as well as the […]

Validating the Fusion Applications Security Components During Installations and Upgrades

Introduction   When installing or upgrading Fusion Applications, it is necessary to validate the security components to ensure that they are functioning correctly. This article provides a list of tasks that can be performed to accomplish this. The order of tasks below follow the dependency that the components have on each other so that if […]

Adding Oracle Identity Federation to an Existing Fusion Applications Deployment Part 1

Introduction This guide is meant for existing FA customers who have deployed FA without OIF and who now wish to add this security component to the deployment to provide federated SSO to FA. Customers who have not yet begun their deployment can and should follow the Oracle® Fusion Middleware Enterprise Deployment Guide for Oracle Identity […]

Adding Oracle Identity Federation to an Existing Fusion Applications Deployment Part 2

Introduction This is the second part of a two-part article. Click here to view Part 1. This guide is meant for existing FA customers who have deployed FA without OIF and who now wish to add this security component to the deployment to provide federated SSO to FA. Customers who have not yet begun their deployment […]

Expiration Checklist for Fusion Applications

Two main things when expired will significantly affect the operations of Fusion Applications. These are database passwords and certificates. As such these expiration dates need to be checked and maintained properly.

Check for expiring database account passwords

Fusion Applications have many schema users in the Fusion Application database.  Many of these schema users by default have no expiry date, however some do.  You can check the expiration date for these passwords using sqlplus and connecting to the FA database as sys.  Use the following command to check the expiry_date:
select username, account_status, expiry_date, sysdate from dba_users where expiry_date is not null;
TODO:  Keep track of when database accounts will expire.  When the database accounts will soon expire, update the accounts and reset the expiry_date according to your established corporate security policy requirements.  Note: You can reuse the existing password when resetting these schema accounts.

Check for expiring certificates

Fusion Application will fail when certificates expire.  It’s important to check all certificate stores (JKS for WebLogic and PKCS#12 for OHS) for expiring keys and certificates so that they can be renewed in a controlled and timely manner.


For Fusion JKS Certificates Stores

You should maintain a list of all certificate stores so that they can be located easily.  
The fusion jks stores are fusion_trust.jksand <hostname>_fusion_identity.jks in APPLICATIONS_BASE/fusionapps/wlserver_10.3/server/lib
For each JKS store, use keytool to examine the contents, noting the expiration date for each key and certificate:
$JAVA_HOME/bin/keytool -list -v -keystore <keystore filename>


Note:  fusion_trust.jks contains the keys and certificates in each of the <hostname>_fusion_identity.jks.  When replacing the key and certificates, you must replace each <hostname>_fusion_identity.jks and fusion_trust.jks separately.

For Webgate Certificate

You should note down the expiration date of the webgate certificate and replace them as appropriate.  The webgate certificate is in APPLICATIONS_CONFIG/CommonDomain_webtier/config/OHS/ohs1/webgate/config/simple. To check the certificate expiration date, use keytool to examine the contents:

$JAVA_HOME/bin/keytool -printcert -v -file aaa_cert.pem



For PKCS#12 Certificates Stores

The location of the certificate stores used by FA OHS instances can be found in the OHS configuration files. The following example shows how to determine this:
cd APPLICATIONS_CONFIG/CommonDomain_webtier/config/OHS/ohs1


cat *.conf ./moduleconf/*.conf | grep SSLWallet filename
Each of these should be opened with the orapki utility to examine the content and verify the certificate expiration. The orapki utility is described in detail here:



Tips and Tricks When Upgrading IDM for Fusion Applications – RUP1 through RUP3

In this article, I want to share some lessons learned from doing the IDM portion of the FA upgrade. Overall, the process of upgrading the security components is straightforward if you follow the instructions carefully, but there’s no denying that the IDM part of the upgrade is a little more hands-on and potentially confusing that the FA part.

A Note About Versions

When you start reading the documentation below, you may notice like I did that the naming convention for the upgrades is changing — up to now, the upgrades have been referred to RUP1, RUP2, etc., but the documentation no longer reflects that:
Version              Old Name                   New Name          FA 11gR1 RUP1         FA 11gR1 Update 1          FA 11gR1 RUP2         FA 11gR1 Update 2          FA 11gR1 RUP3         FA 11gR1 Update 3

You will see references to both naming conventions, so you should keep both in mind when searching in My Oracle Support or in your favorite search engine for information.
Important Note: These are incremental and not cumulative patches. You will need to apply RUP1 before you can apply RUP2, and the same goes for RUP3 (RUP1 and RUP2 need to be applied first).

The Documentation

Starting with RUP2/Update 2, the documentation you need to upgrade consists of three separate docs:

For RUP1/Update 1:
Release Notes: MOS Document 1382781.1
Patching Guide:

For RUP2/Update 2:
Release Notes: MOS Document 1439014.1
Patching Guide:
IDM Upgrade Procedure: MOS Document 1435333.1

For RUP3/Update 3:
Release Notes: MOS Document 1455116.1
Patching Guide:
IDM Upgrade Procedure: MOS Document 1441704.1

You should always start with the release notes before looking at the patching guide. The IDM upgrade procedure is a complement to the patching guide — while following the patching guide for the upgrade you’re performing, it will refer you to the IDM upgrade doc, which will then refer you back to the patching guide to continue.
Important Note: These docs have all been updated over time — make sure you check online before starting your upgrade to make sure that you have the most recent version!

Upgrading IDM from RUP1/Update 1 to RUP2/Update 2

Most of this consists of patching, but there are a couple of major changes on the IDM side, namely, the upgrade of the IDM Suite to from which in turn requires the upgrade of WLS in the IDM domain to 10.3.6.

Things You Should Do Before You Start the Upgrade

As of the time I’m publishing this, the RUP2 binaries on eDelivery come with the wrong copy of the IDM Suite installer (version instead of the required This means you will need to separately download the correct version from eDelivery. Once you have everything downloaded and ready to go, it’s worth going through the binaries to identify where the IDM-specific patches are so you don’t have to search for them later. In my case, everything was unpacked to “/u01/backup/rup2” on the machine where I first tried this:

Patch                Location
13797139        /u01/backup/rup2/installers/oracle_common/patch/13797139
13642895        /u01/backup/rup2/installers/oracle_common/patch/13642895
13686287        /u01/backup/rup2/installers/oracle_common/patch/13686287
13579026        /u01/backup/rup2/installers/oracle_common/patch/13579026
13782459        /u01/backup/rup2/installers/pltsec/patch/13782459
13620505        /u01/backup/rup2/installers/pltsec/patch/13620505
13399365        /u01/backup/rup2/installers/iam_patches/13399365
13115859        /u01/backup/rup2/installers/iam_patches/13115859
13684834        /u01/backup/rup2/installers/iam_patches/13684834
13477091        /u01/backup/rup2/installers/webgate/ext

Note that the last patch (13477091) is an OAM 10g patch for WebGate and is not applied with OPatch but run as a standalone executable.
In a similar vein, the doc references some common paths, and the example paths in the doc do not reflect the paths that are specified in the Enterprise Deployment Guide. I went through the paths on one of my lab servers which does follow the EDG:

IDM_ORACLE_HOME                             /u01/app/oracle/product/fmw/idm
OID_ORACLE_HOME                             /u01/app/oracle/product/fmw/idm
IDM_ORACLE_COMMON_HOME        /u01/app/oracle/product/fmw/oracle_common
IAM_ORACLE_HOME                            /u01/app/oracle/product/fmw/iam
OIM_ORACLE_HOME                            /u01/app/oracle/product/fmw/iam
IAM_ORACLE_COMMON_HOME       /u01/app/oracle/product/fmw/oracle_common
SOA_ORACLE_HOME                           /u01/app/oracle/product/fmw/soa
OHS_ORACLE_HOME                           /u01/app/oracle/product/fmw/web
WEB_ORACLE_HOME                          /u01/app/oracle/product/fmw/web
OHS_WEBGATE_ORACLE_HOME      /u01/app/oracle/product/fmw/oam/webgate
OHS_ORACLE_COMMON_HOME      /u01/app/oracle/product/fmw/oracle_common

Again, this is an exercise worth doing before you begin to make the upgrade steps go a little more quickly.

Following the Upgrade Doc

The upgrade doc divides the tasks into 13 steps, and I will highlight some specific comments below for the steps where I encountered an issue or where I felt that some clarification was warranted. For example, there are four patches (13797139, 13642895, 13686287 and 13579026) that you are instructed to apply in Step 6 and then again in Steps 7 and 8 and this made no sense at first. On a closer read, however, it became clear that the doc was covering a very generic deployment where the IDM Suite, the IAM Suite and the Web Tier were on separate WebLogic domains. And that’s not the case for FA installations that follow the EDG. So you really only need to apply the four patches in Step 7 because the WLS-based security components are all in one domain.

Step 4: Create Backups

Do not neglect this step! Upgrading FA is a large and complex effort.

Step 5: Download Required Patches

This step was a bit of a puzzle for me, because it belongs in the “Before Upgrading” section before Step 1. Besides, with the exception of the IDM Suite, everything else is included in the RUP2 binaries that you download from eDelivery.

Step 6: Upgrade the IDM Node

There should be no drama in this section, but remember from above that you can wait until Step 7 to apply those four patches. If you don’t, you will need to roll back and reapply those patches in the next step.

 Step 7: Upgrade the IAM Node

Remember that if you followed the EDG, the IDM and IAM nodes are one and the same and you have already upgraded WLS (and don’t need to do it again).

Apply Oracle Identity Manager patch 13399365

There are two things that happened to me when I tried this the first time. The first thing was an error from OPatch saying that this patch conflicts with patch 12961473. Turns out that this is a known issue and the fix is to simply roll back patch 13399365 and then reapply it. The second thing was deploying the new weblogic.profile file. There were some unfamiliar parameters that needed to be given values, and to save you some time, here is a copy of what it should look like for FA:

# For passwords if you dont want to put password </optional> in this file just comment it out from here, you will be promted for it in rumtime.
#Neccessary env variables [Mandatory]
#DB configuration variables [Mandatory]
# Database password is optional. if you want to  give it on terminal itself leave it commented. Otherwise uncomment it.
# If you have milty-tenancy enabled in your environment
mdsDB.user=<MDS DB Schema owner>
#Password is optional,  if you want to  give it on terminal itself leave it commented. Otherwise uncomment it.
#mdsDB.password=<MDS DB Schema password><MDS DB Host>
mdsDB.port=<MDS DB port>
mdsDB.serviceName=<MDS DB ServiceName>
#For domain level configurations [Mandatory]
# put here your admin server related credentials
#Password is optional,  if you want to  give it on terminal itself leave it commented. Otherwise uncomment it.
#oim specific domain level parameters [Mandatory]
#SOA specific details [Mandatory]
#put here the name of the targets of taskdetails. in non cluster it will be soa server name and in cluster it will be something like cluster_soa
#Following params is needed only if you have enabled OHS in your env
#If your env is FA, you can set this var false or ignore this if your env is non FA.

The script is in the same directory as weblogic.profile. You may see an error in the last step that it runs, and this seems to be due to the fact that the script tries to deploy SOA stuff before the server is fully up. This is another known bug, and the easiest fix is to check for undeployed apps in the WebLogic console and manually deploy them.

Upgrading IDM from RUP2/Update 2 to RUP3/Update 3

Most of this consists of patching, but there are a couple of changes on the IDM side, namely, the upgrade of the WebGates to 11g from 10g. The other thing (and it’s not quite spelled out in the doc) is that the FA database is upgraded to, and while this is mandatory for the FA DB it is optional for the IDM DB. For that reason alone, if you are going to upgrade the IDM DB as well, I recommend that you wait until the overall upgrade is complete and smoke tested before doing it.

Things You Should Do Before You Start the Upgrade

Once again, you should map the homes as above to have those paths handy, and locate the patches in the RUP3 binaries before rather than during the upgrade:

Patch 12575078, which contains Oracle Access Manager WebGates (
Patch 13453929 for Oracle Access Manager WebGates
Patch 13735036 for Oracle Identity Manager
Patch 13768278 for IDM Tools. Note: Be sure you download the version of this
patch, which is named
Patch 13787495 for Oracle Access Manager Config Tool
Patch 13879999 for Oracle Internet Directory
Patch 13893692 for Oracle SOA Manager
 /u01/rup3/installers/oracle_common/patch/13893692 < for ORACLE_COMMON_HOME
 /u01/rup3/installers/soa/patch/13893692 < for SOA_HOME
Patch 13901417 for Oracle Access Manager

Important Note: Patch 13893692 has two parts that are applied separately, and the two parts live in different directories.
Finally, when you install the 11g WebGate(s), you will need a couple of specific gcc libraries present, and for me, my server had the 32 bit versions but not the required 64 bit versions. You can build these from scratch by downloading the source from GNU, but if you’re like me and in a hurry sometimes, you can just get them via yum:

 yum update libgcc
 mkdir /u01/app/oracle/product/fmw/gcc_lib
 cd /u01/app/oracle/product/fmw/gcc_lib
 ln -s /lib64/ ./
 ln -s /usr/lib64/libstdc++.6.0.8 ./

You will need this in Step 9.

Following the Upgrade Doc

Same applies here as above — there are 14 steps in this doc with a special section that applies to Solaris.

Step 8: Upgrade the IAM Node

When you apply patch 13893692, you will get a message stating that it is already present. Just to be safe, I did a roll back and reapplication of it.

Step 9: Upgrade the OHS Node

This is where you need those gcc libraries — from the installer, specify the path that contains the symbolic links above (/u01/app/oracle/product/fmw/gcc_lib for me) for the location of the gcc libraries. The file names have to match those above.
In the new httpd.conf file, you may need to add the following lines to ensure that all logins work as they previously did:
You may need to add one line to httpd.conf to ensure that login works correctly:

 #*******Default Login page alias***
 Alias /oamsso “/u01/app/oracle/product/fmw/oam/webgate/access/oamsso”


So that’s it. If you’re careful and methodical, you shouldn’t encounter any issues for the IDM part of the upgrade. The two upgrade docs, while a bit confusing in some respects, are complete and should work for you. If anybody has their own experiences to share, please do.