X

Best Practices from Oracle Development's A‑Team

Important log files and their location in Fusion Applications

Naveen Nahata
Solution Architect

Overview

Log files are critical for the maintenance of system health, debugging performance problems, and functional or technical issues. Fusion applications (FA) is an integrated suite of products and hence it is important to know the log file locations for all the products and components to effectively troubleshoot a problem. A single transaction can span various components with the application, identity and database tiers. The best way to look at and get all the log files is through Enterprise Manager (EM) - you can choose to view them via the individual domains EM Applications Control and Database Control for DB, or you can view them all from a single centralized UI using EM Cloud Control (EMCC). EM Cloud control provides a centralized view of the entire FA stack.

Having said that, it is important to be able to view the files from the command prompt for various reasons - the most important being the ability to script and non-availability of EMCC. This post provides the list of most important log files within the entire FA stack. Viewing and gathering log files from EM will be handled in a separate post.

Main Article

A fully provisioned Fusion Applications environment contains 2-3 databases (1 for FA and 1 or 2 for IDM depending on whether you chose to have IDM and IAM share the same database or not); a full IDM stack comprising of system components - Oracle Internet Directory (OID) / Oracle Virtual Directory (OVD), Oracle HTTP Server (OHS) protected by WebGate,  and java components - OAM, OIM, SOA and OIF in a WebLogic domain named IDMDomain; and Fusion Applications application tier comprising on various WebLogic domains for various product families like HCM, CRM etc. and system components for BI, OHS and GOP. Here is a listing of all the log files for these components.

This post refers to FA Release 9, but most of it will apply to FA installations of lower releases. Sometimes, minor tweaking may be required, but the concepts presented in this post will help you find the alternate locations.

Database:

Alert log and background processes logs:

alert.log is the most important database log file. It lists important message and errors reported by the database. Other background processes logs are less important and you may need to view them only under rare circumstances. These log files go in a location defined by the database initialization parameter background_dump_destination. The parameter value can be found by querying v$parameter database view or using the SQL*Plus "show parameter" command as shown in the following screenshot

SELECT name, value FROM v$parameter WHERE name like 'background_dump_dest';

DB_Dump_Destination

Session trace files:

Session trace files are generated either automatically - when a DB error like ORA-600 or ORA-07445 occurs - or they can be generated on demand by turning trace on a DB session which is usually done for identifying performance issues and sometimes for getting errorstack. These logs go in a location defined by the database initialization parameter user_dump_destination. The parameter value can be found by querying v$parameter database view or using the SQL*Plus "show parameter" command as shown in the following screenshot:

SELECT name, value FROM v$parameter WHERE name like 'user_dump_dest';

DB_User_Dump_Destination

Install Logs:

Software install log files are located under the logs directory, which is under the central inventory. If you use a single central inventory for all oracle products, then this directory can be located using the central inventory location file /etc/oraInst.loc (AIX and Linux) or /var/opt/oracle/oraInst.loc (Solaris). If you use different central inventory location for various products, then the system oraInst.loc may not be a definitive source of the inventory location. In such a case, it is best to look at the local oracle inventory location pointer file $ORACLE_HOME/oraInst.loc

DB_OraInst_Loc

OPatch logs:

OPatch is a tool to patch oracle binaries. OPatch logs are required to troubleshoot issues encountered during patching or for analyzing patch conflicts. These logs can be found under $ORACLE_HOME/cfgtoollogs/opatch directory.

Listener logs:

Listener log files can be found using the command "lsnrctl status <listener_name>"

The following screenshot shows an example:

DB_Listener_Log

Identity Management:

Identity Management logs are stored in the shared configuration location which is specified during IDM provisioning. We will refer to this location as IDM_CONFIG_HOME in the rest of the post.

Similarly, the install base for all the products under IDM is specified during provisioning and we will refer to this location as IDM_INSTALL_BASE in the rest of the post.

IDM_Provisioning_Install_Config_Location

Domain Logs:

Java components for IDM stack, namely, OIM, OAM, SOA and optionally OIF run under individual managed servers within a WebLogic Domain which by default is named IDMDomain. Each WebLogic Domain has a single domain log and log files for each Admin and Managed Server. The four important logs files for each WLS Server (Admin or Managed) are:

1. The diagnostic log file - <server_name>-diagnostic.log

2. The container log file to which WLS logs messages are written - <server_name>.log

3. The standard error and output file (commonly referred to as the out file) to which all standard output and error messages are redirected - <server_name>.out

4. Access log file to which all incoming HTTP requests for that server are logged. This file is typically named access.log or access.log.<timestamp>

 

To identify these log files, you first need to identify the DOMAIN_HOME for IDMDomain on the particular server. For instructions on identifying the DOMAIN_HOME for any WebLogic domain on a particular server, please refer to Appendix A.

The location for various log files is:

Domain Log: <DOMAIN_HOME>/servers/AdminServer/IDMDomain.log

Diagnostic Log for each server: <DOMAIN_HOME>/server/<server_name>/logs/<server_name>-diagnostic.log

WLS Log: <DOMAIN_HOME>/server/<server_name>/logs/<server_name>.log

Out File: <DOMAIN_HOME>/server/<server_name>/logs/<server_name>.out

An example of these files for AdminServer where DOMAIN_HOME is set to /u01/app/r8/idm/config/domains/IDMDomain is shown in the following screenshot:

IDM_Domain_Logs

As a general rule system component logs are located in INSTANCE_HOME/diagnotics/logs/<component_type>/<component_name> which for each component is located in <IDM_CONFIG_HOME>/instances/<instance_name> directory.

OHS Logs:

OPMN logs list messages written by the Oracle Process Monitor which is used to start and monitor OHS processes. These log files are located in <IDM_CONFIG_HOME>/instances/ohs1/diagnostics/logs/OPMN/opmn

OHS logs contain the OHS error and access logs. These log files are located in <IDM_CONFIG_HOME>/instances/ohs1/diagnostics/logs/OHS/ohs1

OID/OVD Logs:

OPMN logs: <IDM_CONFIG_HOME>/instances/ohs1/diagnostics/logs/OPMN/opmn

OID logs: <IDM_CONFIG_HOME>/instances/oid1/diagnostics/logs/OID/oid1

OVD logs: <IDM_CONFIG_HOME>/instances/oid1/diagnostics/logs/OVD/ovd1/

OID Audit logs: <IDM_CONFIG_HOME>/instances/oid1/auditlogs/OID/oid1

OVD Audit logs: <IDM_CONFIG_HOME>/instances/oid1/auditlogs/OVD/ovd1

Patch Logs:

There are various products installed in the IDM stack. Opatch is used for applying patches to various Oracle Homes. All Oracle Homes are installed under various subdirectories within <IDM_INSTALL_BASE>. Here is the information about a few of these homes:

OID/OVD Home: <IDM_INSTALL_BASE>/products/dir/oid

OHS Home: <IDM_INSTALL_BASE>/products/ohs/ohs

IAM Home: <IDM_INSTALL_BASE>/products/app/iam

 

OPatch logs for each of these products are stored in their respective Oracle Homes. The location for these logs is $ORACLE_HOME/cfgtoollogs/opatch

Fusion Applications:

Fusion Applications logs are stored in the shared configuration location which is specified during FA provisioning. We will refer to this location as FA_CONFIG_HOME in the rest of the post.

Similarly, the install base for all the products under FA is specified during provisioning and we will refer to this location as FA_INSTALL_BASE in the rest of the post.

FA_Provisioning_Install_Config_Location

Domain Logs:

A provisioned FA environment contains up to 9 WebLogic Domains. The exact number depends on the products you provisioned. Each WebLogic Domain has a single domain log and log files for each Admin and Managed Server. The four important logs files for each WLS Server (Admin or Managed) are:

1. The diagnostic log file – <server_name>-diagnostic.log

2. The container log file to which WLS logs messages are written – <server_name>.log

3. The standard error and output file (commonly referred to as the out file) to which all standard output and error messages are redirected – <server_name>.out

4. Access log file to which all incoming HTTP requests for that server are logged. This file is typically named access.log or access.log.<timestamp>

 

To identify these log files, you first need to identify the DOMAIN_HOME for IDMDomain on the particular server. For instructions on identifying the DOMAIN_HOME for any WebLogic domain on a particular server, please refer to Appedix A.

The location for various log files is:

Domain Log: <DOMAIN_HOME>/servers/AdminServer/IDMDomain.log

Diagnostic Log for each server: <DOMAIN_HOME>/server/<server_name>/logs/<server_name>-diagnostic.log

WLS Log: <DOMAIN_HOME>/server/<server_name>/logs/<server_name>.log

Out File: <DOMAIN_HOME>/server/<server_name>/logs/<server_name>.out

ESS Logs:

Enterprise Scheduler Service is responsible for scheduling and running batch jobs. The ESS component itself runs in a WLS managed server named ess_server<n> in each domain that requires scheduling services. These logs can be viewed using the location identified using the procedure defined under section "Domain Logs." The log and output files for individual ESS jobs resides in the following location:

ESS Logs: <FA_CONFIG_HOME>/ess/rfd/<requestid>/log

ESS Output: <FA_CONFIG_HOME>/ess/rfd/<requestid>/out

For individual requests, this information can be viewed in the database using the following query:

SELECT
      processgroup,
      logworkdirectory,
      outputworkdirectory
FROM  fusion_ora_ess.request_history
WHERE requestid = <request_id>;

 

Patching Logs:

FA Patch Manager is a tool to apply patches to FA. It calls OPatch to patch middleware components and adpatch to apply database artifacts to FA database.

FA Patch log files are located in <FA_CONFIG_HOME>/atgpf/logs directory.

 

Often times, the middleware patches may be applied directly to individual Oracle Homes using OPatch. OPatch logs are located under $ORACLE_HOME/cfgtoollogs/opatch. Each ORACLE_HOME is listed in the central inventory. This directory is specified during provisioning. inventory.xml under ContextsXML sub-directory of the central inventory lists each Oracle Home.

 

OHS Logs:

OPMN log files are located in <FA_CONFIG_HOME>/CommonDomain_webtier/diagnostics/logs/OPMN/opmn

OHS logs files are located in <FA_CONFIG_HOME>/CommonDomain_webtier/diagnostics/logs/OHS/ohs1

 

BI Logs:

BI is composed of java components (running in WLS domain) and system components (managed by OPMN).

WLS Domain housing BI component is named BIDomain and you can follow the procedure defined to identify other domain logs to get the BI logs.

System components reside under an OPMN instance. The instance home is <FA_CONFIG_HOME>/BIInstance. Log files for each BI system component such as Presentation Server, Cluster controller, Scheduler etc. are located in the following location:

<FA_CONFIG_HOME>/BIInstance/diagnostics/logs/<process-type>/<ias-component>

Here are a example of 2 such components:

Presentation Server: <FA_CONFIG_HOME>/BIInstance/diagnostics/logs/OracleBIPresentationServicesComponent/coreapplication_obips1

Cluster Controller: <FA_CONFIG_HOME>/BIInstance/diagnostics/logs/OracleBIClusterControllerComponent/coreapplication_obiccs1

 

Appendix A: Identifying DOMAIN_HOME for a particular WebLogic Domain

To identify DOMAIN_HOME for a particular Weblogic domain, you must first identify the MIDDLEWARE_HOME. Middleware home is the top level directory under which WLS Server and various other Oracle products are installed.

For IDM, it is <IDM_INSTALL_BASE>/products/app

And for FA, is it <FA_INSTALL_BASE>/fusionapps

Each domain is registered in domain-registry.xml file under these middleware homes. A single domain can be across multiple servers and can optionally be scaled out. Hence a single domain may be listed multiple times. Each of the listing represents a distinct domain home under which various managed servers may be running. Below are the screenshots of domain-registry for IDM and FA where a domain resides complete on a single server:

IDMDomain:

IDM_Domain_Registry

Various FA Domains:

FA_Domain_Registry

Be the first to comment

Comments ( 0 )
Please enter your name.Please provide a valid email address.Please enter a comment.CAPTCHA challenge response provided was incorrect. Please try again.Captcha