Understanding the BI Web Catalog Patching Process

Introduction

Following on from a recent blog article for trouble-shooting RPD Patching issues, which you can find here, this article will cover the BI Web Catalog Patching process. In addition to an overview of the process, it will provide a methodology to understand exactly what changes the Patching Process is endeavoring to make.

This could be useful in troubleshooting patching issues, or even providing enough information to make the decision to skip a patch altogether if the objects being changed are deemed not relevant to the environment being patched.

Please note that the Web Catalog Patching Process for FA Release R9 and onwards will be changing to a new process known as ‘One True Flow.‘ This will be the subject of a future blog article. This post will cover the patching process for FA R8 and all previous FA versions, as well as being similar to the patching methodology used in stand alone OBIEE environments.

Main Article

The RPD contains information on where the data for reports can be found. It contains the database connection strings, it ‘knows’ which tables or view objects to go to to pull the data for a report. It simplifies the complexity for report writers by creating a presentation layer where they can literarily drag and drop fields into a report, and not have to know where the data is coming from or how the SQL is being generated.

The Web Catalog is where the reporting content itself is stored in Fusion BI. The reports, analyses, dashboards, folders, shortcuts, filters, prompts and KPIs. Whereas the RPD is a single encrypted file, the Web Catalog is stored in the form of many files – two for every object – in a file based directory structure.

The default location of the Web Catalog within both Fusion Applications and stand alone OBIEE is:

$ORACLE_INSTANCE/bifoundation/OracleBIPresentationServicesComponent/coreapplication_obips1/catalog

To demonstrate the structure, take a look at this screenshot from the Fusion Analytics Portal. Notice the Catalog location of:

/Shared Folders/Financials/General Ledger/Account Analysis/Prompts

and the 2 objects, ‘Account Analysis Report’ and ‘Account Analysis Report Hidden Prompt’ contained within that folder.

Windows7_x86

On the file system – those files can be found in the exact same path, within the Web Catalog. The image below, showing the full path to the Web Catalog, highlighting the file structure within that which matches the path shown in the browser image above.

Windows7_x86

Every object in the Web Catalog has a matching attribute file which contains metadata about the object, including the access control list (ACLs).

Those attribute files can be seen as the ‘.atr’ suffixed files in the image above, for the two objects –  ‘Account Analysis Report’ and ‘Account Analysis Report Hidden Prompt’.

You can learn more information about Web Catalog Files from this blog article.

 

Web Catalog Patching Process

Changes to a Web Catalog, through patching, will include one or more of the following:

– new content will be added to the Web Catalog

– previous content will be removed from the Web Catalog

– previous content will be updated in the Web Catalog

 

The patching process for the Web Catalog has similarities to that of the RPD, in that there are three versions of the Web Catalog involved.

Base Catalog (BASE), also known as original or baseline. This is the initial Web Catalog before any changes or customizations were made.

Late Catalog (LATE), also known as the Patch or Current Web Catalog. This is the version of the catalog, from a new release or a patch fix, that contains the latest version of the catalog items that need to be applied.

Production Catalog (PROD), also known as the Modified or Customized Web Catalog. This is the version of the catalog that is being used by the user base in the current environment. This could include changes, or new content from the original Base Catalog as users have created new reports or content.

 

Step 1 of the patching process compares the BASE and LATE Catalog looking for the differences between the two at the file level, and creates a text-based ‘DIFF’ file of the changes that have occurred.step1Step 2 takes the ‘DIFF’ file from Step 1, and using the applyPatch command, applies these changes to the PROD Web Catalog, adding, removing or updating the content.

UntitledSince both the Patch, and DIFF files are text based, they can be opened and examined in a text editor.

For the remainder of this article we will examine the contents of the DIFF file, created in Step 1 above, that compared the BASE and LATE web catalogs to produce a list of differences. These would then be applied to the PROD catalog if the applypatch process was run.

If a patch fails – being able to understand the Patch or DIFF file and see what the patch is doing, can be an invaluable tool for trouble-shooting.

 

Dissecting the DIFF File

The first header section of the DIFF file contains the arguments that were used to produce it. This is an example from a Fusion Applications DIFF file:

#BIFNDN_LABEL=BIFNDN_MAIN_LINUX.X64_130929.0802
VERSION=11.1.1.7.0c3
#DIFF_FILE_FORMAT_VERSION=11.1.1.7.0c3 (This is the same as VERSION above)

VERSION and #DIFF_FILE_FORMAT_VERSION tells the Catalog Manager tool (CatMan) what formatting is being used in the diff or patch file. The CatMan cannot process versions later than itself – so a 11.1.1.6 install of CatMan could not process a 11.1.1.7 Patch or DIFF file. In this example both versions are the same.

BASELINE=/scratch/testenv/view_storage/testenv_fainteg/fusionapps/special_patch_root/previous/OracleFusionAppsSharedSkeleton=
LATEST=/scratch/testenv/view_storage/testenv_fainteg/fusionapps/special_patch_root/current/OracleFusionAppsSharedSkeleton=
PRODUCTION=/scratch/testenv/view_storage/testenv_fainteg/fusionapps/special_patch_root/production/OracleFusionAppsSharedSkeleton=
-folder=/shared:/system/metadata
-skipFolder=

BASELINE, LATEST and PRODUCTION are the web catalogs being diffed or patched, and their file path location

-folder is the list of Web Catalog folders being diffed or patched. Default is /shared and /system/metadata.

-skipFolder is the list of Web Catalog folders that should NOT be included in the diff or patch. In this case it’s null, so no folders will be skipped.

-verbosity=change:conflict:detail
-ignore=users:creator:modifier:msgdb
-ignoreProperties=Build
-escapeAllSpecialChars=
-winsConflict=production

-verbosity is a list of flags for how much context or information to give in the file. More than one can be used. Valid values are: all, change, conflict, same, detail. In this case we’re looking for detailed information on changes and conflicts.

-ignore is a list of metadata fields to not diff or patch. Valid values are: all, properties, permissions, ownerattributes, timestamps, creator, modifier, content, msgdb, type, target, signature, path, users, groups, contentstate.

In this case, we are not going to diff User accounts, and for Web Catalog items we are going to ignore their Creator, Modifier and MsgDB (translated messages). This will become clearer in the examples below.

-winsConflict says in the event a web catalog item is changed in both LATE and PROD, what to do with such a conflict. Valid values are: baseline, latest, production, blendThenLatest, blendThenProduction.

In this case the current Production Web Catalog will ‘win’ any conflicts.

 

The next section of the DIFF file is for Renamed Metadata Columns, and would be used to list any RPD Subject Areas, Tables or Columns that have changed that were explicitly being called out by the person running the patch. This is a manual entry and so not something that general patching would use.

Patching via tag is disabled, so the differences will all be based on the path.


Renamed Metadata Columns (Applications)…
Diffs based on groups…
Diffs based on SAppID tags…
***********
Diffs based on paths…
***********

Each file based difference is broken down into sections.

Firstly the file path that is changing, and then whether the changes are within the BASE, LATE or PROD web catalogs.

Within each of those sections the differences are broken into the following metadata types, separated by tabs. A missing value means there was no difference in that metadata type. The format for the DIFF file is as follows:

***********
#PATH of object being impacted
#Catalog (BASE, LATE, PROD) and then whether items are ‘—‘ being removed or were missing, or ‘+++’
being added.
#TYPE #ATTRIBUTES #OWNER #ACL #PROPERTIES #INTERNALPROPERTIES #CONTENT_STATE  #CREATOR  #MODIFIER
#CONTENT #MSG
 #INTERNAL PROPERTIES and #CONTENT STATE were added in 11.1.1.7)

#INTERNAL PROPERTIES and #CONTENT STATE were added in 11.1.1.7.

To demonstrate – this first example shows an entry in the DIFF file where content is being added to the Latest (LATE) version of the Web Catalog:


***********
/Shared/Financials/Test/page1
BASE —
+++LATE Dashboard Page == Administrator +*Presentation Server Administrators=F:^authenticateduser=RX: <no props> <no internal props> NotSet <?xml version=”1.0″ encoding=”UTF-8″?>
<sawd:dashboardPage xmlns:sawd=”com.siebel.analytics.web/dashboard/v1.1″ xmlns:saw=”com.siebel.analytics.web/report/v1.1″ xmlVersion=”200810300″ isEmpty=”false” duid=”kpe0ts2quo2toatn” appObjectID=”09ec868c-74e4-40bf-8ab1-483110b59409~7.9.6.2″>
<sawd:dashboardColumn name=”Column 0″ duid=”ebukvp802pc7s0hf”>
<sawd:dashboardSection name=”Section 0″ duid=”v0ak7od1mm378vb3″ showSectionTitle=”false” appObjectID=”6df71b5d-3c70-445d-8743-752542163972~7.9.6.2″>
<sawd:htmlView name=”HTML 0″ duid=”4q6ipt4297q566oo”>
<sawd:HTML fmt=”html”>[b]Second test dashboard[/b]</sawd:HTML>
</sawd:htmlView>
</sawd:dashboardSection>
</sawd:dashboardColumn>
</sawd:dashboardPage>
PROD —
***********

The #PATH for the object is:  /Shared/Financials/Test/page1

The #CATALOG is “BASE” and the ” —” means this folder doesn’t exist in the BASE Web Catalog

The “+++LATE” shows the change that was added to the Latest Catalog, and that would be applied to Production when patched.

#TYPE  – it is a ‘Dashboard Page’

#ATTRIBUTES – with no attributes

#OWNER – owned by “Administrator”

#ACL – where the Internal Web Catalog Group “Presentation Server Administrators” has Full Control, and the Application Role “authenticateduser” has Read and Execute permissions

#PROPERTIES   – with no properties

#INTERNAL PROPERTIES – with no internal properties

#CONTENT STATE – with the value ‘NotSet’ (the valid options for this are ‘NotSet‘, ‘FactoryContent‘, ‘FactoryContentCustomized‘, ‘NonFactoryContent‘)

#CONTENT – and then the content of the dashboard page which takes up the remainder of the message

These other metadata fields were excluded in the header information, so were not part of the DIFF:

#CREATOR
#MODIFIER
#MSG

 

This next example shows content that has been deleted from the LATEST catalog:


***********
/Shared/Financials/Test/page1
BASE Dashboard Page == Administrator *Presentation Server Administrators=F:^authenticateduser=RX: <no props> <no internal props> NotSet <?xml version=”1.0″ encoding=”UTF-8″?>
<sawd:dashboardPage xmlns:sawd=”com.siebel.analytics.web/dashboard/v1.1″ xmlns:saw=”com.siebel.analytics.web/report/v1.1″ xmlVersion=”200810300″ isEmpty=”false” duid=”kpe0ts2quo2toatn” appObjectID=”09ec868c-74e4-40bf-8ab1-483110b59409~7.9.6.2″>
<sawd:dashboardColumn name=”Column 0″ duid=”ebukvp802pc7s0hf”>
<sawd:dashboardSection name=”Section 0″ duid=”v0ak7od1mm378vb3″ showSectionTitle=”false” appObjectID=”6df71b5d-3c70-445d-8743-752542163972~7.9.6.2″>
<sawd:htmlView name=”HTML 0″ duid=”4q6ipt4297q566oo”>
<sawd:HTML fmt=”html”>[b]Second test dashboard[/b]</sawd:HTML>
</sawd:htmlView>
</sawd:dashboardSection>
</sawd:dashboardColumn>
</sawd:dashboardPage>
+++LATE —
PROD == == == == == == == == == ==
***********

Here, the “+++LATE —” says the item was deleted from the Latest Web Catalog, and thus would also be removed from Production when the patch is applied there.

This example shows ACL / Permission changes:

/shared/Procurement/Sourcing
BASE —
+++LATE Folder System +^biadministrator=F:^gl_accounting_setup_management_duty_obi=RX …..

the folder that is going to have permissions changed ‘/shared/Procurement/Sourcing’

“BASE —” means this folder doesn’t exist in the BASE Web Catalog
“+++LATE” means this folder is in the LATEST Web Catalog, and will be added to production when the patch is applied.
“^biadministrator=F:” means the patch is giving Full Control (F) to the biadministrator role, and Read / Execute (RX) to the gl_account_setup_management_duty_obi role.

This tables provides a key for the ACL / Permission codes available within Web Catalog patching.

Screenshot_9_18_14__2_54_PM

One final example:

***********
/shared/Financials
BASE Folder System ^biadministrator=F:^biauthor=F:^biconsumer=RXLSV: Caption=kcap-1042084931_2: <no internal props> NotSet
+++LATE == == == == +Desc=This folder contains sets of standard Analysis, for each subject area in the Financials Analytics OBI application:+LocalizedDesc=kcap12387045_2334: == ==
***********

#TYPE – it is a ‘Folder’ that is already in the BASE catalog
#ATTRIBUTES – with no attributes
#OWNER – owned by “System”
#ACL – where biadministrator and biauthor have full control, and biconsumer has Read, Execute, and BIP run, schedule and view output rights

Note that the BASE has a #PROPERTIES field that contains “Caption=kcap-1042084931_2”,while the LATE has a #PROPERTIES field that contains “+Desc=This folder contains sets of standard Analysis, for each subject area in the Financials Analytics OBI application:+LocalizedDesc=kcap12387045_2334”

The “+” sign, means “new” properties. Also, there is no minus “-” sign for the Caption on LATE, this means the BASE’s Caption still exists in LATE, so is not duplicated in the LATE section.

 

Summary

This article provided a background to Web Catalog patching within Fusion Applications through Release 8, and also in stand alone BI installations. It furnished the reader with a methodology to examine in more detail the DIFF or Patch files so that they might understand what changes the Patching process is applying. This knowledge can be very useful when trying to resolve patching issues.

Add Your Comment