EBS -- EBS 12.2.5 - 12C DB installation on Linux

In this blog post, we will see an example EBS 12.2 installation , which is done on a Linux x86-64 system.

The aim of this installation is to obtain an EBS 12.2.5 instance running on Oracle Linux 6 64-bit.

We will use the latest startCD, so we will have an 12C database in our newly installed EBS 12.2.5 instance. This document involves some post upgrade steps, which may be required or changed according to your needs. The underlying server system used for this installation is based on a single node server environment. However, the steps are quite similar in a multi node system.

The general overview of the process is to install Oracle Linux, install EBS 12.2.0 and upgrade it to EBS 12.2.5.  This blog post is a little different than my earlier post, as it will be like a compilation, but still it will give the valuable filtered information required for EBS 12.2. installations.

Please do not take the configuration values, licensed products as well as the post upgrade steps used in this document as the orders :), because they are only there for giving you the general explanation of the installation process, as they may vary according to your environments and needs.

Let 's see, how we do it.

Main Document: Oracle E-Business Suite Installation and Upgrade Notes Release 12 (12.2) for Linux x86-64 (Doc ID 1330701.1)

1. Installation Linux 6.5

2. Installing rpms for EBS 12.2 and RDBMS 12C

[root@ermantest yum.repos.d]# wget http://public-yum.oracle.com/public-yum-ol6.repo

Note : use vi editor to open repo and change enable 0 to 1 for addon and latest repositery

yum install -y oracle-ebs-server-R12-preinstall

yum install –y oracle-rdbms-server-12cR1-preinstall

3. Check all rpms

rpm -qa --qf "%{n}-%{v}-%{r}.%{arch}\n" | grep openmotif21

4. Run ldconfig –v command to relink all symbolic libraries

5. Disable selinux and shutdown iptables,ip6tables services also on run levels

vi /etc/selinux/config ( change SELINUX=enforcing to SELINUX=disabled )

service iptables stop

service ip6tables stop

chkconfig --level 0123456 ip6tables off

chkconfig --level 0123456 iptables off

6. Check kernel settings for the parameters are correctly set

Note: If the current value for any parameter is higher than the value listed in the following table, then do not change the value of that parameter.

vi /etc/sysctl.conf

kernel.semmsl 256

kernel.semmns 32000

kernel.semopm 100

kernel.semmni 142

kernel.shmall 20971524

kernel.shmmax Half the size of the physical memory (in bytes), and at least 42949672952

kernel.shmmni 4096

kernel.msgmax 8192

kernel.msgmnb 65535

kernel.msgmni 2878

fs.file-max 6815744

fs.aio-max-nr 1048576

net.ipv4.ip_local_port_range 9000 65500

net.ipv4.tcp_tw_recycle 05

net.core.rmem_default 262144

net.core.rmem_max 4194304

net.core.wmem_default 262144

net.core.wmem_max 1048576

Note : Parameter kernel.sem = 250 32000 100 128 is changed to kernel.sem = 250 32000 100 142

7. Check Domain Name System (DNS) Resolver Parameters , it should be below

options attempts:5
options timeout:15

8. Verifying Host Names

· Verify that the /etc/hosts file is formatted as follows:

127.0.0.1 localhost.localdomain localhost
[ip_address] [node_name].[domain_name] [node_name]

· Verify that the /etc/sysconfig/network file is formatted as follows:
HOSTNAME=[node_name].[domain_name]
or as follows:

HOSTNAME=[node_name]
NISDOMAIN=[domain_name]

· If the /etc/sysconfig/networking/profiles/default/network file exists, remove it.

· If you changed any files in the previous steps, restart the system

9. Modify ulimit settings in limits.conf
vi /etc/security/limits.conf

# oracle-rdbms-server-11gR2-preinstall setting for nofile soft limit is 1024
oracle soft nofile 4096
# oracle-rdbms-server-11gR2-preinstall setting for nofile hard limit is 65536
oracle hard nofile 65536
# oracle-rdbms-server-11gR2-preinstall setting for nproc soft limit is 2047
oracle soft nproc 2047
# oracle-rdbms-server-11gR2-preinstall setting for nproc hard limit is 16384
oracle hard nproc 16384
# oracle-rdbms-server-11gR2-preinstall setting for stack soft limit is 10240KB
oracle soft stack 10240
# oracle-rdbms-server-11gR2-preinstall setting for stack hard limit is 32768KB
oracle hard stack 32768

10. Download EBS Installation Packages, Oracle E-Business Suite (12.2.5)

V100052-01_1of3.zipOracle E-Business Suite Release 12.2.0 Rapid Install Start Here 725.3 MB
V100052-01_2of3.zipOracle E-Business Suite Release 12.2.0 Rapid Install Start Here 81.6 MB
V100052-01_3of3.zipOracle E-Business Suite Release 12.2.0 Rapid Install Start Here 97.3 MB
V100059-01_1of2.zipOracle E-Business Suite Release 12.2.0 for Linux x86-64 Rapid Install Databases PROD - Disk 1 1007.0 MB
V100059-01_2of2.zipOracle E-Business Suite Release 12.2.0 for Linux x86-64 Rapid Install Databases PROD - Disk 1 1.6 GB
V100060-01_1of3.zipOracle E-Business Suite Release 12.2.0 for Linux x86-64 Rapid Install Databases PROD - Disk 2 1.6 GB
V100060-01_2of3.zipOracle E-Business Suite Release 12.2.0 for Linux x86-64 Rapid Install Databases PROD - Disk 2 1.1 GB
V100060-01_3of3.zipOracle E-Business Suite Release 12.2.0 for Linux x86-64 Rapid Install Databases PROD - Disk 2 1.0 GB
V100061-01.zipOracle E-Business Suite Release 12.2.0 for Linux x86-64 Rapid Install Technology one-off Patches 839.9 MB
V100078-01.zipOracle E-Business Suite Release 12.2 Supplemental Documentation - REVIEW FIRST 21.6 MB
V100079-01.zipOracle E-Business Suite Release 12.2 Supplemental Media 603.2 MB
100102-01.zipOracle Database 12c Release 1 (12.1.0.2.0) EXAMPLES for Linux x86-64 476.1 MB
V77840-01_1of9.zipOracle E-Business Suite Release 12.2.5 Update Pack for Linux x86-64 514.5 MB
V77840-01_2of9.zipOracle E-Business Suite Release 12.2.5 Update Pack for Linux x86-64 527.6 MB
V77840-01_3of9.zipOracle E-Business Suite Release 12.2.5 Update Pack for Linux x86-64 459.4 MB
V77840-01_4of9.zipOracle E-Business Suite Release 12.2.5 Update Pack for Linux x86-64 530.4 MB
V77840-01_5of9.zipOracle E-Business Suite Release 12.2.5 Update Pack for Linux x86-64 571.4 MB
V77840-01_6of9.zipOracle E-Business Suite Release 12.2.5 Update Pack for Linux x86-64 554.9 MB
V77840-01_7of9.zipOracle E-Business Suite Release 12.2.5 Update Pack for Linux x86-64 526.7 MB
V77840-01_8of9.zipOracle E-Business Suite Release 12.2.5 Update Pack for Linux x86-64 596.3 MB
V77840-01_9of9.zipOracle E-Business Suite Release 12.2.5 Update Pack for Linux x86-64 100.5 MB
V77972-01.zipOracle E-Business Suite Release 12.2.5 Documentation Library 730.2 MB

Oracle Database (12.1.0.2.0)

V46095-01_1of2.zipOracle Database 12c Release 1 (12.1.0.2.0) 1.6 GB
V46095-01_2of2.zipOracle Database 12c Release 1 (12.1.0.2.0) 967.5 MB

Oracle E-Business Suite (12.2.0)

V35802-01.zipOracle E-Business Suite Release 12.2.0 for Linux x86-64 Rapid Install AS10.1.2 711.6 MB
V35803-01_1of3.zipOracle E-Business Suite Release 12.2.0 for Linux x86-64 Rapid Install APPL_TOP - Disk 1 1.2 GB
V35803-01_2of3.zipOracle E-Business Suite Release 12.2.0 for Linux x86-64 Rapid Install APPL_TOP - Disk 1 1022.8 MB
V35803-01_3of3.zipOracle E-Business Suite Release 12.2.0 for Linux x86-64 Rapid Install APPL_TOP - Disk 1 1.3 GB
V35804-01_1of2.zipOracle E-Business Suite Release 12.2.0 for Linux x86-64 Rapid Install APPL_TOP - Disk 2 1.2 GB
V35804-01_2of2.zipOracle E-Business Suite Release 12.2.0 for Linux x86-64 Rapid Install APPL_TOP - Disk 2 1.6 GB

Oracle Web Tier (11.1.1.9.0)

V75792-01.zipOracle Fusion Middleware Web Tier Utilities 11g Patch Set 7 (11.1.1.9.0) for Linux x86-64 2.0 GB

Oracle WebLogic Server (10.3.6.0.0)
V29856-01.zipOracle WebLogic Server 11gR1 (10.3.6) Generic and Coherence 1018.5 MB

11. Create Stage

[root@ermantest EBS1225]# cd startCD/Disk1/rapidwiz/bin/

[root@ermantest bin]# ./buildStage.sh

Oracle E-Business Suite Rapid Install
Version 12.2.0

Press Enter to continue...
Build Stage Menu
------------------------------------------------------
1. Create new stage area
2. Copy patches to existing stage area
3. List files in TechPatches directory
4. Exit menu

Enter your choice [4]: 1

Rapid Install Platform Menu

1. Oracle Solaris SPARC (64-bit)
2. Linux x86 (64-bit)
3. IBM AIX on Power Systems (64-bit)
4. HP-UX Itanium
5. Exit Menu

Enter your choice [5]: 2

Running command:
……………..
Specify the directory containing the zipped installation media:
/ebssetupfiles/EBS1225/

File list:
/ebssetupfiles/EBS1225/startCD/Disk1/rapidwiz/bin/stageData/zipFiles.dat

The following files are missing from /ebssetupfiles/EBS1225/.
Return to the Oracle Software Delivery Cloud and download them:
====================================================

Part number: V100053-01

Description:
Oracle E-Business Suite Release 12.2.0 for Linux x86-64 Rapid Install Databases VISION - Disk 1
File name: V100053-01_1of2.zip
====================================================
Part number: V100053-01
Description:
Oracle E-Business Suite Release 12.2.0 for Linux x86-64 Rapid Install Databases VISION - Disk 1
File name: V100053-01_2of2.zip

====================================================

Part number: V100054-01
Description:
Oracle E-Business Suite Release 12.2.0 for Linux x86-64 Rapid Install Databases VISION - Disk 2
File name: V100054-01.zip
====================================================

Part number: V100055-01

Description:
Oracle E-Business Suite Release 12.2.0 for Linux x86-64 Rapid Install Databases VISION - Disk 3
File name: V100055-01.zip

====================================================

Part number: V100056-01

Description:
Oracle E-Business Suite Release 12.2.0 for Linux x86-64 Rapid Install Databases VISION - Disk 4
File name: V100056-01.zip

====================================================

Part number: V100057-01

Description:
Oracle E-Business Suite Release 12.2.0 for Linux x86-64 Rapid Install Databases VISION - Disk 5
File name: V100057-01.zip

====================================================

Part number: V100058-01

Description:
Oracle E-Business Suite Release 12.2.0 for Linux x86-64 Rapid Install Databases VISION - Disk 6
File name: V100058-01.zip

====================================================

WARNING: Some of the required zip files have not yet been downloaded. Do you want to continue, and obtain them later? (Y)/N

Y

Note that: During creation of stage , enter to continue couple of times

12. Start E-Business Installation / EBS 12.2.0 installation

[root@ermantest rapidwiz]# pwd

/ebssetupfiles/EBS1225/startCD/Disk1/rapidwiz
[root@ermantest rapidwiz]# ./rapidwiz

13. Now follow the screens

If we encounter some errors for database pre-install checks and/or for filesystems,  fix them as documented below ;

Open new terminal with root user and run xhost +

Database Pre-installchecks;

Create oraInst.loc file and point it to the inventory location, which can be ORACLE_BASE directory.
ORACLE_HOME==/u01/oracle/TEST/12.1.0
ORACLE_BASE==/u01/oracle/TEST

So the orainventory should be in ora base directory

[oracle@ermanrtest app]$ cat /etc/oraInst.loc 

inventory_loc=/u01/oracle/TEST/oraInventory
inst_group=dba
For filesystem check error ;
chmod –R 777 /u01   (or any appropriate file permissions)

After installation is completed we can connect to the EBS and check the version 12.2.0 and go for next step...

At this point, EBS 12.2.0 installation is complete, so we continue with the EBS 12.2.5 upgrade.

--PREPARING for 12.2.0 to 12.2.5 upgrade--

APPLYING TECHNOLOGY PATCHES FOR 12.2.0 Oracle E-Business Suite Release 12.2.5 Readme (Doc ID 1983050.1) Master Note

1. EBS Technology Codelevel Checker (ETCC) (Doc ID 1594274.1)

Note: This patch verifies the missing bugfixes which is not applied for db and middle tiers. You can apply the patches directly to the related component but I recommend you go for the next steps and crosscheck the output of the log with the table you will see in section 3.2

Download patch 17537119

Upload patch to DB and APP node

For DB Node :

mkdir –p $ORACLE_HOME/appsutil/etcc
copy all files in the patch to the directory “mkdir –p $ORACLE_HOME/appsutil/etcc”
cd $ORACLE_HOME/appsutil/etcc
./checkDBpatch.sh

For APP Node :

cd <patchdirectory>
./checkMTpatch.sh

2. Apply RDBMS (12.1.0.2) Patches

Note: Shutdown database before apply patches

Not included in any startCD	Linux x86-64
	21841318
	22098146
	21286665
	21967332	opatch rollback -id 18893947
	21387964

3. After applying database patches, ensure the following additional tasks are performed:

Copy $APPL_TOP/admin/adgrants.sql to db node
sqlplus "/ as sysdba" @adgrants.sql apps
sqlplus "/ as sysdba" @$ORACLE_HOME/rdbms/admin/utlrp.sql

4. Apply Consolidated Seed Table Upgrade Patch (Required)

Note: If the Release 12.2.0 Vision or PROD instance (New Installation) was created using startCD 12.2.0.51 (Patch 22066363), you must apply Consolidated Seed Table Upgrade Patch 17204589:12.2.0 by merging with Patch 21900871:12.2.0 on the run file system using adop hotpatch mode.

sh $ADMIN_SCRIPTS_HOME/adstpall.sh “ kill -9 `ps -ef|grep applmgr | awk '{print $2}'` “
sh $ADMIN_SCRIPTS_HOME/adadminsrvctl.sh start

Note : copy patches to the patch_top directory “/app/oracle/PROD/fs_ne/EBSapps/patch “

adop phase=apply patches=17204589,21900871 merge=yes hotpatch=yes

5.   3.1 Path A — Upgrade and New Installation Customers. Run ETCC (1617461.1 )

Note: Applying the AD and TXK Delta 7 release update packs checks for the existence of the ETCC tables. Even if you already have all the required database patches installed, you must execute the latest version of ETCC on your database before proceeding to Section 3.

6. Technology Inventory Utility (Optional )

The Technology Inventory Utility generates a report that lists the installed technology stack components, as well as patches that have been applied. Any patches listed in this section that are not shown in the generated report must be installed. Refer to Oracle E-Business Suite Maintenance Guide for instructions on generating the technology report.

perl $FND_TOP/patch/115/bin/TXKScript.pl -script=$FND_TOP/patch/115/bin/txkInventory.pl -txktop=$APPLTMP -contextfile=$CONTEXT_FILE -appspass=apps321 -outfile=$APPLTMP/Report_Inventory.html

7. Apply Oracle Reports and Forms Patches ( 10.1.2.3.0 )

Note : Any configuration file changes suggested in post-install steps can be skipped, as those changes are controlled through technology stack templates initialized by AutoConfig. Any recommendation in any Forms patch readme to redeploy Forms ear file can be skipped as it is automatically deployed by Oracle E-Business Suite service control scripts before starting services. Please regenerate jars from adadmin after applying any of the forms patch, do not opt to force the regeneration of all JAR files. If the patch is being applied in the patch file system then ensure Online Patching prepare phase (adop phase=prepare) has completed successfully first before running adadmin. If the patch is being applied in the run file system, run adadmin after applying the patch

Note: The 10.1.2.3 distribution is 32-bit, even on 64-bit platforms. Download the available patch for your platform.

	Linux x86-64
Not included in any startCD	21539521

Default Oracle Home path is already set to 10.1.2 directory but if you need to set or to be make sure use below command

export ORACLE_HOME=/u01/oracle/TEST/fs1/EBSapps/10.1.2
export PATH=$ORACLE_HOME/OPatch:$PATH
opatch lsinventory

[applmgr@ermantest~]$ opatch lsinventory

Oracle Home : /u01/oracle/TEST/fs1/EBSapps/10.1.2
Oracle Home Inventory : /u01/oracle/TEST/fs1/EBSapps/10.1.2/inventory
Central Inventory : /u01/oracle/TEST/oraInventory
from : /etc/oraInst.loc
OUI location : /u01/oracle/TEST/fs1/EBSapps/10.1.2/oui
OUI shared library : /u01/oracle/TEST/fs1/EBSapps/10.1.2/oui/lib/linux/liboraInstaller.so
Java location : /u01/oracle/TEST/fs1/EBSapps/10.1.2/jdk/jre/bin/java

Opatch apply
After apply patch
Run adadmin
Generate jar files ( dont use force )

8. Apply Oracle Home Patch (Version 10.1.0.5.0)

There is no patch to apply if you installed EBS with the latest startCD. Check “Table 3 - Patches and bug numbers for 10.1.0.5.0” in the document.

9. Apply patch for Oracle Fusion Middleware (FMW) webtier 11.1.1.9

Note: When obtaining patches listed in these tables, download and install the 64-bit version of the patches.

	Linux x86-64
Not included in any startCD	22107540

Set the ORACLE_HOME environment variable to Webtier home and Opatch version should be 11.1.0.9.0 or higher

export ORACLE_HOME=/u01/oracle/TEST/fs1/FMW_Home/webtier

export PATH=$ORACLE_HOME/OPatch:$PATH
[applmgr@ermantest ContentsXML]$ opatch lsinventory

Oracle Interim Patch Installer version 11.1.0.12.9
Copyright (c) 2016, Oracle Corporation. All rights reserved.
Oracle Home : /u01/oracle/TEST/fs1/FMW_Home/webtier
Central Inventory : /u01/oracle/TEST/oraInventory
from : /u01/oracle/TEST/fs1/FMW_Home/webtier/oraInst.loc
OPatch version : 11.1.0.12.9
OUI version : 11.1.0.11.0
OPatch detects the Middleware Home as "/u01/oracle/TEST/fs1/FMW_Home"

Installed Top-level Products (1):
Oracle WebTier and Utilities CD 11.1.1.9.0
cd 22107540/
opatch apply -jre $ORACLE_HOME/jdk/jre

10. Apply patch for Oracle Fusion Middleware (FMW) oracle_common 11.1.1.9

Not included in any startCD	Linux x86-64
	21366277
	21628307

Set the ORACLE_HOME environment variable to oracle_common home and Opatch version should be 11.1.0.9.0 or higher

export ORACLE_HOME=/u01/oracle/TEST/fs1/FMW_Home/oracle_common
export PATH=$ORACLE_HOME/OPatch:$PATH
To be make sure for the oracle home is correctly set. Use opatch lsinventory command

cd <patchdirectory>
opatch apply

11. Apply patch for Oracle Weblogic Server 10.3.6.0

Footnote 1 - This is the 10.3.6.0.12 PSU. When applying a PSU all conflicting patches (including overlays and previous PSUs) would need to be removed. For a list of bug fixes included in this PSU, refer to My Oracle Support Knowledge Document 2030251.1, Oracle WebLogic Server Patch Set Update 10.3.6.0.12 Fixed Bugs List.

Footnote 2 - This overlay patch is specific to 10.3.6.0.12 PSU; apply the PSU Patch 20780171 then this overlay patch.

Footnote 3 - This patch is specific to the Windows Platform.

Footnote 4 - This Oracle WebLogic Server patch is required for Oracle Fusion Middleware 11.1.1.9.

Ensure you have applied R12.TXK.C.Delta.7 (Patch 20784380) and Patch 21846184.

Refer to My Oracle Support Knowledge Document 1617461.1, Applying the Latest AD and TXK Release Update Packs to Oracle E-Business Suite Release.

Patch 21846184 includes TXK code that will enable Java Secure Socket Extension (JSSE) within Oracle E-Business Suite Release 12.2 when Oracle Fusion Middleware version is 11.1.1.9.

Follow the instructions in the patch readme, except for the last step (Step 6) in the post-install instructions.
Patch Installation Steps ;

· Copy content of this zip file with the exception of README file to your SmartUpdate cache directory ($FMW_HOME/utils/bsu/cache_dir by default)

· cd $FMW_HOME/utils/bsu

· you will see the oracle smart update interface and below of the screen displays the patches that needs to be apply

· Start to apply patches and follow their readme,footnotes.

· To apply patches, first you have to apply patch number 20780171 but you may have conflict warning with the previous patches which has older PSU patches like 10.3.6.0.7 because 20780171 patch number is PSU 10.3.6.0.12 patch

·we removed the below patches and then run ./checkMTpatch.sh

Y5AP,BLTG,1LRI,VKXF,2GYW,CW9T, FCX7

· According to the output we have to apply the below patches fort he PSU 10.3.6.0.12

12.2.0.51	13964737 (see Footnote 4)	YVDZ	13964737
Not included in any startCD	20780171 (see Footnote 1)	EJUW	20780171
	13729611 (see Footnote 2)	L34G	13729611
	17319481 (see Footnote 2)	N5FK	17319481
	19687084 (see Footnote 2)	8FBW	19687084
	22128205 (see Footnote 2)	GK5N	22128205,13337000, 21562338

12. Apply R12.TXK.C.Delta.7 patches

Applying the Latest AD and TXK Release Update Packs to Oracle E-Business Suite Release 12.2 (Doc ID 1617461.1)

· Run ETCC according to the section “3.1.1 Prerequisite.
· Source the run edition environment file.

. /u01/oracle/TEST/fs1/EBSapps/appl/APPSTEST_ermtest.env

sh $ADMIN_SCRIPTS_HOME/adadminsrvctl.sh start ( It is already started for upgrade patch )

· Start up only Weblogic AdminServer service on the run edition application tier file system

· Apply below patches
Unzip patches and run adgrants
Patch 20745242 (R12.AD.C.Delta.7)
Patch 22123818:R12.AD.C
Patch 20784380 (R12.TXK.C.Delta.7)
Patch 22363475:R12.TXK.C
Patch 22495069:R12.TXK.C

Note : Compare adgrants.sql versions with in the patch directory and $APPL_TOP/admin

vi 20745242/admin/adgrants.sql
vi $APPL_TOP/admin/adgrants.sql

sqlplus / as sysdba @/app/oracle/PROD/fs_ne/EBSapps/patch/20745242/admin/adgrants.sql apps

· Copy all patches to the $PATCH_TOP (/u01/oracle/TEST/fs_ne/EBSapps/patch )

adop phase=apply patches=20745242 hotpatch=yes
sqlplus / as sysdba @/app/oracle/PROD/fs_ne/EBSapps/patch/22123818/admin/adgrants.sql apps

run etcc on db and app nodes

adop phase=apply patches=22123818 hotpatch=yes
adop phase=apply patches=20784380,22363475,22495069 hotpatch=yes merge=yes

After applied the patch with merge parameter. ( applied Delta Patches ) you will see the env file in your system base directory

/u01/oracle/TEST
-rw-r--r-- 1 applmgr oinstall 6467 Mar 10 15:06 EBSapps.env

· Migrate Latest Code to Database Tier
On the Application Tier (as the APPLMGR user):
Run AutoConfig on the run file system.
. /u01/oracle/TEST/EBSapps.env run
cd ADMIN_SCRIPTS_HOME
adautocfg.sh

Create the appsutil.zip file in <INST_TOP>/admin/out.
perl $AD_TOP/bin/admkappsutil.pl
Copy appsutil.zip file to database $ORACLE_HOME

On the Database Tier (as the ORACLE user):
Source the environment variables for RDBMS ORACLE_HOME.
Uncompress appsutil.zip, under <RDBMS ORACLE_HOME>.
cd <ORACLE_HOME>
backup appsutil folder
unzip -o appsutil.zip
cp –pr appsutil_backup/jre appsutil/
perl adbldxml.pl ( there is no Context File in new appsutil directory so regenerate it )
Run AutoConfig on <RDBMS ORACLE_HOME>

13.   Restart app and DB

--UPGRADING EBS FROM 12.2.0 TO 12.2.5--

Oracle E-Business Suite Release 12.2.5 Readme (Doc ID 1983050.1) Section 7 and 8
Path A — Upgrade and New Installation Customers upgrading to Oracle E-Business Suite 12.2.5 Release Update Pack

14.  Source the run edition applications environment.

It is already set in .bash_profile

15. Stop Weblogic AdminServer and Node Manager services.

Applying the patch in downtime mode requires all application tier services to be down. Therefore, Weblogic AdminServer and Node Manager that were started previously when applying the latest Release Update Packs for AD and TXK for Release 12.2 need to be shut down.

sh $ADMIN_SCRIPTS_HOME/adadminsrvctl.sh stop
sh $ADMIN_SCRIPTS_HOME/adnodemgrctl.sh stop

16. Set the “optimizer_adaptive_features” database parameter to FALSE (required for Database version 12.1.0.2). 

sqlplus "/as sysdba"
alter system set optimizer_adaptive_features=false scope=both;

System altered.

show parameter optimizer_adaptive_features

NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
optimizer_adaptive_features boolean FALSE

17. Apply Oracle E-Business Suite 12.2.5 Release Update Pack Patch 19676458 on the run edition application environment, using downtime mode

cd $ADMIN_SCRIPTS_HOME
adstpall.sh apps/password
adop phase=apply apply_mode=downtime patches=19676458

Note : Restarting 12.2.5 patch fails with "ERRORMSG: Cannot identify any valid application tier nodes/ORA-01400: Cannot Insert NULL Into ("APPLSYS"."FND_NODES"."NODE_NAME")" (Doc ID 2112122.1)

18. Set the “optimizer_adaptive_features” database parameter to TRUE again

sqlplus "/as sysdba"

alter system set optimizer_adaptive_features=true scope=both;

System altered.

19. Start all Application tier services on the run file system.

sh $ADMIN_SCRIPTS_HOME/adstrtal.sh

20. Perform adop cleanup action.

adop phase=cleanup

21. Synchronize file systems

Synchronize the file systems using the command shown below. This action will copy the new run edition code and configuration to the other file system, to ensure that both file systems are in sync before applying patches using the regular adop cycle on the other file system.

adop phase=fs_clone

--POST UPGRADE STEPS--

22. Apply Oracle E-Business Suite Release 12.2.5 Online Help Patch 19676460 using adop hotpatch mode on the run file system.

adop phase=apply patches=19676460 hotpatch=yes

        

23. To apply NLS ( Turkish ) Language patch follow the instructions in correct order

· Activate the new language using the Oracle Applications Manager (OAM) License Manager.

· Maintain snapshot information

· Run the AD Administration utility to maintain multi-lingual tables from the Maintain Applications Database Entities menu.

· Log on to your Applications' administration node as the APPLMGR user and source the APPL_TOP environment file. If you are on Release 12.2.0 or above, make sure you are connected to the RUN edition. Use the manifest generation tool to generate a manifest file by executing the following command:

perl $AD_TOP/bin/adgennls.pl

The manifest generated from the tool is located at :
$APPL_TOP/admin/$TWO_TASK/out/adgennls.txt 

Request a translation sync path using "https://updates.oracle.com/TransSync"
Upload and apply patch with adop through an online patching cycle (prepare,apply,finalize,cutover,cleanup)

24. Apply Localizations patches and implement localizations if necessary.

At this point, we reached the end. That is our EBS 12.2.5 is ready for use. But, before I put an end to this blog post, I want to share some useful documents with you.
Following is a list of  document that may be useful according to your needs.

For NLS- >Oracle E-Business Suite NLS Release Notes, Release 12.2 (Doc ID 1314621.1)
For HR Legislative: DATAINSTALL AND HRGLOBAL APPLICATION: 12.2 SPECIFICS (Doc ID 1469456.1)
For Localization : Creating CLE Product & Applying Mandatory Patches for R12.2 EMEA Add-on Localizations (Doc ID 1628719.1) --optional
EMEA Add-on Localizations - Upgrade steps to 12.2 (Doc ID 1593750.1) --optional
For Turkish Localizations -> R12 EMEA Add-on Localizations - Turkey (Doc ID 472686.1)

EBS -- Active/Passive Cluster + DR minimal configuration

In this post, I will share some Active/Passive EBS 12.2 Architectures extended with simple DR solutions.
These configurations can support new EBS 12.2 environments up to 100 users and  are scaled in a way that requires minimum hardware and software fees.

Note that, this configuration can be modified , extended , scaled up or down and scaled vertically according to the environmental needs. Sizing may also vary according to the expectations, needs and the EBS modules that are planned to be implemented.

First of all our hardware resource can be sized as the following;

Note that, these are based on a standard EBS, that includes HR and standard Financial modules.

In the paragraphs below , I give active/passive + DR architecture as this post is aimed at this type of configuration. (cheap, redundant, stable in a way and fast enough)

Source site:

2 servers in the source site. 1 server active, 1 server passive.
Both of the source servers are connected to a shared storage (fiber/HBA connection preferred)
Failover operations can be done manually, or using a clustering software(Oracle Restart may be a solution, or in cases of IBM, HACMP will work)

EBS apps and db tier is placed on the same server, altough it is not recommended. (we are aimed to be minimal)

DRC site.

DRC is one server.
The replication can be storage based or dataguard based or VM based, It depends on the architecture.
If using dataguard, the target server/DR server should be licensed as well.
Application filesystem synchronization can be done using an utility like rsync or using similar tools which do the same job.
Storage replication, on the other side, does not require any Oracle licenses or an utility for filesystem replication.
However, storage replication /snapshot /snapmirror utilities require their own licenses. (so this should be kept in mind)
Also, storage based replications work in block level, so they do not understand Oracle's block format. From this perspective Dataguard is the best solution for creating and feeding Oracle Replicas/Standby/DR servers. On the other hand, in these days, Storage vendors have solutions/integrated solutions for Oracle and when these solutions are implemented, Storage replications know that they are dealing with the Oracle Databases when replicating the storage level data.

As for the sizing;

Production Site:

For the Active node:

OS: Oracle Linux 6 64 bit, disks from Storage, Local disks can be minimum(300GB-500GB), 64GB memory, 16 cores, 1x HBA(2 HBA -- preffered for multipath)

For the Passive node:

OS: Oracle Linux 6 64 bit 64GB memory, disks from Storage, Local disks can be minimum (300GB-500GB), 16 cores, 1x HBA(2 HBA -- preffered for multipath)

Storage : 

A Storage with 2 controller, 15k disks (1.2 TB disk space -- usable space , which remains after raid configuration) -- disk size is dependent on the expected % of yearly growth .
Optionally, for automatic failover -> a clustering software, and for an additional replication -> a storage in the DR site , which can do replication/snapmirror (Netapp,EMC,Oracle ZFS etc..)

DRC Site:

OS: Oracle Linux 6 64 bit, 1.2 TB disk space(local), 36GB memory, 8 cores.

Optinally a storage, in case storage replication is required, in addition to the rsync and Dataguard. If that's the case, a Storage with 1.2TB can do the job(ofcourse, it should be capable of doing replication/snapmirror kind of works)

In our first configuration we use an ODA machine for the source site. We use ODA in a way to reduce the RAC license needs. That is, we use a virtualized ODA to place our application tiers and databases.
We utilize ODA_BASE domain for placing our EBS databases, but we use only one node of ODA_BASE to keep our license fees minimal.
This configuration gives us the ability to run our EBS databases or applications in the second ODA node in case of a failure.
In case of the disaster, we use Dataguard (async). We choose our DR server to be a single  node, which has the power to handle the workload(at least may handle the workload during 1 day) in case it should be used when a planned maintanence is required in the source servers or in case of a disaster in the source site.
By using the ODA's virtualization, we use the production machine as a consolidated environment, on which PROD, TEST and DEV instances of EBS (app + db ) run.
Note that, dataguard only replicates the Database file, so Oracle binaries, Oracle Homes , Application filesystem should be replicated in to that.. (using rsync or some tool like that)

Another configuration can be based on the traditional servers in the production site;

In this configuration, we still use Dataguard and rsync for the replications , so the only difference is, this time, we use traditional servers like HP , Dell or Sun and a Storage in the Production site to provide the active/passive clustering.

As for the DR site, we still have only one server, which can even be a VM.

Another alternative, may be using a storage based replication directly without having a dataguard. This configuration can be preferred in case one doesn't want to license the target/DRC environment.

In this case, the topology can be like the following;

                                     

The network between Prod and DRC is also important. A low latency will increase the recoverability.
As for the servers, here are some notes;

Production servers are preferred to be physical (For ex: HP servers, Sun Servers)
DRC server can be a VM (Oracle VM Server , ESX etc..)
Test environments can be placed on the passive production node, or they can be placed on Virtual server outside this configuration. For every TEST environment, 32GB Memory, 750 GB disk sapce and 8 core CPU is enough.
CPU Cores specified in this note are Intel. The latest generation intel cpus are prefferred.
There can be an additional storage in the Production Site, to increase the storage level fault tolerancy , as well.

Lastly, as for the "U" sizes, here are some notes;

ODA is a 4U Rack moutnable system.
The traditional servers supporting these sources can be between 1U and 4U sizes as well and if VMs are preffered, they dont even count for the U sizes, as they are virtual.Storage for this capacity can be handled with a 4U architecture and
With a quick calculation, 8U free space is okay for the production site. DR site can be somewhere between 1U-4U according to the configuration.

Important note:  The values used for sizing are completely dependent on the environment. So it may vary accordingly. All this sizing information is given for a new EBS 12.2 environment, which is expected to have the ability to support 70 users. %10 of these users are expexted to currently access the EBS applications. The environment is expected to grow %10 in a year. The configuration is sized to support 3 years. This is for an EBS environment, on which 60000 invoices are made out in a single month, and 40000 bills invoices are received in a single month.

EBS -- Upgrading EBS 11i(11.5.10.2) to 12.2, a compilation

Here is a enterance level document to be used to have a general picture before the EBS 11i to 12.2 upgrade.
It covers the navigation for the documents that are used in the upgrade process, as well as the general steps that should be performed for upgrading EBS 11i to EBS 12.2.
Note that: This document is just a compliation of Oracle documents, but  I believe it will save you time as reading it , you will not have to go back and forth among all the related documents for determining the required upgrade tasks.

Main Document :Oracle E-Business Suite Release Notes, Release 12.2 (Doc ID 1320300.1)
    ->Section 3: Instructions for Upgrade Customers :"Oracle E-Business Suite Upgrade Guide, Release 11i to 12.2 or Oracle E-Business Suite Upgrade Guide: Release 12.0 and 12.1 to 12.2 guides"

Direct Document:
https://docs.oracle.com/cd/E26401_01/doc.122/e48834/toc.htm (Oracle E-Business Suite Upgrade Guide Release 11i to 12.2 Part Number E48834-15)
----
Upgrade Path:
11.5.10 CU2 -> Minimum Patch Requirements for Sustaining Support for 11.5.10 -> Release 12.2
------

Steps:
Before starting the process, it is important to note that it is required to have a new application tier for placing the EBS 12.2's apps tier. This is required as EBS 11i 's Apps Tier is not supported to run on Linux x86-64 bit and EBS 12.2's Apps and DB tiers are delivered as 64 bit.
So at least for Linux installations, there is a need to have a supported Linux OS for placing the EBS 12.2 's Apps Tier.

1)
Upgrading to Oracle E-Business Suite Release 12.2 requires your database to be at the minimum version 11.2.0.4.
To complete the upgrade to Release 12.2, you must upgrade your database to 11.2.0.4 or higher.
Follow the instructions in Database Preparation Guidelines for an Oracle E-Business Suite Release 12.2 Upgrade (Doc ID: 1349240.1).

2)
INFO:Completing this upgrade brings your system to the 12.2.0 release.
You must apply the 12.2.5 Release Update Pack (RUP3) or later(12.2.5) to your existing Release 12.2 system for production use.

3)
Do the sizing CPU,Memory,Disk Space, IO subsystem.

4)
Know that IDO 10g and OSSO 10g will be desupported. Think about implementing OAM and Oracle Access Gate.

5)
Apply 11i.AD.I.7 (conditional) -- if you are not on AD.I.7

6)
Run TUMS utility and reviw: Download and apply AD TUMS patch

Download patch 18342870 and apply it to the administration server node on your Release 11i APPL_TOP.
It supplies you with the script (adtums.sql) you need to generate the TUMS report (tumsr12.html).
Generate the TUMS report->
$ cd $AD_TOP/patch/115/sql
$ sqlplus <APPS username> @adtums.sql <DIRECTORY>

7)
Backup the database and apps tier including customizations (cold backup preferred)
Note that: Customization is not handled during the upgrade, so the customization should be reapplied when the upgrade is done.

8)
Prepare yourself for the customizations.

You may have customized your system for business use. Take note of these important considerations before upgrading custom database objects:
The Oracle E-Business Suite Developer's Guide contains extensive instructions about naming standards and issues related to upgrading custom database objects. Familiarize yourself with this information before you begin the upgrade.
Run several test upgrades and track their impact on your custom database objects.
Rename any custom database objects with Applications prefixes that you have created so that they do not conflict with Oracle object names.

Note: Failure to test the impact on custom database objects before the upgrade can result in a loss of functionality.
At your discretion, and depending on the customizations in your system, you should also perform the following tasks:
Preserve the CUSTOM library by making a backup copy of CUSTOM.pll. You can use this copy later in the upgrade process to migrate your CUSTOM library to Release 12.2.
If you have customized forms with Oracle Forms 6i, then upgrade them to Oracle Forms 10g after the upgrade.

9)
Convert to Multiple Organizations architecture (required).
Converting to Multiple Organizations does not require the use of multiple operating units or sets of books - it does enable you to use this functionality at any time in the future.

If you are converting from a Single Organization architecture to a Multiple Organization architecture, complete the following steps:
Create an operating unit.
Assign the operating unit you created to the profile option MO:Operating Unit.
Note: See the following documentation references: Use of Multiple Organizations in Release 12 in Use of Multiple Organizations (Multi-Org) in Release 11i (Doc ID: 210193.1), and MOAC in Oracle Purchasing (Doc ID: 404800.1). HRMS users should also see Setting Up Multiple Organizations for Oracle HRMS (Doc ID: 259546.1).

10)
Drop event alert triggers in custom schemas (conditional).
To drop all event alert database triggers in custom schemas, run the alrdtrig.sql script, located in $ALR_TOP/admin/sql (in your Release 11i system). Re-create the triggers after the upgrade is complete.

11)
Review sizes of old and new tablespaces (required).
Make sure you allocate sufficient tablespace.
For guidelines based on an upgrade of the largest Oracle production system (oraprod), see Applications Release 12 Upgrade Sizing and Best Practices (Doc ID: 399362.1).

12)
Run AD preparation scripts (conditional).

Download and unzip the new patch 13435302. Follow the instructions in the readme for running these scripts:

adgncons.sql

The tablespace model for this release (OATM) is based on database object type rather than product affiliation. The adgncons.sql script prepares adcrtbsp.sql, configures the database to hold the new products to be added during the upgrade, and switches your system to use the new tablespace model.

adcrtbsp.sql

Generated by adgncons.sql, this script creates the new tablespaces, allocates unlimited tablespace to all APPS users, updates fnd_product_installation table with correct data and index tablespace information, assigns default tablespace to all APPS users, and sets the new_ts_mode flag in fnd_product_groups to Y. Be sure to review the locations of the dbf files for each new tablespace listed in adcrtbsp.sql, and change the locations in the script as necessary. Ensure that you have adequate disk space for all the changes and additions that adcrtbsp.sql is going to make.

Note: All the tablespaces created by adcrtbsp.sql are critical and required for 12.2 as they hold critical objects. Therefore, you should not drop any of these tablespaces at any point.
Note: Oracle has tested successfully an extent size of 128 K for small systems (for example, 100 GB database or the Vision database), and, for large multi-terabyte system, an extent size of 4-10 MB has been tested successfully.

adgrants.sql :(adgrants_nt.sql for Windows) Grants SYS privileges needed by Applications, and creates required views in SYS.

13)
Migrate Existing Objects to New Tablespace Model (recommended).
Running the preparation scripts creates tablespaces and prepares the database objects only for new products.
However, it does not automatically migrate your existing objects.
Oracle strongly recommends that you use the Tablespace Migration Utility to perform this migration now.
Note that this utility is not supported for use after you enable Online Patching, so you cannot perform the migration after your environment is upgraded to Release 12.2.
If you choose not to migrate to OATM now, then you must continue to manage your tablespaces separately.
Check  the Oracle E-Business Suite Setup Guide, Release 12.2 for mor information.

14)
Convert Oracle Alert E-mail Processing to the Workflow Notification Mailer (conditional)
If you use response processing alerts and you have not already converted Oracle Alert to the Workflow Notification Mailer, convert now by applying Oracle Applications Technology 11i.ATG_PF.H Rollup 6 (patch 5903765) to your Release 11i APPL_TOP. This roll up patch causes Oracle Alert to use the Workflow Notification Mailer for new alerts, but allows you to continue to run the Alert Response Processor for incoming responses sent before the conversion. Continue to run the Response Processor until you have no more outstanding responses of this kind.
Note: See About Oracle Applications Technology 11i.ATG_PF.H Rollup 6 (Doc ID: 444524.1) for more information.

15)
Oracle XML Gateway:  In release 12.2, Oracle XML Gateway Web services depend on Oracle E-Business Suite Integrated SOA Gateway which has product dependencies on Oracle SOA Suite and Oracle Fusion Middleware Adapter for Oracle Applications (also called Oracle E-Business Suite Adapter). See Installing Oracle E-Business Suite Integrated SOA Gateway, Release 12.2 (Doc ID: 1311068.1).


16)
Ensure that the GUEST account is valid and active and that the fnd_user USER_ID for the GUEST account is set to a value of '6'.

17)
Do Product specific upgrade tasks documented in : Preparing for the Upgrade secetion of "Oracle E-Business Suite Upgrade Guide Release 11i to 12.2 Part Number E48834-15" - https://docs.oracle.com/cd/E26401_01/doc.122/e48834/T527285T527359.htm
-Customer Relationship Management Tasks
-Financials and Procurement Tasks
-Human Resource Management (HRMS)
-Lease and Finance Management
-Projects tasks
-Public Sector Tasks
-Supply Chain Management Tasks

18)
Gather schema statistics (required)
FND_STATS.GATHER_SCHEMA_STATISTICS('ALL', 10, :parallel_degree, 'NOBACKUP', NULL, 'LASTRUN', 'GATHER AUTO', 10, 'N');
Where: parallel_degree is set to the value of the database initialization (init.ora) parameter parallel_max_servers for your instance.

19)
Install JRE on the database tier (conditional)
If you are planning to run Rapid Install in Upgrade Mode by using the Use Existing ORACLE HOME option, then you must install JRE in the Database ORACLE_HOME/appsutil as follows:
Download the latest JRE 7 Update. For optimum stability, performance, scalability, and OS vendor support, use the latest available update of JRE for the Oracle E-Business Suite database tier. The JRE download location is: http://www.oracle.com/technetwork/java/javase/downloads/index.html

20)
Reset init.ora parameters (required)
Follow the instructions in Database Initialization Parameters for Oracle E-Business Suite Release 12.2 and reset the init.ora parameters as needed.

21)
Run Rapid Install (required)
Use the Rapid Install wizard to lay down the file system and install the new technology stack for your Release 12.2 Oracle E-Business Suite system.

-Start Rapid Install by typing rapidwiz on the command line. The Welcome screen lists the components that are included in, or supported by, this release of Oracle E-Business Suite. Click Next.
-On the Wizard Operation screen, select Upgrade to Oracle E-Business Suite Release 12.2. Click Next.
-On the Select Upgrade Action screen, select Create Upgrade File System.
-In the associated screen flow, enter the parameters required to set up your new environment. Then, run Rapid Install.
-Note: Oracle E-Business Suite Installation Guide: Using Rapid Install contains complete instructions for running Rapid Install for both new installations and upgrades. Chapter 3 contains the information specific to running an upgrade.

22)
Synchronize values of APPLPTMP with UTL_FILE_DIR for PL/SQL based Concurrent Requests (required)

23)
Backup the database and apps tier including customizations (cold backup preferred)

24)
At this point, the database must be 11.2.0.4 or higher. So if not upgraded earlier, upgrade the database.
If you have not already done so, you can upgrade your production database to 11g Release 2 (11.2.0.4) or higher now, before the upgrade downtime. Follow the instructions in Database Preparation Guidelines for an Oracle E-Business Suite Release 12.2 Upgrade (Doc ID: 1349240.1).
Note: If you are upgrading to 11gR2 or 12cR1 from 10g or 9i, then you MUST set the parameter sec_case_sensitive_logon = False in your init.ora file to avoid login issues with sqlplus. For more information on this parameter and other database initialization parameters, refer to Database Initialization Parameters for Oracle Applications Release 12 (Doc ID: 396009.1).
If you are upgrading to a Release 12c database, then ensure that your sqlnet_ifile.ora contains the line: SQLNET.ALLOWED_LOGON_VERSION_SERVER = 8

25)
Disable AOL Audit Trail (conditional)

26)
Shut down application tier listeners and concurrent managers (required)

27)
Update init.ora with upgrade parameters (required)

28)
Set FAILED_LOGIN_ATTEMPTS to UNLIMITED for Oracle E-Business Suite schema

29)
Disable custom triggers, constraints, and indexes (conditional)

30)
Drop MRC schema (conditional) --it is no longer needed, it can be dropped.

31)
Back up the database (recommended)

32)
Enable maintanence mode

33)
Download and apply the AD Upgrade Patch for 12.2 (patch 10117518)(required) by following the patch read me.

34)
Apply all Consolidated Upgrade Patches (CUPs) (required)
--Apply the Consolidated Upgrade Patches (CUP) and all of the pre-install patches listed in the Oracle E-Business Suite Release Notes, Release 12.2 (Doc ID: 1320300.1)

35)
Run the American English upgrade patch driver (required)
AutoPatch to run the (American English) upgrade patch driver (u10124646.drv) merged with latest Consolidated Upgrade Patch for Oracle E-Businees Suite Release 12.2 . Release 12.2 upgrade patch driver (u10124646.drv) is located under $AU_TOP/patch/115/driver on Run File System.

36)
Disable Maintenance Mode (required)

37)
Follow the instructions in Database Initialization Parameters for Oracle Applications Release 12 (Doc ID: 396009.1) and reset the init.ora parameters as needed.

38)
Backup the database and apps tier

39)
Review the recommended security processes documented in Best Practices for Securing Oracle Applications Release 12 (Doc ID: 403537.1)

40)
Configure Release 12.2 E-Business Suite instance (required)
  -create appsutil using admkappsutil.pl
  -copy appsutil to database tier.
  -install jre on the database tier.
  -create EBS/autocofig related directories in the database tier such as $ORACLE_HOME/network/admin/context_name
  -create context file in the database tier using adbldxml.pl
  -exec fnd_conc_clone.setup_clean to clean the fnd_nodes
  -Synchronize values of APPLPTMP with UTL_FILE_DIR for PL/SQL based Concurrent Requests
  -run autoconfig on db tier
  -Run AutoConfig on the Apps Tier.
  -Run Rapid Install to configure Release 12.2 E-Business Suite instance. (second time Rapid install run)

41)
Upgrade Add-on Localization products using note:  Add-on Localizations - Upgrade Consideration (Doc ID: 1491965.1)

42)
Integrate custom objects and schemas (conditional).
follow the steps in Oracle E-Business Suite Developer's Guide to reintegrate these customizations with the APPS schema.

43)
Re-enable custom triggers, constraints, and indexes

44)
Enabling Online Patching

45)configure init.ora
The parameters described in this section apply to Oracle E-Business Suite Release 12.2. For details, refer to Database Initialization Parameters for Oracle E-Business Suite Release 12 (Doc ID: 396009.1).

46)
Apply the Latest AD and TXK for Release 12.2


47)
Apply the Latest Oracle E-Business Suite Release Update Pack for Release 12.2 --"12.2.5 at the moment"

48)
Reset ORACLE schema passwords (recommended)

49)
Reset ORACLE schema passwords (recommended)

50)
Verify completion of concurrent programs (recommended)

51)
Drop Obsoleted Product Schema (optional)

Obsolete Product Schemas that can be considered for dropping include:
ABM, AHM, AMF, AMW, BIL, BIV, BIX, BSC, CSS, CUE, CUF, CUI, CUN, CUP, CUS, DDD, EAA, EDWREP, EVM, FEM, FII, FPT,
FTP, GCS, HCC, IBA, IBP, IGF, IGS, IGW, IMT, IPD, ISC, ITA, JTS, ME, MST, OKB, OKI, OKO, OKR, OZP, OZS, PFT, POA,
PSB, RCM, RHX, RLA, VEH, XNC, XNI, XNM, XNS, ZFA, ZPB, ZSA.

52)
Drop dangling synonyms by using "sqlplus APPS/****@DB @$AD_TOP/sql/adzd_drop_synonyms.sql"

53)
Install online help (recommended)

54)
Apply latest product patches (required)
Determine the latest product-specific patches. Then, download the American English patches. You must apply the patches using AD Online Patching (adop).
Note: See Patch Wizard Main Page in Oracle E-Business Suite Maintenance Guide Release 12.2.

55)
Installing NLS upgrade patch driver and NLS Online Help (conditional) by requesting requesting a translation patch,
See Requesting Translation Synchronization Patches (Doc ID: 252422.1)

56)
Update and verify custom responsibilities (conditional)

57)
Grant flexfield value set access to specific users (required) by following:  Flexfield Value Set Security chapter in Oracle E-Business Suite Flexfields Guide, Release 12.2 (Part No. E22963-07).

58)
Migrate custom development to new technologies (recommended)
Check for general custom development :Preparing Custom Development for Next Oracle Applications Release (Doc ID: 374398.1)
Check for mod_plsql custom development:Application Framework Developer's Guide (Doc ID: 1315485.1) f

59)
Migrate the CUSTOM library
Place a copy of the new CUSTOM library (CUSTOM.pll) in a safe place. It is located in the $AU_TOP/resource directory (UNIX), or the %AU_TOP%\resource directory (Windows). Then, make a copy of the old Oracle Forms CUSTOM library and place it in the new directory. Upgrade to Oracle Forms Developer 10g by regenerating the library. Or, you can cut and paste the existing custom code into the new library, and then regenerate it.

60)
Copy and re-customize modified scripts or reports (conditional)
Copy custom shell scripts or reports to the custom application directories and re-customize the copy as necessary.

61)
Copy existing custom start scripts  (for example if you have customized concurrent manager start script in 11i earlier...)

62)
Review user responsibility assignments
Although user/responsibility assignments are preserved during the upgrade, the effective permissions granted by the seeded responsibilities, menus, functions, and report security groups may have change

63)
Configure applications client software for forms applet (required).
See:Deploying JRE (Native Plug-in) for Windows Clients in Oracle E-Business Suite Release 12 (Doc ID: 393931.1)

64)
Associate organization names with custom Alert definitions

65)
Set operating unit mode for customizations (conditional)

66)
Update Status Monitor URLs (required)

67)
Do Product specific post upgrade tasks documented in : Preparing for the Upgrade secetion of "Oracle E-Business Suite Upgrade Guide Release 11i to 12.2 Part Number E48834-15" - https://docs.oracle.com/cd/E26401_01/doc.122/e48834/T527285T527359.htm
-Customer Relationship Management Tasks
-Financials and Procurement Tasks
-Human Resource Management (HRMS)
-Lease and Finance Management
-Projects tasks
-Public Sector Tasks
-Supply Chain Management Tasks

68)
Perform System Maintanence Tasks documented in : "Post-Upgrade Tasks section" of "Oracle E-Business Suite Upgrade Guide Release 11i to 12.2 Part Number E48834-15" - https://docs.oracle.com/cd/E26401_01/doc.122/e48834/T527285T527359.htm

69)
Implement New Product and Country-specific Functionality

70)
Back Up Oracle E-Business Suite

EBS - DR - Dataguard - switchover tests & flashback

Normally EBS dataguard switch over tests including the application tier switch over and switch back operations can be done using the document:

"Business Continuity for Oracle E-Business Release 12.1 Using Oracle 11g Release 2 Physical Standby Database (Doc ID 1070033.1)" , this is for R12 (12.1)

The section named "Role Transitions" is easy to implement and the documentation is accurate.

The one thing to consider is the need for running the apps tier postclone/adcfgclone.pl in case all the required steps is not done in preparing the DR environment, but except that, the documentation is perfect.

Here in this post, I will share you an alternative disaster test method, which is based on the Oracle 's Flasback database technology.

In this method, we can test our DR environments, without doing a Dataguard switchover. So it is easier to implement and it may be needed in some cases.

Here is the action plan(more or less) for accomplishing that;

On Primary DB tier:

Stop the log shipping;

alter system set log_archive_dest_state_2=defer;

On Standby DB tier:

alter database recover managed standby database cancel;

alter database flashback on;

create restore point OPEN_STANDBY guarantee flashback database;

alter database activate standby database;

alter database open;

sqlplus apps/apps_password
exec fnd_conc_clone.setup_clean;
commit;

cd $ORACLE_HOME/appsutil/bin

perl adbldxml.pl tier=db appsuser=apps

sh adconfig.sh

On Standby Apps tier:

cd $ADMIN_SCRIPTS_HOME
perl adcfgclone.pl appsTier

Start the services.

--update conc request queues and wf mailer conf. if necessary

Fail/Switch Back:
On Standby DB tier:

sqlplus "/as sysdba"
SHUTDOWN IMMEDIATE;
STARTUP MOUNT;

FLASHBACK DATABASE TO RESTORE POINT OPEN_STANDBY;

DROP RESTORE POINT OPEN_STANDBY;

ALTER DATABASE CONVERT TO PHYSICAL STANDBY;

SHUTDOWN IMMEDIATE;
STARTUP NOMOUNT;
ALTER DATABASE MOUNT STANDBY DATABASE;
ALTER DATABASE RECOVER MANAGED STANDBY DATABASE DISCONNECT;

On Primary DB Tier:
alter system set log_archive_dest_state_2=enable scope=both;

This procedure reflects the actual work that is need to be done(more or lesss), in case of a DR test.
What we do in this procedure is, we stop the managed recovery, create guaranteed flashback restore point, then activate our standby database and then configure our application tier (standby app tier) to this newly activated database and let our functional teams to test the DR environment. After the test, we restore our restore point, start the managed recovery and get back where we start.

This routine may be changed/enhanced according to the needs of the environment, but the important thing is that you get the idea...

EBS 12.2.5 - adopmon

As a new feature of EBS 12.2.5 , we have adopmon(adop monitor, Online Patching Monitoring utility) utility for monitoring the online patching cycles. It provides a readable output, letting us monitor the events that occur in the online patching cycle phases.
The overall progress of an online patching cycle can be followed by running the new Online Patching Monitoring utility (adopmon), while seeing the various individual adop actions being taken (in a readable format).adopmon is executed after setting the run environment. It requires us to provide the apps password and that's it.
Here is an example run, captured during a cutover phase.

[root@ermanserver~]# su - applmgr

  E-Business Suite Environment Information

  ----------------------------------------

  RUN File System           : /app/oracle/PROD/fs1/EBSapps/appl

  PATCH File System         : /app/oracle/PROD/fs2/EBSapps/appl

  Non-Editioned File System : /app/oracle/PROD/fs_ne

  DB Host: ermanserver  Service/SID: PROD

Sourcing the RUN File System ...

[applmgr@ermanserver ~]$ adopmon

Running script. Press Ctrl-C to quit.

Enter the APPS password:

Validating credentials...

Printing the log statements starting from sequence #632872

Timestamp           Node name           Message Type Message Text

---------------------------------------------------------------------------------------------------------------------------

2016/05/11 15:43:55 ermanserver         EVENT        Log: /app/oracle/PROD/fs_ne/EBSapps/log/adop/13/adop_20160511_154324.l

                                                     og

2016/05/11 15:43:56 ermanserver         EVENT        Validating configuration on node: [ermanserver].

2016/05/11 15:43:56 ermanserver         EVENT        Log: /app/oracle/PROD/fs_ne/EBSapps/log/adop/13/cutover_20160511_15432

                                                     4/PROD_ermanserver

2016/05/11 15:44:17 ermanserver         EVENT        Checking if finalize required.

2016/05/11 15:44:17 ermanserver         EVENT        Finalize phase is not required

2016/05/11 15:44:18 ermanserver         EVENT        Submitting request for Internal Concurrent Manager shutdown.

2016/05/11 15:44:18 ermanserver         EVENT        Log file: /app/oracle/PROD/fs1/inst/apps/PROD_ermanserver/logs/appl/ad

                                                     min/log/adcmctl.txt

2016/05/11 15:44:18 ermanserver         EVENT        Cancelling ADZDPATCH concurrent request

2016/05/11 15:44:19 ermanserver         EVENT        Waiting for Internal Concurrent Manager to go down.

EBS 12.2 -- sync errors due to missing patch files/directory does not exists.

In Oracle documents, we see ;
"The directories where you extracted the patches applied in a given patching cycle must be retained, in the same location and with the same contents, until the next prepare phase completes. This is also a requirement for patches applied in a hotpatch session."

See what happened in the next prepare phase, after we applied a patch and then deleted it from PATCH_TOP.

You see, adop complained about the patch directory that was not exists.
This was actually an expected behaviour..

So  according to Oracle , what needed to be done is to download the missing patch from Oracle Support and restore it to the PATCH_TOP.

However, what we did in one of our test environment and want to share with you in this post is a little different.

That is, as we have fs_clone utility, we aborted the patch cycle in which the prepare phase was encountering this error and then executed fs_clone for the solution.

After the fs_clone, we saw the filesystem objects were synched and then we executed the prepare phase again. This time prepare phase completed with success.

Altough we don't suggest this approach, it might become handy one day.

executing fs_clone is actually useful some other cases as well.

As documented in  E-Business Suite - ADOP Basic Usage Training Videos [Video] ( Doc ID 2103131.1 ),  sometimes we can even execute fs_clone by our own decision to decrease the job of the next prepare phase.

Bytheway, this missing patch error, the oracle support solution and fs_clone solution are valid for both online and hot patches.

So the following scenario works; (not recommended..)

time 0: Suppose we apply a hotpatch using adop hotpatch=yes
time 1 : Suppose the patch (in hotpatch mode) applied successfully
time 2: Suppose we delete the patches (including the hotpatch at time0, that we have applied using hotpatch option) from PATCH_TOP
time 3: Development asks us to apply an online patch
-- now we need to start a new patching cycle to apply the online patch requested at time 3.
time 4: To apply the patch requested at time 3, we execute adop phase=fs_clone before executing adop phase=prepare for creating the new patching cycle , because " we deleted the patch files from PATCH_TOP, so adop prepare won't  able to sync the filesystems."

RDBMS -- enabling custom maintanence for the deployments

During the deployment cycles, we may want to have a normal database, running with all of its components including its listener. We may want to have a database in a form equal to its runtime environment and we want to be the only one that is able to connect to the database.
These kinds of desires may arise for having the deployment processes, which may require no one except the administrator to be connected on an Oracle Database, for which we need all the database functionality up&running. So we are talking about a custom maintanence mode here.

Let's see how we implement such a maintanence mode functionality

We first create a trigger(in this example, it is "LOGON_PREVENT_DURING_DEPLOY") to read the sys_context and prevent the login requests except the one coming from the administrator's machine. (in this example: it only lets the user who has "ermanarslan" as its OS user name, to connect to the database). Note that: This trigger may be modified according to our needs.

Trigger:

CREATE OR REPLACE TRIGGER SYSTEM.logon_prevent_during_deploy
   AFTER LOGON ON DATABASE
DECLARE
   v_osuser    v$session.osuser%TYPE;
   v_program   v$session.program%TYPE;
   CURSOR user_prog
   IS
      SELECT osuser, program
        FROM v$session
       WHERE audsid = SYS_CONTEXT ('USERENV', 'SESSIONID');
BEGIN
   OPEN user_prog;
   FETCH user_prog
    INTO v_osuser, v_program;
   CLOSE user_prog;

   IF LOWER (v_osuser) != ('ermanarslan')
   THEN
      raise_application_error
         (-20001,
          '<<<<< The System is in maintanence mode right now. Check again later  >>>>>'
         );
   END IF;
END;
/
ALTER TRIGGER LOGON_PREVENT_DURING_DEPLOY DISABLE;

Then we create 2 procedure for enabling and disabling the trigger. For enabling the trigger(actually enabling the maintanence mode) before we begin our deployment activities , we execute the enabling procedure and for disabling the trigger(actually the maintanence mode), we execute the disabling procedure.

Procedure for enabling the maintanance mode:

CREATE OR REPLACE procedure Enable_maintanence
 IS
 counter INTEGER := 1 ;
 count_son number;
 sqlStmt VARCHAR2(1000);
 cursor c1 is select a.sid,a.job,b.serial# from dba_jobs_running a, v$session b where a.sid=b.sid;
 jobs c1%ROWTYPE;
BEGIN
select count(*) into count_son from dba_jobs_running;
execute immediate 'alter trigger system.logon_prevent_during_deploy enable';
execute immediate 'revoke ADMINISTER DATABASE TRIGGER from dba';
execute immediate 'revoke ADMINISTER DATABASE TRIGGER from IMP_FULL_DATABASE';
loop
exit when count_son=0;
open c1 ;
fetch c1 into jobs;
DBMS_JOB.REMOVE(jobs.job);
sqlStmt := 'ALTER SYSTEM KILL SESSION ''' ||
                TO_CHAR(jobs.sid) ||
                ',' ||
                TO_CHAR(jobs.serial#) ||
                '''' ;

EXECUTE IMMEDIATE sqlStmt;

EXIT WHEN count_son=counter;
counter := counter+1;
close c1;
END LOOP;
execute immediate 'alter system set job_queue_processes=0';
END;
/

Procedure for disabling the maintanance mode:

CREATE OR REPLACE procedure Disable_maintanence
 IS
BEGIN
execute immediate 'alter trigger system.logon_prevent_during_deploy disable';
execute immediate 'grant ADMINISTER DATABASE TRIGGER to dba';
execute immediate 'grant ADMINISTER DATABASE TRIGGER to IMP_FULL_DATABASE';
execute immediate 'alter system set job_queue_processes=20';
END;

/

Of course, the enabling procedure can be modified for killing all current user sessions before performing any deployment/maintanence tasks. Lastly, you can always use the standard restricted mode of the Oracle Database (ALTER SYSTEM ENABLE RESTRICTED SESSION). This custom maintanence mode functioanlity just an alternative.