Wednesday, April 30, 2014

EBS -- 12.2.0 to 12.2.3 upgrade + Ad.C.delta.4 Txk.C.delta.4 upgrade + Legislative Data Install

This document written to describe the upgrade process of the Oracle EBS 12.2 systems.
I saw a need for this document, as Oracle requires you to upgrade newly installed EBS 12.2 systems to the latest release, which is 12.2.3 ... It seems like 12.2.0 is not even supported by Oracle.
So, by following this document you will be able to upgrade 12.2.0 to 12.2.3 , and fix the issues which may be encountered during your upgrade..

Following diagram describes the Upgrade Paths for 12.2.3 ..  So our path will be 12.2.0 to 12.2.3, as I marked it in red.


Also, I will mention about Ad.C.delta.4 and Txk.C.delta.4 upgrades, because these upgrades seems to be required for a proper installation/implementation of EBS 12.2, as these upgrade actions fix some  of the issue encountered on the way.
I suppose that you already installed EBS 12.2.0 fresh environments, but if you need to EBS 12.2 install first, follow the documents below;



For Vision Installation:
http://ermanarslan.blogspot.com.tr/2014/03/ebs-oracle-ebs-122-vision-installation.html

For Fresh Installation:
http://ermanarslan.blogspot.com.tr/2014/04/ebs-oracle-ebs-1220-fresh-installation.html

Also we will take a lot of action with Adop, check the following document for the infomation about adop..
http://ermanarslan.blogspot.com.tr/2014/03/ebs-122-applications-dba-online.html

Also For Client Browser Requirements and Java 7 issues, I recommend you to read the following docs..
Recommended Browsers for Oracle E-Business Suite Release 12 (Doc ID 389422.1),
http://ermanarslan.blogspot.com.tr/2014/04/ebs-and-java-7-security-missing.html

Note that: The explained actions in this document are specifically for Linux x86-64 ,but generally the steps and methods are similar for the other platforms..On the other hand, keep in mind that : the needed patches and some of the needed actions may change according to your environment..

12.2.0 to 12.2.3 UPGRADE:
The root documents that you will need to follow are listed below;

  • For Fresh install -> Oracle E-Business Suite Installation and Upgrade Notes Release 12 (12.2) for Linux x86-64 (Doc ID 1330701.1)
  • For Preparation for Upgrade on the Database tier, and on other components like webtier,weblogic 10.1.2 home and common home-> Oracle E-Business Suite Release 12.2: Consolidated List of Patches and Technology Bug Fixes (Doc ID 1594274.1)
  • For the Upgrade ->Oracle E-Business Suite Release 12.2.3 Readme (Doc ID 1586214.1)
  • For NLS- >Oracle E-Business Suite NLS Release Notes, Release 12.2 (Doc ID 1314621.1)
  • For AD and TXK Upgrade -> Applying R12.AD.C.Delta.4 and R12.TXK.C.Delta.4 Release Update Packs (Doc ID 1617461.1) 
  • For HR Legislative: DATAINSTALL AND HRGLOBAL APPLICATION: 12.2 SPECIFICS (Doc ID 1469456.1)
  • For Localization : Creating CLE Product & Applying Mandatory Patches for R12.2 EMEA Add-on Localizations (Doc ID 1628719.1) --optional
  • EMEA Add-on Localizations - Upgrade steps to 12.2 (Doc ID 1593750.1) --optional
  • For Turkish Localizations -> R12 EMEA Add-on Localizations - Turkey (Doc ID 472686.1)
Okay, lets start..

First of all, the prerequisite actions needed for the upgrade depends on the StartCD version used in creating the stage and installing EBS 12.2.. I mean, If you installed the EBS 12.2.0 from the StartCD 12.2.0.46 , you need to install a lot of patches for the technology components before upgrading to EBS 12.2.3, but if you installed EBS 12.2.0 from the StartCD 12.2.0.47, then your work is easier, as those patches are applied during the installation automatically.
Normally when you download EBS 12.2 installation media, you will get startCD 12.2.0.46, but after building your stage, you can download startCD 12.2.0.47( patch17445880) and unzip it to the same directory that startCD 12.2.0.46 resides. This action will overwrite 12.2.0.46, so it will be upgraded to 12.2.0.47.. After than action you can install your EBS 12.2.0 with startCD 12.2.0.47..

I mean;
go to stage -> cd/stage
and
unzip using overwrite -> unzip -o p17445880_R12_GENERIC.zip

If you are upgrading an already built stage. No need to use buildstage.sh in my opinion, again.
But if you didnt built the stage yet, you need to run buildStage.sh with Option 1 first "Create new stage area" followed by "Option 2" "Copy new patches to current stage area."

Ref: Oracle Support:
buildStage.sh
After creating the stage area directory it is required to manually unzip only the "Oracle E-Business Suite Release 12.2.0 Rapid Install Start Here" or unzip Patch 17873425 files. Do not manually unzip the other stage files. After unzipping the startCD, the buildStage.sh script should be run to unzip the rest of the installation software.

Running the buildStage.sh script is a mandatory pre-installation step. To create a successful R12.2 stage, you must run buildStage.sh as suggested below with "Option 1" first "Create new stage area" followed by "Option 2" "Copy new patches to current stage area.". 


Note that : A new version named RI STARTCD 12.2.0.47_5  is released nowadays.. The patch number is 17873425.  So you better use this upgdated Start Cd..

Okay, after this action we directly start patching our technology components in order to prepare our system for the EBS 12.2.3 upgrade.

Following table is from Oracle Support. So look at the included in startCD column, and apply patches according to the startCD that you have used during 12.2.0 installation. so dont install the the patches listed as included in your StartCD version.

So , for preparing the 12.2.3 upgrade, we basically patch our components in EBS 12.2 architecture.
These components are Oracle Rbms 11.2.0.3, Forms and Report Homes, Webtier, OHS 11.1.16 and Oracle Fusion Middleware 11.1.1.6.0 and Weblogic 10.3.6.0.
Following picture represents EBS R 12.2.. I have put this picture just for a general information ..

So we start patching our components one by one in order to be prepared fort he 12.2.3 upgrade.
Note that : You will find tables below. Those tables contain infomation about needed patches.. They are gathered from Oracle Support.

DATABASE PREPARATION:

These are standart RDBMS patches, to be applied through opatch.
Basically, if you have installed your EBS 12.2.0 using startCD 12.2.0.47, just apply the patches listed in "not included in any startCD section".

Note that : the patch 17461865 could not be applied because There is only the Solaris version of that patch on Oracle Support. So if your system is Linux, you can not apply it at all.
Also note that: After applying 17693770 , the document says that "apply AD.C.DELTA.4", so I skipped this step as  we would apply AD.C.DELTA.3  on the way for upgrading 12.2.3. 

Patch (Version 11.2.0.3.0)
Bug Numbers
Included in startCDLinuxx86-64Oracle Solaris on SPARC (64-bit)IBM AIX on Power Systems (64-bit)HP-UX ItaniumIBM Linux on System z seeFootnote 5Oracle Solaris on x86-64 (64-bit) seeFootnote 5
12.2.0.4612942119 (seeFootnote 1)12942119 (seeFootnote 1)12942119 (seeFootnote 1)12942119 (seeFootnote 1)12942119 (seeFootnote 1)12942119 (seeFootnote 1)12942119
N/AN/A13443029N/AN/AN/A13443029
N/A13920888N/AN/AN/AN/A13920888
13931044139310441393104413931044139310441393104413931044
14123213141232131412321314123213141232131412321314123213
16105745161057451610574516105745161057451610574516105745
16303726 (seeFootnote 2)16303726 (seeFootnote 2)16303726 (seeFootnote 2)16303726 (seeFootnote 2)16303726 (seeFootnote 2)16303726 (seeFootnote 2)12574386,10019002, 10427365
1634248616342486163424861634248616342486163424864247037, 8547978, 9858539, 12780983, 12845115, 12849688, 12949905, 12949919, 12950694, 12951696, 12955701, 12965899, 12971775, 12985184, 13004894, 13023632, 13036331, 13040331, 13070939, 13146719, 13258936, 13259364, 13366202, 13366268, 13388104, 13466801, 13477790, 13495307, 13499128, 13506110, 13528551, 13544396, 13780816, 13808632, 13923995, 14005749, 14013094, 14162791, 14207902, 14237793, 14296972, 14587310, 14698700, 14751895 , 15982436
16438289164382891643828916438289164382891643828912834800, 14398795
16587934 (seeFootnote 9)16587934 (seeFootnote 9)16587934 (seeFootnote 9)16587934 (seeFootnote 9)16587934 (seeFootnote 9)16587934 (seeFootnote 9)14158012, 14215010, 14465787
12.2.0.4713001379130013791300137913001379130013791300137913001379
16924879169248791692487916924879169248791692487916924879
16929165169291651692916516929165169291651692916516929165
16992075169920751699207516992075169920751699207516992075
17047617 (seeFootnote 10)17047617 (seeFootnote 10)17047617 (seeFootnote 10)17047617 (seeFootnote 10)17047617 (seeFootnote 10)17047617 (seeFootnote 10)12960302, 15938047, 14140535, 14641969, 12923168, 14204172, 12657924
17184721171847211718472117184721171847211718472117184721
17461865 (seeFootnote 7)17461865 (seeFootnote 7)17461865 (seeFootnote 7)17461865 (seeFootnote 7)17461865 (seeFootnote 7)17461865 (seeFootnote 7)11071989, 12646784, 12764337, 12913474, 13332439, 13502183, 13584130, 13786142, 13839641, 13945708, 14163397, 16024441, 16101465, 16187992, 16505333, 16710753, 16864048, 17050888, 13035360
17468141 (seeFootnote 3)17468141 (seeFootnote 3)17468141 (seeFootnote 3)17468141 (seeFootnote 3)17468141 (seeFootnote 3)17468141 (seeFootnote 3)17468141
17502060 (seeFootnote 4)17502060 (seeFootnote 4)17502060 (seeFootnote 4)17502060 (seeFootnote 4)17502060 (seeFootnote 4)17502060 (seeFootnote 4)12749597, 14692386, 17421340
Not included in any startCD17600719176007191760071917600719176007191760071917600719
17693770 (seeFootnote 8)17693770 (seeFootnote 8)17693770 (seeFootnote 8)17693770 (seeFootnote 8)17693770 (seeFootnote 8)17693770 (seeFootnote 8)12749597, 14692386, 17421340, 17487358
17875948178759481787594817875948178759481787594814158012, 14215010, 14465787, 17654622
18116376181163761811637618116376181163761811637612811829, 12657924, 12923168, 12960302, 14140535, 14204172, 14641969, 15938047
18259911 (seeFootnote 6)18259911 (seeFootnote 6)18259911 (seeFootnote 6)18259911 (seeFootnote 6)18259911 (seeFootnote 6)18259911 (seeFootnote 6)4189542, 11071989, 11896575, 12646784, 12764337, 12913474, 13035360, 13332439, 13502183, 13584130, 13786142, 13839641, 13945708, 14163397, 16024441, 16101465, 16505333, 16710753, 16864048, 17050888, 17174582, 17478145, 17614134, 17781991, 17883081, 18241194

Footnote 1 - If Patch 12942119 was installed via Patch 16342486, you should roll it back and re-install it with the version of Patch 12942119 from My Oracle Support.
Footnote 2 - After applying the patch, you must run following Oracle E-Business Suite Release 12.2 specific script (which is not documented in the patch readme):
S perl $OH/nls/data/old/cr9idata.pl
Footnote 3 - If Patch 16702016 was installed, you should roll it back and re-install it with the version of Patch 17468141 from My Oracle Support.
Footnote 4 - If Patch 16819180 was installed or if Patch 14832335 was installed, you should roll it back before installing Patch 17502060 from My Oracle Support. For a more recent version of this fix refer to Patch 17693770 and its associated footnote in 'Not included in any startCD'.
Footnote 5 – IBM: Linux on System z and Oracle Solaris on x86-64 (64-bit) are the “Database Tier only” platforms for Oracle E-Business Suite Release 12.2. 'Included in startCD' column is not relevant for these two platforms. Apply all the patches listed for your platform.
Footnote 6 - If Patch 17461865 was installed, you should roll it back before installing Patch 18259911 from My Oracle Support.
Footnote 7 - Database bug fix 16187992 has been replaced with database bug fix 18241194, this is included in Patch 18259911 and should be installed instead of Patch 17461865.
Footnote 8 - This database patch has an AD code dependency; the following will help you identify the patch to apply after Patch 17693770 has been applied. In both cases ensure you follow the AD patch documentation; if you do not intend to apply one of the mentioned AD patches afterwards then you must not apply Patch 17693770.
For new Oracle E-Business Suite Release 12.2 installations you should install Patch 17766337
For upgrades to Oracle E-Business Suite Release 12.2 from an earlier Oracle E-Business Suite release you should install Patch 17903664
If Patch 17502060 was installed, you should roll it back before installing Patch 17693770 from My Oracle Support.
Footnote 9 - For a more recent version of this fix refer to Patch 17875948 in 'Not included in any startCD'.
Footnote 10 - For a more recent version of this fix refer to Patch 18116376 in 'Not included in any startCD'.

FORMS AND REPORTS PREPARATION: (10.1.2.3 and 10.1.0.5 Oracle Homes)

These patches will be applied using opatch mostly. 
Basically, if you have installed your EBS 12.2.0 using startCD 12.2.0.47, just apply the patches listed in "not included in any startCD section".

Note that : There is an issue with patch 17303472. It fails in command "dcmctl redeployApplication -f $ORACLE_HOME/forms/j2ee/formsapp.ear -a formsapp -co OC4J _BI_Forms",  the produced error is "no such file or directory. java.io.FileNotFoundException -> sysmgmtProperties.dtd"
For this error, I suspected from OUI version and  checked it by running runInstaller from 10.1.2 home. In Installer dialog box, I clicked the "about Oracle Universal Installer ", and saw that it is 10.1.0.6.. So this was not be the problem. Also opatch version was 1.0.0.0.64. So this was not the problem neither. 
The problem was the directory structure was missing some files. Anyways, I ignored that error, as it is related with OC4J_BI_Forms.

Also note that : Patches for 10.1.2 and 10.1.5 are both applied to the 10.1.2 Oracle Home.,

Patch (Version 10.1.2.3.0)
Bug Number
Included in startCDLinux x86-64Oracle Solaris on SPARC (64-bit)IBM AIX on Power Systems (64-bit)HP-UX Itanium
Microsoft Windows x64 (64-bit)
12.2.0.46N/AN/A5705981N/AN/A5705981
5917053591705359170535917053174161175917053
6371228637122863712286371228174161176371228
7016961701696170169617016961129535287016961
7121788712178871217887121788N/A7121788
714040571404057140405714040571404056116405, 6764875
8339196833919683391968339196174161178339196
852819485281948528194852819485281948528194
855701985570198557019855701985570198557019
N/AN/A8925337N/AN/A8925337
N/A8904424N/AN/AN/A8904424
10152652101526521015265210152652129535286497684, 9726201
117806691178066911780669117806691741611711780669
12379038123790381237903812379038N/A12379038
12837860128378601283786012837860128378646490362, 8447875, 8351021, 9273888, 9974899, 12434128, 12419228, 12828135
128482281284822812848228128482281284822812848228
128870681288706812887068128870681741611712887068
128984231289842312898423128984231295352812898423
N/AN/A13686664N/AN/A13686664
14353879143538791435387914353879N/A14353879
N/AN/A14029977N/AN/A7230832, 13579719
143745871437458714374587143745871437458714374587
14825718 (see Footnote 1)14825718 (see Footnote 1)14825718 (see Footnote 1)14825718 (see Footnote 1)14825718 (seeFootnote 1)14825718
162095201620952016209520162095201620952016209520
12.2.0.478300196 (see Footnote 3)8300196 (see Footnote 3)8300196 (see Footnote 3)8300196 (see Footnote 3)8300196 (seeFootnote 3)8300196
14703419 (see Footnote 4 &Footnote 6)N/AN/AN/AN/A14703419
16207165 (see Footnote 5 &Footnote 6)N/AN/AN/AN/A16207165
16241466 (see Footnote 6)N/AN/AN/AN/A8551790, 16234436
16271876 (see Footnote 6)N/AN/AN/AN/A16271876
167783821677838216778382167783821741611716778382
17303472 (see Footnote 2)17303472 (see Footnote 2)17303472 (see Footnote 2)17303472 (see Footnote 2)17303472 (seeFootnote 2)17303472
N/AN/AN/AN/A1741611717026167
Not included in any startCD565959456595945659594565959456595945659594
8943095894309589430958943095174161178943095
159002141590021415900214159002141661562515900214
17645157 (see Footnote 6 &Footnote 7)N/AN/AN/AN/A17645157
17907988 (see Footnote 6)N/AN/AN/AN/A7121788, 17594101

Footnote 1 - This is the Forms Bundle Patch (10.1.2.3.2). For a full list of inclusions and OUI/OPatch requirements, refer to the patch README.
Footnote 2 - This is the Forms Overlay Patch on 10.1.2.3.2. For a full list of inclusions and OUI/OPatch requirements, refer to the patch README.
Footnote 3 - If a conflict is reported with 13367912, either roll back 13367912 first (opatch rollback, then apply 8300196), or allow opatch to roll it back when applying Patch 8300196.
Footnote 4 - If a conflict is reported with 14029881, either roll back 14029881 first (opatch rollback, then apply 14703419), or allow opatch to roll it back when applying Patch 14703419.
Footnote 5 - If a conflict is reported with 13579719, either roll back 13579719 first (opatch rollback, then apply 16207165), or allow opatch to roll it back when applying Patch 16207165.
Footnote 6 - This fix is only required on Oracle E-Business Suite Release 12.2 installation using JDK 7.0 and must not be applied when JDK 6.0 is used. Refer to My Oracle Support Knowledge Document 1530033.1 for additional information on using JDK 7.0.
Footnote 7 - If a conflict is reported with 16207165, either roll back 16207165 first (opatch rollback, then apply 17645157), or allow opatch to roll it back when applying <Patch 17645157>.


Patch (Version 10.1.0.5.0)
Bug Number
Included in startCDLinux x86-64Oracle Solaris on SPARC (64-bit)IBM AIX on Power Systems (64-bit)HP-UX ItaniumMicrosoft Windows x64 (64-bit)
12.2.0.463559326355932635593263559326Bundle Patch 44 (13413002) (see Footnote 1)3559326
5261796526179652617965261796Bundle Patch 44 (13413002) (see Footnote 1)3662022, 4598613, 5056864
5394728539472853947285394728Bundle Patch 44 (13413002) (see Footnote 1)5394728
5651245565124556512455651245Bundle Patch 44 (13413002) (see Footnote 1)5651245
6400501640050164005016400501Bundle Patch 44 (13413002) (see Footnote 1)6400501
7695070769507076950707695070Bundle Patch 44 (13413002) (see Footnote 1)7695070
129499051294990512949905129499051294990512949905
16275529162755291627552916275529162755295010130, 8604052, 10019002, 10427365, 12332340, 12807257
12.2.0.474047619404761940476194047619176045404047619
5604120560412056041205604120176045405604120

WEBTIER and OHS 11.1.1.6 PREPARATION:

Patch (Version 11.1.1.6.0)
Bug Number
Included in startCDLinuxx86-64Oracle Solaris on SPARC (64-bit)IBM AIX on Power Systems (64-bit)HP-UX ItaniumMicrosoft Windows x64 (64-bit)
12.2.0.46106387581063875810638758106387581063875810638758
136432111364321113643211136432111364321113104787,13588565, 13637410
146168191461681914616819146168191461681914616819
161998941619989416199894161998941619989416199894
12.2.0.47173255591732555917325559173255591732555912959202, 13025456, 13837241, 14003476, 14110089, 14345153, 14386112, 16021044, 16856341
Note: Release 11.1.0.7 should be selected for patch download, and installed in the 11.1.1.6 ORACLE_HOME.


Patch (Version 11.1.0.7)
Bug Number
Included in startCDLinux x86-64Oracle Solaris on SPARC (64-bit)IBM AIX on Power Systems (64-bit)HP-UX ItaniumMicrosoft Windows x64 (64-bit)
12.2.0.467695070769507076950707695070126952787695070
129499051294990512949905129499051294990512949905


ORACLE FUSION MIDDLEWARE COMMON 11.1.1.6 PREPARATION:

Note that: There is Oracle Home named Common. That's why the patches for Common should be applied to that home. Similarly, Webtier patches should applied to the webtier .

Patch (Version 11.1.1.6.0)
Bug Number
Included in startCDLinuxx86-64Oracle Solaris on SPARC (64-bit)IBM AIX on Power Systems (64-bit)HP-UX ItaniumMicrosoft Windows x64 (64-bit)
12.2.0.46138775591387755913877559138775591387755913877559
14386347143863471438634714386347143863476256668, 10427953, 12752475, 12940997, 12941016, 13001791, 13004138, 13096119, 13384467, 13444166, 13444266, 13494184, 13522675, 13531123, 13589107, 13597115, 13737695, 13974083, 14111409, 14156116
145885991458859914588599145885991458859914588599
161998941619989416199894161998941619989416199894
12.2.0.47127145671271456712714567127145671271456712714567
Not included in any startCD990568599056859905685990568599056859905685
139476081394760813947608139476081394760813947608
173943191739431917394319173943191739431917394319

  ORACLE WEBLOGIC 10.3.6.0 PREPARATION:

Note that , the weblogic patches are applied through Oracle Smart Update utility. This utility is really amazing. It is easy to use. The best patching utility I have ever seen:) You just copy your patch to the stage, and then choose the patch from the gui, and click the apply button :)
Oracle Smart Update can be run from : $FMW_HOME/utils/bsu directory, using bsu.sh




Included in startCDPatch NumberBSU Patch IDBug Number
12.2.0.4613572948LL4G13572948
13845626YHJK13845626
14272383S9MN14272383
14597598VFS814597598
14621810TTEM14621810
1364248532I213642485
12.2.0.4713729611 (see Footnote 1)YIJF13729611
16083651 (see Footnote 2)D33T16083651
Not included in any startCD17495356CIH817220061, 17331802, 17351504

Footnote 1 - This is the 10.3.6.0.4 overlay patch. You should apply the 10.3.6.0.4 version of this patch after applying 16083651.
Footnote 2 - This is the 10.3.6.0.4 PSU. For a list of inclusions refer to My Oracle Support Knowledge Document 1544308.1.
There are two conflict scenarios you may encounter with the 10.3.6.0.4 PSU or its overlay patch:
If you encounter conflicts when applying the PSU, you should roll back all conflicting WLS patches, apply this PSU patch, and then apply overlay Patch 13729611 (YIJF). For more information, refer to Document 1306505.1 Announcing Oracle WebLogic Server PSUs (Patch Set Updates).
If you encounter conflicts with 14049348 (3ZY8) and 13583235 (WCD1), perform the following steps in the order shown:
Uninstall the conflicting patches.
Install WLS 10.3.6.0.4 PSU patch 16083651 (D33T).
Install the 10.3.6.0.4 version of overlay patch 13729611 (YIJF) on top of WLS 10.3.6.0.4.
APPLYING NEEDED PREREQ  PATCHES FOR THE UPGRADE:

After preparing our technological environment for the upgrade. We start patching EBS using adop..
Adop is a new utility form patching EBS 12.2 environments. The patches are applied online. Also we have hotpatch option as in the old releases.

So we start with patch 17204589. This patch is Consolidated Seed Table Upgrade. We apply it as hotpatch.
To apply 17204589,  stop your application services using adstpall.sh :
sh $ADMIN_SCRIPTS_HOME/adstpall.sh
Then open only the weblogic server : This is required for adop..
sh $ADMIN_SCRIPTS_HOME/adadminsrvctl.sh start
Finally apply the patch using hotpatch..
adop phase=apply patches=17204589 hotpatch=yes

NLS version of 17204589 is not required even if you have other languages on your system. Although It is not required,  you may apply it.. But be aware that you may encounter errors on this.

Note that , when patching using adop; always unzip your zip files into the $APPL_TOP_NE/../patch directory.. For example: unzip p11111111.zip -d $APPL_TOP_NE/../patch

Also Note that: when you use hotpatch=yes as command line argument for adop. Adop uses run filesystem to apply the patches. So the patches are applied directly to the running filesystem..
The new Filesystem concept in EBS 12.2 can be described by the following picture:


as you see above, we have to filesystems. One is for running environment, and the other one is for patching.. When we apply patches without using hotpach we deploy the new things in to the patch filesystem so without effecting the running environment.. then we cutover and switch the filesystem in a appropriate time, but if we use hotpatch=yes, we deploy the new thing directly to the running filesystem:.

Okay , lets come back to our topic..
After applying patch 17204589, we continue with patch 17023760, which is R12.AD.C.Delta.3..

Apply patch 17023760 using hotpatch option:
adop phase=apply patches=17023760 hotpatch=yes

Then continue with AD.TXT.C.Delta.3 upgrade;
adop phase=apply patches=17021789 hotpatch=yes

After applying those patches, you will need to Instantiate  admin.conf, ssl.conf and mod_wl_ohs.conf on the run filesystem. You can use the following commands to do that..
You will also need to reinitiate thise configuration in next steps..

$ADJVAPRG oracle.apps.ad.autoconfig.InstantiateFile \
-e $CONTEXT_FILE \
-d $FND_TOP/admin/driver/ohstmpl.drv \
-tmpl $FND_TOP/admin/template/ssl_conf_FMW.tmp

$ADJVAPRG oracle.apps.ad.autoconfig.InstantiateFile \
-e $CONTEXT_FILE \
-d $FND_TOP/admin/driver/ohstmpl.drv \
-tmpl $FND_TOP/admin/template/mod_wl_ohs_conf_FMW.tmp

$ADJVAPRG oracle.apps.ad.autoconfig.InstantiateFile \
-e $CONTEXT_FILE \
-d $FND_TOP/admin/driver/ohstmpl.drv \
-tmpl $FND_TOP/admin/template/admin_conf_FMW.tmp

perl $FND_TOP/patch/115/bin/txkSetAppsConf.pl \
-contextfile=$CONTEXT_FILE \
-outfile=$INST_TOP/logs/appl/rgf/TXK/erman.log

After initiating the conf file, run adop fs_clone to syncronize the run and patch filesystems, as we  used only run filesystem using hotpatch option for the patches above..

adop phase=fs_clone

APPLYING THE UPGRADE PATCH:

Now the upgrade patch can be applied.
The name of the patch is EBS 12.2.3 Patch 17020683. 
You need to apply this patch using adop online.
Firstly , download and unzip the patch to the $APPL_TOP_NE/../patch directory.
(always unzip the patches in to the $APPL_TOP_NE, as long as you have free space on the underlying disk)
Then set your run filesystem environment and start with adop prepare phase.

adop phase=prepare
Then use adop apply to apply the patch
adop phase=apply patches=17020683
You can also apply the online help patch(17020685) together with the 17020683 , so if you want to do so, use the following command;
adop phase=apply patches=17020683,17020685

After completing the apply phase reinitiate the needed configuration files again..

$ADJVAPRG oracle.apps.ad.autoconfig.InstantiateFile \
-e $CONTEXT_FILE \
-d $FND_TOP/admin/driver/ohstmpl.drv \
-tmpl $FND_TOP/admin/template/ssl_conf_FMW.tmp

$ADJVAPRG oracle.apps.ad.autoconfig.InstantiateFile \
-e $CONTEXT_FILE \
-d $FND_TOP/admin/driver/ohstmpl.drv \
-tmpl $FND_TOP/admin/template/mod_wl_ohs_conf_FMW.tmp

$ADJVAPRG oracle.apps.ad.autoconfig.InstantiateFile \
-e $CONTEXT_FILE \
-d $FND_TOP/admin/driver/ohstmpl.drv \
-tmpl $FND_TOP/admin/template/admin_conf_FMW.tmp

perl $FND_TOP/patch/115/bin/txkSetAppsConf.pl \
-contextfile=$CONTEXT_FILE \
-outfile=$INST_TOP/logs/appl/rgf/TXK/erman2.log

So after completing the initiations, complete the patch cycle by  using the following adop commands;

adop phase=finalize
adop phase=cutover
adop phase=cleanup

If you want to install a translated version of 17020683 , apply it in a new patching cycle, at this point.
Lastly, syncronize your patch and run filesystem using adop phase=fs_clone..
That's it your EBS is now 12.2.3.
On the other hand, this is not enough for an EBS 12.2.3 implementation. Thus we have post steps..
After the upgrade, it is good to apply Oracle Applications CPU Jan patch 17631896 to our environment. We use adop online patching for applying this patch. This means, after we unzip the revelant patch file into the $APPL_TOP_NE/../patch directory, we issue the following command using run edition environment..
adop phase=prepare
adop phase=apply
adop phase=finalize
adop phase=cutover
adop phase=cleanup

Note that : Environment setting can be made by the following script. 
EBSapps.env. 
You can set run filesystem with . /u01/oracle/TEST/EBSapps.env run
or you can set patch filesystem with . /u01/oracle/TEST/EBSapps.env patch
It is good to put that script in application owner's .bash_profile as follows;
. /u01/oracle/TEST/EBSapps.env run
I prefer to set the run filesystem, because or patching activities requires run filesystem to be sourced..

Okay after applying the CPU patch, we continue with HR legislative..
We use hrglobal.drv for this. Again adop is used for this step.
Applying hrglobal is a classical process.. The steps to be taken for applying it is almost the same  as 12.1.
We just need to create a patching cycle before executing Data Install Program.
We choose language to install by running  the java data install program:
adop prepare
java oracle.apps.per.DataInstall [apps_un] [apps_pw] thin [connection]
with [connection] in form: hostname:dbport:oraclesid 
For Example: java oracle.apps.per.DataInstall apps apps thin ermanserver:1521:TEST

+-------------------------------------------------+
|             DataInstall Main Menu               |
+-------------------------------------------------+

1.    Select legislative data to install/upgrade

2.    Select college data to install/upgrade

3.    Select JIT to install/upgrade

4.    Exit to confirmation menu

Selected : 1


 # Localisation         Product(s)               Leg. Data? Action
-- -------------------  ------------------------ ---------- -------------
 1 Global               Human Resources          Installed  
 2 Australia            Human Resources                     
 3 Australia            Payroll                             
 4 Belgium              Human Resources                     
 5 Canada               Human Resources                     
 6 Canada               Payroll                             
 7 China                Human Resources                     
 8 China                Payroll                             
 9 Denmark              Human Resources                     
10 Denmark              Payroll                             
11 Finland              Human Resources                     
12 Finland              Payroll                             
13 France               Human Resources                     
14 Germany              Human Resources                     
15 Hong Kong            Human Resources                     
16 Hong Kong            Payroll                             
17 Hungary              Human Resources                     
18 India                Human Resources                     
19 India                Payroll                             
20 Ireland              Human Resources                     
21 Ireland              Payroll                             
22 Italy                Human Resources                     
23 Japan                Human Resources                     
24 Japan                Payroll                             
25 Korea, Republic of   Human Resources                     
26 Korea, Republic of   Payroll                             
27 Kuwait               Human Resources                     
28 Kuwait               Payroll                             
29 Mexico               Human Resources                     
30 Mexico               Payroll                             
31 Netherlands          Human Resources                     
32 Netherlands          Payroll                             
33 New Zealand          Human Resources                     
34 New Zealand          Payroll                             
35 Norway               Human Resources                     
36 Norway               Payroll                             
37 Poland               Human Resources                     
38 Romania              Human Resources                     
39 Romania              Payroll                             
40 Russian Federation   Human Resources                     
41 Saudi Arabia         Human Resources                     
42 Saudi Arabia         Payroll                             
43 Singapore            Human Resources                     
44 Singapore            Payroll                             
45 South Africa         Human Resources                     
46 South Africa         Payroll                             
47 Spain                Human Resources                     
48 Sweden               Human Resources                     
49 Sweden               Payroll                             
50 United Arab Emirates Human Resources                     
51 United Arab Emirates Payroll                             
52 United Kingdom       Human Resources                     
53 United Kingdom       Payroll                             
54 United States        Human Resources                     
55 United States        Payroll                             
56 United States        US Federal Human Resources           

 <Product #><Action> - Change Action
 where <Action> is [I : Install, C : Clear]

Selected 1I , to install Global Legislative data . 

After saving and quiting from Data Install, use adop to apply Hr Global patch located in $PER_TOP/patch/115/driver..

Start a new cycle; and apply the patch using hrglobal.drv


adop phase=apply patchtop=$PER_TOP/patch/115 patches=driver:hrglobal.drv options=nocopyportion,nogenerateportion,forceapply
adop phase = finalize
adop phase = cutover
adop phase = cleanup

Note that , This time we used patchtop, nocopyportion, nogenerateportion and forceapply options for the patch application. These arguments and options are needed in order to avoid adop trying to sync the file system when applying a patch. Because the driver file contains database operations only Also we use forceapply for making adop reapply the patch even if it was applied during an upgrade.. 

Also note that; while applying hrglobal using adop, some workers can encounter an error wile loading pqpglext.ldt.. The command for loading this ldt can be found on workers's log.. Anyways, you can skip this error, and issue the same command after the patch installation, it will be succeeded. 


After the hrglobal we apply the NLS translated version of EBS 12.2.3 patch (17020683) and Online Help patch (if we didnt apply it with the  17020683 already).. Also if the NLS version of this Help patch is ready, apply that too..
These patches are applied with the same method (adop)

After NLS translated version of the upgrade patches, we continue with the AD.C.delta.4 and AD.Txk.C.delta.4 upgrades.
Note that: Before upgrading to Delta.4, the AD_ZD_PREP package may be invalid. If this package is invalid, you can not apply localization patches, because in order to use adsplice for CLE installation, you will need that package.
On the other hand, Delta.4 upgrades will fix that package...

It is also important to know that AD.C.DELTA 4 is dependent on TXK.DELTA.4 .So TXK.DELTA.4 should be applied first.
Review the document  " Applying R12.AD.C.Delta.4(Patch 17766337) and R12.TXK.C.Delta.4(Patch 17893964) Release Update Packs (Doc ID 1617461.1)" carefully, as there are multiple prereq patches which need to be applied during the apply phase of Ad.C.Delta.4. and Txk.delta.4.
Download those prereq patches, too.. Then unzip them into the $APPL_TOP_NE directory before starting the patch application process for AD.TXT.DELTA.4.. They will be merged and applied using the following commands;

Note that , before starting the apply phase, execute adgrants sql comes with the patch 17766337.. The need for this action is documented in readme file of Patch 17766337.
After executing adgrants.sql properly, continue with the apply phase.

Set your run filesystem environment and execute the followings; (supposing all the patches are unzipped into the $APPL_TOP_NE/../patch directory

adop phase=prepare
For TXK Delta4:
adop phase=apply patches=17893964,18332816 merge=yes 
For AD.C Delta4:
adop phase=apply patches=17766337,18628932,18470942,18274429  merge=yes

Note that : we dont use the hotpach option, and we applied the patches in the same patching cycle.

After successfully applying the patches above, reinitiate the configuration files as we have done in previous steps.

$ADJVAPRG oracle.apps.ad.autoconfig.InstantiateFile \
-e $CONTEXT_FILE \
-d $FND_TOP/admin/driver/ohstmpl.drv \
-tmpl $FND_TOP/admin/template/ssl_conf_FMW.tmp

$ADJVAPRG oracle.apps.ad.autoconfig.InstantiateFile \
-e $CONTEXT_FILE \
-d $FND_TOP/admin/driver/ohstmpl.drv \
-tmpl $FND_TOP/admin/template/mod_wl_ohs_conf_FMW.tmp

$ADJVAPRG oracle.apps.ad.autoconfig.InstantiateFile \
-e $CONTEXT_FILE \
-d $FND_TOP/admin/driver/ohstmpl.drv \
-tmpl $FND_TOP/admin/template/admin_conf_FMW.tmp

perl $FND_TOP/patch/115/bin/txkSetAppsConf.pl \
-contextfile=$CONTEXT_FILE \
-outfile=$INST_TOP/logs/appl/rgf/TXK/erman.log

After initiating the needed file, complete the patch cycle by issuing the following commands;

adop phase=finalize
adop phase=cutover
adop phase=cleanup

Lastly, it is good to make an fs_clone to prepare the system for future upgrades/patches.
adop phase=fs_clone

Okay.. Our work is finished. At this point, we have a fresh installed  AD.C and AD.TXK. Delta.4 upgraded EBS 12.2.3 with translations and  legislative patches are applied.

After this point, according to the project we may apply localization patches , or confiure additional components like Governor, Hr Organization Chart etc.
I will explain localization patches next..  I didnt put localization process for EBS 12.2.3 because it is a little buggy.. I mean that , there are problems in localization patches for 12.2.3..  I m working with Oracle Support to fix the issues. Once the issues will be fixed, I will make a dedicated blog post about EBS 12.2.3 localization patches. 

I hope you will find this document useful..