EBS R12 -- Hr Organization Chart redirects to the home page!

I have alrady written the article "Implementing Hr Organization in EBS"
Covered the steps , shared the screenshots and give some diagnostics info which may be used against the problems on the way..
http://ermanarslan.blogspot.com.tr/2014/05/ebs-122-implementing-hr-organization.html

But there are some problems, that can not be corrected easily..
Such problems make us configure our EBS environment according to the needs of Hr Org Chart application..
Yes! I m talking about the domain names..
Hr Org Chart want us to have the same domain name for all of the servers involved between the process for reaching Hr Org Chart from the EBS..

Load Balancer, Application server , SSO server, Hr Org Chart Server , you name it :) , they all need to have the same domain.. We need to see the same domain in fnd_nodes for all the nodes:)
Even if we dont have such an environment, we will end  with cookie problems.. Thus when we click on the Hr Org Chart link in EBS (For ex: Manager Self Service - > Talent Profile), the browser will refresh the page , it will go to the Hr Org Chart url but will come back to the same page again.. In short; Hr Org Chart will not open...
I have talked about this with Oracle Support, I have spoken with the associated developer .. This unfortunetaly is the fact.. 

EBS R12-- Problems in Enabling SSL on SSO 10g

In this post, I will point out some important aspects about enabling SSL on SSO 10g..
As you may know, support for SSO 10g is limited.. Thus, Oracle directs us to use Oracle Access Manager rather than SSO 10g.. So, the information that we can reach in the internet  is also limited ..
This become a problem when we hit some bug, or when we need a special configuration.. In such situations , unfortuneatly we stand alone.. That is; we need to solve the problems ourselves by making advanced diagnostics..

When we talk about SSL on SSO 10g. We actually talk about 3 tiers.
One of them is the Application that uses SSO to authenticate its users..
In this story; this application is EBS R12 ..
Ofcouse the midd tier is SSO 10g which is something like an  Oracle Application Server.
The 3rd tier is the OID and its database.. In this story , it is OID 11g which runs on Weblogic Server..
,
In order to diagnose  the integration problems, we need to know the following;

The txkrun.pl script stored in EBS .. This script makes the EBS to register itself with SSO through OID.
The scripts like ssocfg.sh, ssoreg.sh stored in SSO 10g server.. ssocfg.sh configures the SSO and ssoreg.sh register the SSO  and also updates the mod_osso registration record in osso.conf.
Owm in SSO 10g ( oracle wallet manager) which is used to store our SSL certificates.
OID 11g configuration tables to check the registered partner application configuration from the backend..
odsm in OID 11g to check and configure things like operationsUrl for such a configuration change..

Okay.. I will give two problems and their solutions to show what to check while dealing problems in an EBS integrated SSO10g-OID11g configuration after enabling SSL in SSO.

First of all, we can follow the steps described in the following link to enable SSL on SSO 10g.
http://ermanarslan.blogspot.com.tr/2015/02/sso-10g-enabling-ssl-on-ebs-integrated.html

This steps seems sufficient for such an operation, but still some problems may arise .

Possible problems:

1)
unable to logout from EBS after enabling SSL on SSO...
When we encounter this problem, we need to check the urls.. I mean the urls used in SSO login and SSO logout. I can say that : a consistency problem between these urls will create logout problem.
For example: if the SSO login url is https://ermanhost:4443/... and if te logout URL is https://ermanhost.ermandomain:4443/... , then we will have this logout problems.. That is, domain name is important..
To check the SSO urls used in EBS logout&login we can use  orasso.wwsso_papp_configuration_inf_t table in the OID database..
Browser http trace may also help..
If we see an inconsistency, then we need to use ssoreg.sh to update this records accordingly..
Lastly, we take the action to deregister and register EBS using txkrun.pl

2)
certificate warnings in Browsers..
If domain name used in login/logout urls  is different than the domain name in the ssl certificate , then we will have certificate not trusted warning in Browsers..
For example: our login url is ermanhost.ermandomain:4443 but the certificate we use in SSO 10g is signed for ermanhost..
In such a situation, the solution can be changing(using ssoreg.sh) the login/logout urls to match the info stored in ssl certificate. Lastly, we take the action to deregister and register EBS using txkrun.pl
 Alternatively , we can request a new certificate that matches our login/logout urls from the CA  and make the SSO use that certificate.
CN in the certificate should match the FQDN used in login/logout urls..
For example , if we have CN=ermanhost in the certificate , then we need to have https://ermanhost:4443.. written in our login/logout urls..

3)
The page can not be displayed.. Unable to reach the sso login page from the browsers using its https url even if everything seems ok.
In this situation, SSL may be disabled in our Browsers.. Security policy in our environment may want us to use TLS instead of SSL ..
If that 's the scenario, this is described in  TLS 1.0 Handshake Fails With "SSL call to NZ function nzos_Handshake failed with error 29014" ( Doc ID 470123.1 )
The fix is applying patch 6370967.. Ppply this patch in SSO Oracle Home and retest..


In conclusion, I can say that dealing with problems in SSO infrastructure is not an easy thing.. We cant get enough support when something strange happens on the way :)

However, I write this post to light the way for us to realize the facts in such a integrated configuration. Having the general knowledge about the configuration architecture as a whole let us to analyze the problems on our own, as  enabling SSL in SSO 10g affects the configurations stored in OID and EBS , too.. Even the browser congurations may affect our success in such an environment...

Weblogic -- a quick look at WLST and typical use case for addressing edit lock problems in EBS 12.2

WLST in Weblogic is a java based cli, that can be used to monitor and administer Weblogic servers and domains. We can also say that WLST is command line utility , which eases some of the administration works.

An example of the benefits of the WLS can be ability to access the Node Manager and starting,stopping or suspending both local or remote managed servers from the command line.. No need to have an admin server up & running, no need for a browser and so on.

Usages : Online and Offline

There are 2 usage for using WLST. Online and offline.When using WLST online , we can connect to Admin or Managed Servers . So we can access the Mbeans(Java objects which provides managed interfaces) .
In offline use, offcourse we have limited functionality.. Still, we can create a new domain or create/change some configuration files like config.xml..

Modes: 

There are 3 modes for using WLST.

Direct/Interactive/Command Line Mode, Script mode, Embedded Mode..

Interactive mode is a standard cli mode, like sqlplus, we execute commands and WLST returns back the response interactively..

We can also supply scripts to WLST. Scripts containing WLST commands in a text file with .py extension. This is called Script mode.

We can also use embedded mode for using WLST .. That is ;we can embed WLST interpreter in our Java code and use it to run WLST commands and scripts.

Okay.. After a general introduction, lets proceed with an example of the usage ;

Lets start by checking the status of a managed server;

First we set our domain env.. -> . $EBS_DOMAIN_HOME/bin/setDomainEnv.sh

java weblogic.WLST

wls:/offline> connect('weblogic','welcome1','t3://ermanhost:7001')

wls:/EBS_domain_ORATEST/serverConfig> cd ('Servers')

wls:/EBS_domain_ORATEST/serverConfig/Servers> ls()

dr--   AdminServer

dr--   forms-c4ws_server1

dr--   forms_server1

dr--   oacore_server1

dr--   oafm_server1

Note that ; we can use ls and cd commands :) ls() or even cd ('..') works well :)

Okay, we have 5 servers in this domain.. This is an EBS 12.2 instance bytheway..

Lets check the status of oacore_server1

wls:/EBS_domain_ORATEST/serverConfig>  print state('oacore_server1')

Current state of 'oacore_server1' : RUNNING

Okay. lets use some help(all) and see what more we can do with it.

 help('all') 

    help('activate')         Activate the changes.

    help('addListener')      Add a JMX listener to the specified MBean.

    help('addTemplate')      Extend the current domain.

    help('adminHome')        Administration MBeanHome.

    help('assign')           Assign resources to one or more destinations.

    help('assignAll')        (Deprecated) Assign all applications or services.

    help('cancelEdit')       Cancel an edit session.

    help('cd')               Navigate the hierarchy of beans.

    help('closeDomain')      Close the current domain.

    help('closeTemplate')    Close the current domain template.

    help('closestore')       Closes a store.

    help('cmo')              Current Management Object.

    help('compactstore')     Compacts and defragments the space occupied by a file store.

    help('config')           (Deprecated) Navigate to the last MBean in configuration hierarchy.

    help('configToScript')   Convert a domain configuration to WLST script.

    help('connect')          Connect WLST to a WebLogic Server instance.

    help('connected')        Variable indicating whether WLST is connected.

    help('create')           Create a configuration bean.

    help('createDomain')     Create a new domain.

    help('currentTree')      Return the current location in the hierarchy.

    help('custom')           Navigate to the root of custom MBeans.

    help('delete')           Delete a configuration bean.

    help('deploy')           Deploy an application.

    help('disconnect')       Disconnect WLST.

    help('distributeApplication')Copy the deployment bundle to targets.

    help('domainConfig')     Navigate to last domain configuration MBean or root.

    help('domainCustom')     Navigate to the root of domain custom MBeans.

    help('domainName')       Name of the domain to which WLST is connected.

    help('domainRuntime')    Navigate to last domain runtime MBean or root.

    help('domainRuntimeService')DomainRuntimeServiceMBean MBean.

    help('dumpStack')        Display stack trace from the last exception.

    help('dumpVariables')    Display all the variables used by WLST.

    help('dumpstore')        Dumps store contents in human-readable format to an XML file.

    help('edit')             Navigate to last edit configuration MBean or root.

    help('editService')      EditServiceMBean MBean.

    help('encrypt')          Encrypt the specified string.

    help('exit')             Exit WLST from the user session.

    help('exitonerror')      Variable indicating whether WLST exits on error.

    help('exportDiagnosticData')Execute a query (offline).

    help('exportDiagnosticDataFromServer')Execute a query (online).

    help('find')             Find MBeans and attributes.

    help('get')              Return the value of the specified attribute.

    help('getActivationTask')Return the latest ActivationTaskMBean.

    help('getAvailableCapturedImages')Returns the list of Diagnostic Images available for downloading.

    help('getConfigManager') Return the ConfigurationManagerMBean.

    help('getMBI')           Return the MBeanInfo.

    help('getMBean')         Return the MBean by browsing to path.

    help('getPath')          Return the MBean path.

    help('getWLDM')          Return the WebLogic DeploymentManager object.

    help('getopenstores')    Returns a list of opened stores (for script access).

    help('getstoreconns')    Returns a list of connections in the specified store (for script access).

    help('home')             Local MBeanHome.

    help('invoke')           Invoke a management operation on the current bean.

    help('isAdminServer')    Variable indicating whether WLST is connected to an Admin Server.

    help('isRestartRequired')Determine whether a server restart is required.

    help('jndi')             Navigate to the JNDI tree.

    help('listApplications') Lists all applications that are currently deployed to the domain.

    help('listChildTypes')   List all the child MBeans for the cmo.

    help('liststore')        Lists store names, open stores, or connections in a store.

    help('loadApplication')  Load an application and deployment plan.

    help('loadDB')           Load SQL files into a database.

    help('loadProperties')   Load property values from a file.

    help('lookup')           Look up the specified MBean.

    help('ls')               List all the child beans and/or attributes.

    help('man')              Display help from MBeanInfo.

    help('mbs')              MBeanServerConnection object.

    help('migrate')          Migrate JTA, JMS or Server services.

    help('nm')               Determine whether WLST is connected to Node Manager.

    help('nmConnect')        Connect WLST to Node Manager.

    help('nmDisconnect')     Disconnect WLST from a Node Manager session.

    help('nmEnroll')         Enroll the machine with Node Manager.

    help('nmGenBootStartupProps')Generate boot and startup properties

    help('nmKill')           Kill the specified server instance.

    help('nmLog')            Return the Node Manager log.

    help('nmServerLog')      Return the server output log.

    help('nmServerStatus')   Return the status of the server.

    help('nmStart')          Start a server using Node Manager.

    help('nmVersion')        Return the Node Manager server version.

    help('openfilestore')    Opens a file store.

    help('openjdbcstore')    Opens a JDBC store.

    help('prompt')           Toggle prompt information.

    help('pwd')              Display the current working directory.

    help('readDomain')       Open an existing domain for updating.

    help('readTemplate')     Open an existing domain template.

    help('recording')        Variable indicating whether WLST is recording.

    help('redeploy')         Reload classes and redeploys an application.

    help('redirect')         Redirect WLST output to the specified filename.

    help('removeListener')   Remove a listener that was previously defined.

    help('resume')           Resume a server instance.

    help('runtime')          (Deprecated) Navigates to the last MBean in runtime hierarchy

    help('runtimeService')   RuntimeServiceMBean MBean.

    help('save')             Save the edits that have been made.

    help('saveDiagnosticImageCaptureEntryFile')Downloads an entry from a captured Diagnostic Image.

    help('saveDiagnosticImageCaptureFile')Downloads a captured Diagnostic Image.

    help('serverConfig')     Navigate to the last configuration MBean or root.

    help('serverName')       Name of the server to which WLST is connected.

    help('serverRuntime')    Navigate to the last runtime MBean or root.

    help('set')              Set the specified attribute value.

    help('setDistDestType')  Sets the distributed destination type.

    help('setOption')        Set options related to a domain creation or update.

    help('showChanges')      Show the current changes that were made.

    help('showListeners')    Show all listeners that are currently defined.

    help('shutdown')         Gracefully shut down a server or a cluster.

    help('start')            Start a Managed Server or a cluster.

    help('startApplication') Start an application.

    help('startEdit')        Start a configuration edit session.

    help('startNodeManager') Start Node Manager.

    help('startRecording')   Record all user interactions with WLST.

    help('startServer')      Start the Administration Server.

    help('state')            Return a map of servers/clusters and their state.

    help('stopApplication')  Stop an application, making it unavailable to users.

    help('stopEdit')         Stop current edit session and release lock.

    help('stopNodeManager')  Stop Node Manager.

    help('stopRecording')    Stop recording WLST commands.

    help('stopRedirect')     Stop redirection of WLST output.

    help('storeUserConfig')  Create a user configuration and key file.

    help('suspend')          Suspend a running server.

    help('threadDump')       Display a thread dump for the specified server.

    help('typeService')      TypeServiceMBean MBean.

    help('unassign')         Unassign applications or resources.

    help('unassignAll')      (Deprecated) Unassign all applications or services.

    help('undeploy')         Undeploy an application.

    help('undo')             Revert all unsaved or unactivated edits.

    help('updateApplication')Update an application configuration using a new deployment plan.

    help('updateDomain')     Update and save the current domain.

    help('username')         Name of user currently connected to WLST.

    help('validate')         Validate the changes that have been made.

    help('validateConfig')   Validate a Domain configuration.

    help('version')          Current version of WLS.

    help('viewMBean')        Display information about an MBean.

    help('writeDomain')      Write the domain configuration information.

    help('writeIniFile')     Convert WLST definitions to Python file.

    help('writeTemplate')    Write the domain template.

Look we have stopEdit :) I

 help('stopEdit')         Stop current edit session and release lock.

So, lets put a domain lock on the admin console and leave it there..

Why do I give stopEdit() as an example? Because I want to address the problem in automated EBS 12.2 startups :)

Such as this one :

http://ermanarslan.blogspot.com.tr/2015/01/ebs-122-automatic-start-stop-scripts.html

START SCRIPT:

### ERMAN ARSLAN EBS 12.2 APPS START automated
### 12/30/2013
## supply appspassword and weblogicpassword in sequence as command line arguments
### reviewed and tested 1/15/2015
if [ `whoami` == "root" ]
then
echo you can not run this script with root!
exiting
else
. /u01/apps/EBSapps.env run
{ echo apps; echo $1; echo $2; } | sh $ADMIN_SCRIPTS_HOME/adstrtal.sh -nopromptmsg
fi

The problem is if we have domain lock in the Weblogic(which comes with the EBS 12.2) then our managed server start scripts hang... 

What we normally do to fix the problem is actually -> connecting to admin server and releasing the lock..

But now, we will release that lock using WLST :)

What we need to do is using edit() to reach the edit tree, and then using stopEdit to end the edit session..

wls:/EBS_domain_ORATEST/serverConfig> edit()

Sure you would like to stop your edit session? (y/n)

y

Edit session has been stopped successfully.

That 's all. Our edit lock is released.

When we check it using Admin console, we will see that there is no locks anymore..

As you see in the picture above, it is clean :)

Morover;if we can use this wlst command in our EBS start scripts , we can fix the problem even in there..

Okay lets use WLST in scriptized mode and release the lock by using the following script ..

We create a script named erman.py with the following contents;

connect('weblogic','welcome1','t3://ermanhost:7001')

edit()

stopEdit(defaultAnswer='y')

exit()

Then use this script for releasing the lock;

java weblogic.WLST erman.py

Initializing WebLogic Scripting Tool (WLST) ...

Welcome to WebLogic Server Administration Scripting Shell

Type help() for help on available commands

Connecting to t3://ermanhost:7001 with userid weblogic ...

Successfully connected to Admin Server 'AdminServer' that belongs to domain 'EBS_domain_ORATEST'.

Warning: An insecure protocol was used to connect to the 

server. To ensure on-the-wire security, the SSL port or 

Admin port should be used instead.

Location changed to edit tree. This is a writable tree with 

DomainMBean as the root. To make changes you will need to start 

an edit session via startEdit(). 

For more help, use help(edit)

You already have an edit session in progress and hence WLST will 

continue with your edit session.

Edit session has been stopped successfully.

Exiting WebLogic Scripting Tool.

Script works well.

Then we put this script into our automatic start script and we are fixing the "hang" problem..

EBS 12.2 START SCRIPT:

### ERMAN ARSLAN EBS 12.2 APPS START automated
### 12/30/2013
## supply appspassword and weblogicpassword in sequence as command line arguments
### reviewed and tested 1/15/2015

### 02/20/2015 modified to relase the domain lock before starting the application services
if [ `whoami` == "root" ]
then
echo you can not run this script with root!
exiting
else

. /u01/apps/EBSapps.env run

. $EBS_DOMAIN_HOME/bin/setDomainEnv.sh

java weblogic.WLST erman.py

. /u01/apps/EBSapps.env run

{ echo apps; echo $1; echo $2; } | sh $ADMIN_SCRIPTS_HOME/adstrtal.sh -nopromptmsg
fi

Note that: If we dont have domain lock present at the time that we run this script; we may get the following exception:

WLSTException: Error occured while performing stopEdit : Cannot call stopEdit without an edit session in progress

But this is not a problem, it does not break our script . That 'is ; even if there are exception in WLST, the shell continues to execute the remaining lines of our shell script and able start our EBS services without any problems..

Here is the test ;

Contents of erman.sh: 

java weblogic.WLST erman.py

echo "I could execute commands here :)"

Lets run erman.sh and see if we could execute the commands after getting the exception caused by the absence of the edit lock.

sh erman.sh 

Initializing WebLogic Scripting Tool (WLST) ...

Welcome to WebLogic Server Administration Scripting Shell

Type help() for help on available commands

Problem invoking WLST - java.io.FileNotFoundException: /u01/install/APPS/fs1/FMW_Home/user_projects/domains/EBS_domain_ORATEST/erman.py (No such file or directory)

********I could execute commands here :)   --> So no problems :)

Okay.. In this post, I have tried to give some general information about WLST. I have given some examples for using the commands in this tool and also tried to consolidate our learnings by making a use case in EBS 12.2..

Hope you 'll find it useful.

RDBMS -- 11gR2 & Windows Single Node to Linux RAC(2 nodes) migration

Recently, migrated an Oracle Database 11gR2 from Single Node Windows 64 bit to Oracle Linux 64 2 node RAC system. Source database was on a NTFS filesystem. For the target database, as you may guess, we have used Oracle ASM .

Source and target database versions were 11.2.0.3 64 bit, so upgrading the database was not the point.
On the other hand,  the migration was already tricky, but the method that I have chosen for it, made the operation to be a clean work.
Besides, using rconfig reduces the effort that has to be taken in the post configuration phases.
There were other alternatives; but applied the following action plan;

• Use rman convert the source db to Linux 64 bit.
• Copy the converted database(like an image copy) to the RAC node 1.
• Recreate the controlfile in the target and invalidate then validate all the plsql objects .(using transport.sql) --this operation is for creating a database from the converted datafiles in the local filesystem of rac node 1 . it is an in-place operation.
• Use rconfig to physically move datafile from cooked filesystem(ext3) to ASM. 
• Make the post configuration and deliver.

Okay. Lets take a look at the steps;

We first take our source database into read only mode,

Then use the plsql in the following picture to ensure that the source database is convertable,

Next, we use rman to convert the source database to "Linux x86 64-bit" .. This operation creates the Linux type datafiles in the directory we speficied in the command.

This operation also creates a sql script (TRANSPORT.sql) for us.. This script will be used in the next steps while we ll be configuring the target instance..

At this point; our work with the source is done, so we can take it back to read write mode.. 

Note that : This was a test run, that's why we took the source database into readwrite mode just after finishing our rman convert operation.. This must not be done for the production instance.. Source Production should be closed at this point..

Next, we copy the converted datafiles to the target system .. Copy them to a local mount point in 1st node

We modify the init.ora in the target according to the target node configuration.

Then, we modify the transport.sql for the target environment. We may change the db sid if we want, also we delete the lines for adding the tempfiles. We leave only one add tempfile line, because of the bug

"Bug 13912880 - Rconfig fails with SQLFatalErrorException ORA-25152 "TEMPFILE cannot be dropped"

Next , we set our target database environment and execute the transport.sql

Once the transport.sql is completed successfully, we have our database up and running in the target Linux Rac node 1..

During the run of the transport.sql , we monitor the invalid objects to estimate the finish time of the operation.

We also check the database for compatible parameter comparing with the ASM diskgroup compatabilities accordingly.

Next we configure rman parallelism for the upcoming rconfig operation.. Here we are using 12 parallel processes.. This parallelism should be equal to the disk heads of the associated storage devices.
.

Next, we configure our rconfig configuration xm file, and run rconfig..
Note that we can also run the rconfig by editing the xml file .. The line "Convert Verify = ONLY " makes rconfig to check the possible errors, and reports them.. If we see any problems after running rconfig with "ONLY" argument, we fix the errors and then reexecute the rconfig..
Once all the errors dissapear, we make Convert Verify = YES and run rconfig again.. Rconfig makes the real conversion when "convert verify=Yes"  is used in xml file.

For example: we got the error in verify phase:
LOG ARCHIVE DEST does not exists on all nodes in the cluster.
Thus, we created the missing directories in node 2 before making the real conversion..

 Okay, here we run the rconfig...

Note that : If we dont have init.ora in default location ($ORACLE_HOME/dbs), we can get ORA-01078 error.
It is because rconfig wants to read the default parameter file during its run.

Altough, convert only completes with success we can still encounter errors during the real conversion..
In these situations, we check the log files under cfgtoollogs/rconfig/SID/sqlLog directory

Moreover; we may got the the tempfile error because of the bug 13912880... Workaround is to have only one tempfile before running rconfig.

When rconfig finishes its run, it sets its Result code to 0 , and reports "Operation Succeeded"

Lastly, we make the post configuration( checking FRA sizes, checking local and scan listener configuration etc, setting optimal process and memory related parameters) in our RAC database and we are done !
We dont have to srvctl add commands and anything else.. All comes configured.

RDBMS/RAC -- Bug 13912880 - Rconfig fails with SQLFatalErrorException ORA-25152 "TEMPFILE cannot be dropped"

I have faced the problem recently . The fix comes with Rdbms 11.2.0.4 , also we have a generic 13912880 patch for 11.2.0.3..
I completely recommend the patch and the needed upgrade.. That 's for sure , but I still have some words to say about this :)
Good news is that; if you have only one temp file in your source db; rconfig does not create any problems :)
"Decreasing the tempfile count of the source db to 1 before running rconfig" seems to be a workaround.
I have tried this 2 times (during converting datafiles from local to ASM using rconfig in 11.2.0.3), and it worked.

OS -- create file with initial size, fseek, Rman's convert behaviour, Perform Volume Maintenance Tasks in Windows

This post will be about initialized of the files in Linux and Windows operating system..
These kind of theorical topics have been beyond of my scope for a while.
Anyways, I have decided write about creating a file with initial size , when I saw Oracle RMAN 's behaviour yesterday..
It was a Convert Database operation, and I have seen that just after the start of the operation; the files were initialized with definit sizes in the target directory..
Lets say , if we are converting a database with system01.dbf 10gb in size , we ll see a new system01.dbf in the target directory with 10gb size just after the start of our operation..
So rman initializes its target files, and then write to them.
Is it so? Or is it Os that initializes the file immediately? Is there a way to create a file with a initial size?

The following C code clarified my questions;

#include <fcntl.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <io.h>
#include <stdio.h>
int main()
{

FILE* file = fopen ("erman", "w+");
fseek (file, 199, SEEK_SET);
fprintf (file, "x");
fclose (file);
return 0;

}

As you see  above, it uses fseek to set the initial size of the created file..

If you compile and run the code; you will see an empty file will be created with 200 bytes in size..

On the other hand; if you open the created file with a proper editor (like vi , or notepad ++) , you will see that the file is not actually empty.

You will see that there are special characters written in the file.. This is because the design.. OS makes those writes, actually formats the blocks in the disks for protecting the consistency in the underlying filesystem..

Also, we have fsutil in windows; which can create empty file with a initial size..

for example; following command can create a file named erman.out with 10000bytes in size.

fsutil file createnew erman.out 10000

Again, when you open this file with a tool like notepad++, you will see that the file is not actually empty..

Okay, lets come back to our topic.

Rman,when converting the database; must have been using some code like the above to set its destination file size.

It creates the file with a definit size ; allocates os Disk space and then formats the file with its own format while making its operations.

Why rman does this? 

The reason can be to guarantee the allocated disk space or it can be performance reasons (allocate once, write many).. Maybe both..

So, why does OS writes(someting like formatting) to a newly created but initally sized empty files?

The answer seems the security.

Imagine, you create an empty file with 1 gb in size and OS gives that file to you without formatting it :) 

You can easily read your allocated blocks to see their current contents.. It is by design .. I mean OS does not actually delete the files when you delete them.. So, you can read the old contents of the blocks that is allocated for your files..

Lets look from the Linux perspective to support this;

When you rm a file in Linux, you delete the filesystem information (you remove the link for inode) ..

Other process might sill have the file open.. 

Even if after all the processes will close the deleted file , you can still recover the data.. This shows that deleted file contents are not actually deleted..

Lets look from the Windows and Sql Server's perspective;

Look at kimberly's blog post;

http://www.sqlskills.com/blogs/kimberly/instant-initialization-what-why-and-how/

She talks about the parameter for Sql Server .. The parameter in questiong is Perform Volume Maintenance Tasks.. This parameter works for Sql Server and eliminates the need for initializiation of a newly created file..

it allows file allocation requests to skip zero initialization on creation. As a result, file allocation requests can occur instantly – no matter what the file size.

So , why is it not on by default then?

Kimberly : 

By granting “Perform Volume Maintenance Tasks” to a SQL Server instance, you are giving administrators of the instance the ability to read the encrypted contents of a recently deleted file (ONLY IF the file system decides to use this newly freed space on the creation of a new database – created with instant initialization) with the undocumented DBCC PAGE command.

That 's all for this topic. I hope you will find it useful.

EBS 11i - Oracle RAC 11gR2 -- Using Scan Listeners with EBS 11i

Using scan Listeners in EBS 11i is not supported. We have done a lot of implementations and always have kept that in mind..

In EBS 11i Exadata implementation,  we were usualy doing a Load Balanced and Fail over capable configuration based Vips.

Even today, when I m writing this blog post; I saw that using scan listener in EBS 11i is still not certified by Oracle.
On the other hand; using scan listeners in EBS 11i..It is possible by making a custom configuration which will be supported by Oracle ACS or Oracle Consulting.

I m happy to announce this in my blog.. Actually, I m sharing an information which is present in Oracle Support.. Exactly in Appendix D of the Document: Using Oracle 11g Release 2 Real Application Clusters with Oracle E-Business Suite Release 11i (Doc ID 823586.1)


This is pleasing as we dont have to say the words; "EBS 11i does not support Scan listener" anymore :)

Okay.. Before starting ; lets remember what the scan listener concept is ;

Scan listeners are the first point of contact in a 11gR2 RAC environment. Clients are connect to the scan listeners in the first place. There are multiple scan listeners listening multiple scan ip address in the cluster. All the database services are registered with the Scan listeners.
Client send its connection request to a scan listener.
Note that: Client only knows the scan name. DNS redirects the connection request to one of the scan listeners in the cluster. Once the scan listener takes the client's connection request, it sends back one of the local listener's (least crowded node) address to the client.. The local listener information belongs to the local listener which services for the service name that the client requested.. Then the client and local listener communicate between eachother and the connection becomes established.

For the deails, please read following blogpost of mine,
http://ermanarslan.blogspot.com.tr/2014/02/rac-listener-configuration-in-oracle.html

To understand better, you can take a look to my previous post regarding to the scan listeners http://ermanarslan.blogspot.com.tr/2013/05/database-about-scan-11gr2.html

Using Scan listener in EBS 11i is supported by Oracle ACS and Oracle Consulting; this is only the certified combinations are supported by Oracle under the terms of the Support License. This is considered to be a customized configuration which could be supported by ACS or Oracle Consulting.
A customized configuration which also is not supported by Autoconfig..

The action plan for using Scan Listeners in EBS 11i is as follows;

In Database Tier;

1) set local listener parameter(alter system set local_listener=<sid>_LOCAL scope=both sid='<Instance>')
2) set remote listener parameter (alter system set remote_listener='<SCAN NAME>:<port>' scope=both sid='<Instance>';)
3) Run autoconfig on db nodes

4) Create and ifile in TNS_ADMIN  , for example : <sid>_<node>_ifile.ora .. 

5) cd $TNS_ADMIN

cp tnsnames.ora <sid>_<node>_ifile.ora

open newly created <sid>_<node>_ifile.ora file and remove all  <SID>_LOCAL references .

Also, replace the VIP related Tns entries with the SCAN entries in the ifile..

For ex: 

<sid>=
(DESCRIPTION=
(ADDRESS=(PROTOCOL=tcp)(HOST=<VIP Host>.<DOMAIN>)(PORT=1531))
(CONNECT_DATA=
(SERVICE_NAME=<service name>)
(INSTANCE_NAME=<sid>)
)
)
---> Change it to
SCAN Descriptor
<sid>=
(DESCRIPTION=
(ADDRESS=(PROTOCOL=tcp)(HOST=<SCAN Host>.<DOMAIN>)(PORT=1521))
(CONNECT_DATA=
(SERVICE_NAME=<service name>)
(INSTANCE_NAME=<sid>)
)
)

Note that : All the tns entries excep the local ones should have Scan-host configuration after this action.

In Application Tier;

Supposing your system has been configured properly ( I mean tnsnames.ora and listener.ora files located in $ORACLE_HOME/network/admin and $IAS_ORACLE_HOME/network/admin Also TNS aliases have been generated for load balancing and failover and that all the aliases are defined using the virtual hostname)

Perform these actions in all application tier nodes;

1) Edit context file and change s_dbport and s_dbhost to Scan port and Scan Host

2) Edit context file and change s_jdbc_connect_descriptor_generation value to FALSE

3) Create and ifile in TNS_ADMIN  , for example : <sid>_<node>_ifile.ora .. 

cp tnsnames.ora <sid>_<node>_ifile.ora

4)cd $TNS_ADMIN

open  newly created <sid>_<node>_ifile.ora file and remove all  FNDFS references .

Also, replace the VIP related Tns entries with the SCAN entries in the ifile..

5)Run autoconfig. Check DBC and tns aliases .. Ensure that DBC and Tns entries are defined using Scan host and port.

Again, I think that Oracle has the reasons  to not to support this configuration.. An incapability in the client side network code.. That is ; client side network code does not use round robin for resolving multiple Ip address. In other words, even if you may have round robin in DBS, the EBS 11i client side code can not handle this.

The only workaround seems to be creating custom TNS aliases with multiple SCAN IP addresses , as documented in 823586.1.

Such as:

<alias>=
(DESCRIPTION=
(ADDRESS_LIST= (LOAD_BALANCE=on)(FAILOVER=ON)
(ADDRESS=(PROTOCOL=tcp)(HOST=<IP1> )(PORT=1521))
(ADDRESS=(PROTOCOL=tcp)(HOST=<IP2> )(PORT=1521))
(ADDRESS=(PROTOCOL=tcp)(HOST=<IP3> )(PORT=1521))
(CONNECT_DATA= (SERVICE_NAME=<Service_Name> ) )

EXADATA -- EBS 11i EXADATA migration -- From HPUX to Linux Exadata 2 node RAC

This time migrating EBS 11i database to Exadata... Actually this was my third EBS 11i - Exadata migration and fifth EBS - Exadata implementation.

Last year I migrated an EBS 11i to Exadata, but at that time the source system was Solaris ..
3 month ago, there was a POC . The source system used in POC was Linux..
This time, I had to deal with a HPUX RAC source system..

Anyways;
Source database was an 10.2.0.4 64 bit Oracle Database running on HP UX..
Target is an Exadata X4-2 , so OS is Oracle Linux  5.9 64 bit and it hosts a 2 node Oracle 11gR2(11.2.0.4) RAC environment..
Source Application was running on 3 servers Linux 32 bit(shared appltop), target application server was single and also Linux 32 bit.

The method was "DataPump" , so the operation was a logical migration.

Following describes the whole operation in general;

For this project; my document inventory was consisting of ;

Oracle Docs:

• Interoperability Notes Oracle EBS 11i with Oracle Database 11gR2 (11.2.0) (Doc ID 881505.1)
• Migrating an Oracle E-Business Suite Database to Oracle Exadata Database Machine (Doc ID 1133355.1)
• Export/import notes on Applications 11i Database 11g (Doc ID 557738.1)
• Oracle Applications Installation Update Notes, Release 11i (11.5.10.2) (Doc ID 316806.1)
• Migrating_EBS_to_Exadata_MAA_whitepaper
• Configuring DBFS on Oracle Exadata Database Machine (Doc ID 1054431.1)
• Recreating the Spfile for RAC Instances Where the Spfile is Stored in ASM [ID 554120.1]
• Interoperability Notes Oracle EBS 11i with Oracle Database 11gR2 (11.2.0) (Doc ID 881505.1)
• 11i: adcfgclone fails with signal 11 while executing a Java Command (Doc ID 395689.1)
• EBS 11.5.10.2 Install Fails on OEL5 with AC-50207 and java.lang.Exception: Fatal Exception (Doc ID 948628.1)
• 11i: 'adcfgclone.pl appsTier' Fails with Error: 'txkGenExtSecConf.pl' - ORACLE_HOME env. variable must be set to validate sqlplus connection (Doc ID 1186394.1)
• ORA-20000 APPS_DDL/APPS_ARRAY_DDL Package(s) Missing Or Invalid In Schema CTXSYS (Doc ID 944150.1)
• Error When Trying To Start Discoverer (Addisctl.Sh) (Doc ID 854976.1)

Migrating of Application tier was easy, it involved cloning actions .. It was something like standart cloning.. So I will not go in details about application migration here.. It was just: apply the patches for 11gR2 database interoperability ,and clone the application in to the new application server.

So lets start ;

Lets look at what we have done to migrate an EBS 11i database from HPUX 10.2.0.5 to Exadata X2-4 Linux 11.2.0.4

1)

First of all we checked the patches in 11.2.0.4 Exa db
There were two patches(17468141 ve 17501296). On the other hand; these paches were already subset of the other patches that comes with Exadata.. that 's why , we didnt need to apply them.
[oracle@exaerpdbadm01 17468141]$ opatch prereq CheckConflictAgainstOHWithDetail -ph ./
Oracle Interim Patch Installer version 11.2.0.3.6
Copyright (c) 2013, Oracle Corporation. All rights reserved.

PREREQ session
Oracle Home : /u02/app/oracle/product/11.2.0.4/dbhome_2
Central Inventory : /u01/app/oraInventory
from : /u02/app/oracle/product/11.2.0.4/dbhome_2/oraInst.loc
OPatch version : 11.2.0.3.6
OUI version : 11.2.0.4.0
Log file location : /u02/app/oracle/product/11.2.0.4/dbhome_2/cfgtoollogs/opatch/opatch2015-01-27_14-01-23PM_1.log
Invoking prereq "checkconflictagainstohwithdetail"
Prereq "checkConflictAgainstOHWithDetail" failed.

Summary of Conflict Analysis:

There are no patches that can be applied now.
Following patches are not required, as they are subset of the patches in Oracle Home or subset of the patches in the given list :
17468141

Conflicts/Supersets for each patch are:
Patch : 17468141
Bug SubSet of 17943261
Subset bugs are:
17468141
OPatch succeeded.
[oracle@exaerpdbadm01 17468141]$ opatch lsinventory|grep 17943261
Sub-patch 17943261; "DATABASE PATCH FOR EXADATA (JAN2014 - 11.2.0.4.3) : (17943261)"
[oracle@exaerpdbadm01 17501296]$ opatch prereq CheckConflictAgainstOHWithDetail -ph ./
Oracle Interim Patch Installer version 11.2.0.3.6
Copyright (c) 2013, Oracle Corporation. All rights reserved.
PREREQ session
Oracle Home : /u02/app/oracle/product/11.2.0.4/dbhome_2
Central Inventory : /u01/app/oraInventory
from : /u02/app/oracle/product/11.2.0.4/dbhome_2/oraInst.loc
OPatch version : 11.2.0.3.6
OUI version : 11.2.0.4.0
Log file location : /u02/app/oracle/product/11.2.0.4/dbhome_2/cfgtoollogs/opatch/opatch2015-01-27_14-02-18PM_1.log
Invoking prereq "checkconflictagainstohwithdetail"
Prereq "checkConflictAgainstOHWithDetail" failed.
Summary of Conflict Analysis:
There are no patches that can be applied now.
Following patches are not required, as they are subset of the patches in Oracle Home or subset of the patches in the given list :
17501296
Conflicts/Supersets for each patch are:
Patch : 17501296
Bug SubSet of 18825509
Subset bugs are:
17501296
OPatch succeeded.
[oracle@exaerpdbadm01 17501296]$ opatch lsinventory|grep 18825509
Sub-patch 18825509; "DATABASE PATCH FOR EXADATA (JUL2014 - 11.2.0.4.9) : (18825509)"

2)

We have created 2 directories for expdp.
We create 2 directories, because our source database was big, and HPUX could not handle mount points which are more than 1 tb in size. Our export would be more than 1 tb..
We also configured expdp through its parameter file to make it use multiple dirs for storing its dump files.

DUMPFILE=dir1:aexp01%U.dmp, dir2:aexp02%U.dmp.... )

3)

We use 4 parallel processes for expdp. The important thing in here was that we used FILE keyword in parfile, not DIRECTORY.. Also for declaring the logfile location, we used FILE keyword.. (if we wpuld not state the log file location using FILE keyword, expdp would take the defaults and try to write the logs into the DATA_PUMP_DIR directory and get errors as we might not have DATA_PUMP_DIR directory defined.
We did not export the statistics..
We did not use compress option , as it can only compress the metadata (not the data itself) in 10g..

4)

Export was taking log time.. Especially the estimate phase.. We hit a bug. We could not get the patch of it, because our CSI was not containing the Extended Support option.. (the patch was an extended support patch) Anyways, we requested the patch from Oracle , but HPUX version of this patch was not available..

The patch was for 10.2.0.5 : Patch 12780993: DATA PUMP PERFORMANCE FOR ESTIMATE=STATISTICS IN EXPORT IS BAD
Fortuneatly, we have found another patch 14645826 , which was containing the fix for bug 12780993.. 14645826 was a generic patch, so we applied it.

Note that : we used ESTIMATE=STATISTICS rather than  ESTIMATE=BLOCKS.. This made expdp to gets the info for calculating the estimates from the statistics .. This approach have also increased the speed of estimate phase.

5)

We created the database using dbca in the target ..Thus , our target database became RAC directly.. So, using dbca; we did not have to struggle with the init.ora and srvctl commands for configuring our target database.

Note that: later, we have created the redolog , undo and tempfile after creating the database using dbca.

We took the info about these files from the aucrdb.sql directly.

6)

We took our target database into noarcivelog mode before our impdp operation.

We also put our spfile which was configured with the interop parameters (using EBS 11i with 11gR2 database) into use in the target database.

Recreating the Spfile for RAC Instances Where the Spfile is Stored in ASM [ID 554120.1]
Interoperability Notes Oracle EBS 11i with Oracle Database 11gR2 (11.2.0) (Doc ID 881505.1)

7)

We have created a DBFS to store the dumps form import..
Configuring DBFS on Oracle Exadata Database Machine (Doc ID 1054431.1)

8) 

Created big file tablespaces for the tablespaces which consist of large number of datafiles.

A good comparison: Big file vs Small File

9)

Because the database was created using DBCA, instance were registered properly with the cluster repository and there was no effort needed for us to configure them as Rac instances, as well. But there was a disadvantage of using dbca in this operation... The disadvantage was some components which came by default as installed. I m talking about XMLDB, OLAP etc... They all came as installed and because of this aumsc1120.sql encountered errors, and made OLAP to be invalid in the dba_registry.
For the solution, we uninstalled the components ,which should not be installed by default ,such as XMLDB ..

So , dbca have created problems in this operation, but these problems could be fixed . 

That is ; we dropped these components and made the dba_registry pure again. After cleaning the dba_registry, we executed aumsc1120.sql and errors dissapeared.. Ausmsc has installed XMLDB, OLAP etc and made them installed and valid as they should be.

Note that : OLAP Catalog and Oracle Multimediate was still invalid .. We dropped and reinstalled them . The reinstallation made them valid.

10)

We imported the dump file into the target database which was on Exadata. 

Analyzed the import log ilfe; and found the following errors:

Some compilation errors caused by dblinks.

Some compilation errors caused by Looping Chain of Synonyms

A MW creation error caused by missing privileges.

A problem in table data becuase of insufficient undo (actually ORA-01555)

Problems in creation of some Indexes.. (again privilege)

11) 

The problems in dblinks hava fixed by copying the tnsnames.ora from the source system ,as the problematic dblinks were not created as direct-dblinks.. Also we set GLOBAL_NAMES to FALSE.. (we were encountering ORA-02085 while it was set to TRUE)
In addition to that; increased open links parameter and bounced the databse (just in case)
Note that : no need to recreate dblinks..

12) 

Recreated the problematic MW . It encountered privilege erros in import, because the owner of the MW did not have the necessary privileges defined for the related table.

13)

Recreated the index using Apps schema.. The situation was the same for indexes too.. the owners of them did not have the necessary privileges defined for the related table.

14)

Recompiled the function which encounter looping chain of synonyms error , manually. This action fixed the problem.

15)

sqlplus "/as sysdba" @auque2.sql

16)

sqlplus "/ as sysdba" @adgrants.sql APPS

17) 

sqlplus apps/apps @adctxprv.sql systemsun CTXSYS

18) 

sqlplus "/ as sysdba"
SQL> exec ctxsys.ctx_adm.set_parameter('file_access_role', 'public');

19)

sqlplus apps/apps
exec fnd_conc_clone.setup_clean;
cd $ORACLE_HOME
unzip -o appsutil.zip
Install the $ORACLE_HOME/appsutil/jre directory:
cd $ORACLE_HOME/appsutil
cp -r $ORACLE_HOME/jdk/jre jre
mkdir clone
cp -r $ORACLE_HOME/jdk/jre clone/jre
perl $ORACLE_HOME/appsutil/bin/adbldxml.pl tier=db
After creating xml file; correct the virtualhost definitions and then run autoconfig..
Set the virtual hostname in each nodes context file
Set the value of s_virtual host_name to point to the virtual hostname for the database
host, by editing the database context file:
On dscgigdb03: $ORACLE_HOME/appsutil/VISMIG1_dscgigdb03.xml
On dscgigdb03: $ORACLE_HOME/appsutil/VISMIG2_dscgigdb04.xml
On dscgigdb03: <host oa_var="s_virtual_hostname">dscgig03-vip</host>
On dscgigdb04: <host oa_var="s_virtual_hostname">dscgig04-vip</host>
Set the named listener name in each nodes context file:
<DB_LISTENER oa_var="s_db_listener">LISTENER_VISMIG</DB_LISTENER>
contextfile=/u01/app/oracle/product/11.2.0/ebsmig/appsutil/VISMIG1_dscgigdb03.xml

Run autoconfig on each database node:
$ORACLE_HOME/appsutil/bin/adconfig.sh

20)

Gathered AD stats in restricted mode.

sqlplus "/ as sysdba"
SQL> shutdown normal;
SQL> startup restrict;
SQL> @adstats.sql
SQL> shutdown normal;
SQL> startup;
SQL> exit;

21)

Revise the listener on each database node.
The reason the listener requires revision is:
In 11.2, listeners are configured at the cluster level, and all nodes inherit the port
and environment settings. This means that the TNS_ADMIN directory path must
be the same on all nodes.
The current version of AutoConfig creates a listener.ora file under the
$ORACLE_HOME/network/admin/<context-name> directory.
AutoConfig does support SCAN listeners as of Oracle E-Business Suite 12.1.3.
The SCAN was not used in this example. See MOS ID 823587.1, “Using Oracle
11g Release 2 Real Application Clusters with Oracle E-Business Suite Release 12”
for further details.
Note : Although SCAN listener is supported, if scan_name resolves in DNS to IP1
and IP2 , the client side 10.1.2 network code does not use round robin on the
underlying IPs if the first IP fails. An AutoConfig solution for this is being tracked
through Bug 10427234 and continue to check MOS ID 823587.1. The only
existing workaround is to create custom TNS aliases configured with multiple
SCAN IP addresses.
a. On each database node, add an ifile entry to the Grid Infrastructure listener.ora and
tnsnames.ora network files.
/u01/app/11.2.0/grid/network/admin/listener.ora
/u01/app/11.2.0/grid/network/admin/tnsnames.ora
Point these to the Autoconfig generated files under
$ORACLE_HOME/network/admin/<context-name>/listener.ora
i. Update the Grid Infrastructure listener.ora on each database node to point to
the Autoconfig generated files under
$ORACLE_HOME/network/admin/<context-name>/listener.ora
For example, on database node 1, dscgigdb03, add the following line to the end
of the /u01/app/11.2.0/grid/network/admin/listener.ora file:
IFILE=/u01/app/oracle/product/11.2.0/ebsmig/network/admin/VISMIG1_d
scgigdb03/listener.ora

 Update the Grid Infrastructure tnsnames.ora on each database node to point to
the Autoconfig generated files under
$ORACLE_HOME/network/admin/<context-name>/tnsnames.ora
For example, on database node 1, dscgigdb03, add the following line to the end
of the /u01/app/11.2.0/grid/network/admin/tnsnames.ora file:
IFILE=/u01/app/oracle/product/11.2.0/ebsmig/network/admin/VISMIG1_d
scgigdb03/tnsnames.ora
b. Set the listener and database environment in OCR and verify it works with srvctl:
(add the listener , then set its envrionment as follows)
srvctl add listener -l listener_vismig
srvctl setenv listener -l listener_vismig –T
TNS_ADMIN=/u01/app/11.2.0/grid/network/admin
srvctl setenv database -d VISMIG -T
TNS_ADMIN=/u01/app/11.2.0/grid/network/admin
srvctl stop listener -l listener_vismig
srvctl start listener -l listener_vismig
Also stop/start the database..

22)

Executed autoconfig in db nodes once again..

23)

Followed doc: 395689.1 to not to encounter classes111.zip error before postclone..

24)

We set PATH and CLASSPATH before running adcfgclone.pl clone in the target apps tier

25)

Followed Doc id: 948628.1 to not to get libdb.so.3 errors while running adcfgclone

26)

Executed adcfgclone in the target apps tier..
Note that: it is better to run adpreclone in the source and copy the up-to-date clone directory into the target apps tier, before running adcfgclone.pl. Also it is good to check the doc id: 316806.1 and everyting(rpms, patches etc) is in place before running the post clone in target apps tier.

27)

Encountered erros in autoconfig that executed by the post clone itself..

****************************************************

*******FATAL ERROR*******

PROGRAM : (/u02/eRMAN/ERMANcomn/admin/install/ERMAN_testapps1/txkGenExtSecConf.pl)

TIME : Tue Feb 3 09:59:00 2015

FUNCTION: TXK::SQLPLUS::validateConnectInfo [ Level 3 ]

MESSAGES:

ORACLE_HOME env. variable must be set to validate sqlplus connection

STACK TRACE

TXK::Error::abort('TXK::Error', 'HASH(0x86a39b0)') called at /u02/ERMAN/ERMANappl/au/11.5.0/perl/TXK/Common.pm line 299

TXK::Common::doError('TXK::SQLPLUS=HASH(0x8609200)', 'ORACLE_HOME env. variable must be set to validate sqlplus connec...', undef) called at /u02/ERMAN/ERMANappl/au/11.5.0/perl/TXK/Common.pm line 314

TXK::Common::setError('TXK::SQLPLUS=HASH(0x8609200)', 'ORACLE_HOME env. variable must be set to validate sqlplus connec...') called at /u02/ERMAN/ERMANappl/au/11.5.0/perl/TXK/SQLPLUS.pm line 331

TXK::SQLPLUS::validateConnectInfo('TXK::SQLPLUS=HASH(0x8609200)') called at /u02/ERMAN/ERMANappl/au/11.5.0/perl/TXK/SQLPLUS.pm line 267

TXK::SQLPLUS::setConnectInfo('TXK::SQLPLUS=HASH(0x8609200)', 'HASH(0x8609260)') called at /u02/ERMAN/ERMANcomn/admin/install/ERMAN_testapps1/txkGenExtSecConf.pl line 192

eval {...} called at /u02/ERMAN/ERMANcomn/admin/install/ERMAN_testapps1/txkGenExtSecConf.pl line 68

ERRORCODE = 1 ERRORCODE_END

.end std out.

The solution was setting the APPSORA.env and running autoconfig once again.
Solution 11i: 'adcfgclone.pl appsTier' Fails with Error: 'txkGenExtSecConf.pl' - ORACLE_HOME env. variable must be set to validate sqlplus connection (Doc ID 1186394.1)

28)

-- Do this, else errors will be generated while recreating grants and synonyms using adadmin..
The fix :
Exected the CTXSYS script mentioned in the note 944150.1

cd $AD_TOP/patch/115/sql sqlplus APPS/<Password> @adctxpkg.sql <SYSTEM Password> CTXSYS APPS

29)

Recreated the grants and synonyms using adadmin.

30)

Using perl $AU_TOP/patch/115/driver/dpost_imp.pl [driver file] , create a driver_file (For ex: erman.drv)
And applied this driver file using adpatch..

31)

Configured twotask context file parameters and make them use load balanced tns entries.

Especially, made s_cp_twotask to ERMAN_806_BALANCE (for EBS 11i , it should be ERMAN_806_BALANCE,, for R12 ERMAN_BALANCE, according to the note: Concurrent Processing - How To Ensure Load Balancing Of Concurrent Manager Processes In PCP-RAC Configurations (Doc ID 762024.1)) and executed autoconfig.

jdbc_url was already configured to be able to do load balance and failover.


Section 3.11 Establish Applications Environment for RAC :
Configuring Oracle Applications Release 11i with Oracle10g Release 2 Real Application Clusters and Automatic Storage Management (Doc ID 362135.1)

Also check:
After a 11.5.10.2 Linux Platform Migration AutoConfig Fails With Error "java.lang.Exception: Error while generating listener.ora" at NetServiceHandler.generate806Listener(NetServiceHandler.java) (Doc ID 2158857.1)

( Set <TWO_TASK oa_var="s_tools_twotask" osd="unix"><database_name>_806_BALANCE</TWO_TASK>
  Set <TWO_TASK oa_var="s_weboh_twotask" osd="unix"><database_name>_BALANCE</TWO_TASK>
)

Concurrent Processing - How To Ensure Load Balancing Of Concurrent Manager Processes In PCP-RAC Configurations (Doc ID 762024.1)

By making concurrent processing tns load balances; obtained the ability to spread the database works of  concurrent managers into multiple rac nodes.

Explanation: Apps listener start the Concurrent Managers(FNDSM) ..

Apps listener starts the concurrent managers using a script. This script uses s_cp_twotask to derive the tns to be used by the concurrent managers. 

Name of this script is gsmstart.sh..

So when we change s_cp_twotask in contextfile and run autoconfig, gsmstart script become recreated using the new info (load balanced tns) stored in s_cp_twotask ..

So, the script uses this tns entry and look the corresponding tns string in the tnsnames .ora.. It find the load balanced tns string and starts the FNDSM using that load balanced tns string..

This configuration is called Concurrent Manager. This configuration is done using the Load Balanced database connections for the concurrent manager connnections.

Note that :Using load balanced concurrent processing can be a disadvantage, becasuse using this method, there is no Load balancing based on modules.. The load balancing in this method is in the database level, so it is random in a manner..

That 's why, a data which can be needed by a concurrent program which is in XXERMAN Module , can be in the second node's memory because, the session of another concurrent program which is in XXERMAN Module (same module) can be running in the second.. This will result an increase the interconnect usage.

PCP is naturally a better configuration. In PCP , 1 app node can be associated to 1 db node. Thus, module based load balancing can be achieved. On the other hand; PCP requires multiple app nodes..

32)

Migrated Application Server Linux printer configurations.

To copy printer configurations; transffered /etc/cups from source App to target App and then restarted cups .service cups restart

33)

Compiled Flexfields using adadmin.

34)

addisctl.sh which was run by the adstrtal.sh encountered errors.

8.0.6 $ORACLE_HOME/jre1183o/lib/i686/green_threads/libzip.so: symbol errno, version GLIBC_2.0 not defined in file libc.so.6 with link time reference (libzip.so)
Unable to initialize threads: cannot find class java/lang/Thread
Could not create Java VM

This error was actually expected..Error When Trying To Start Discoverer (Addisctl.Sh) (Doc ID 854976.1)The solution for this was a Discoverer upgrade, but we have ignored this, because there was another Discoverer server already running on another machine in Customer's environment.

35)

Applied patch 7721754 for Oracle Human Resources (HRMS). Note that: this patch colud be applied to the source system, just before the migration , as well.

36)

Disabled maintanence mode and started up application services.

37)

Update fnd_concurrent_queues because their node names were wrong.. (this was actually a cloning problem.. ) Anyways, updated the node_names and target_nodes accordingly..

Especially Standart Manager's and AP Manager's node names were wrong

update fnd_concurrent_queues
set NODE_NAME='TESTAPPS1',
TARGET_NODE='TESTAPPS1'
where CONCURRENT_QUEUE_NAME='APMNGR'

update fnd_concurrent_queues
set NODE_NAME='TESTAPPS1',
TARGET_NODE='TESTAPPS1'
where CONCURRENT_QUEUE_NAME='STANDARD’

38)

For the concurrent program which produce PDF outputs, started a Vnc , and configured the DISPLAY environment variable in the manager start scripts accordingly.

So,

Connected to VNC port 1 , set DISPLAY=testapps1:1.0 and executed xhost + 

Afterwards, modified the scripts in $COMMON_TOP/admin/scripts ; scripts like : adcmctl.sh and gsmstart etc.. we set DISPLAY env variables in these scripts accordingly and restarted the concurrent managers.

39)

Created directories for APPLTMP and APPLPTMP in db site, and include them in the utl_file_dir parameter, then restarted the database and application services.

40)

Created DQM indexes by following these steps:
Log on to Oracle Applications with the "Trading Community Manager" responsibility
Click Control > Request > Run
Select "Single Request" option
Enter "DQM Staging Program" name
Enter the following parameters:
Number of Parallel Staging Workers: 4
Staging Command: CREATE_INDEXES
Continue Previous Execution: NO
Index Creation: SERIAL
Click "Submit"

41)

Configured notification mailer and delivered the system connection information to the Application Test Team.

That 's it. The rest depends on your needs.. You may drop some indexes, archive/compress some data using Exadata specific functionalities, configure IORM and so on..