EBS -- "make -f inst_reports.mk install" completes with error : ld:0706-006 Cannot find or open library file -l jvm

Problem:

make -f inst_reports.mk install command completes with error..

Note: This error also reproduces on applying Ebs patch 14029977..

The error stack is long..

The important information is gathered from the stack "ld:0706-006 Cannot find or open library file -l jvm..."

Analysis:

So, error message above gives "jvm cannot be found" message... (-l jvm)
The "-l" option of the ld linker is below;

-l namespec --> in our case it s -l jvm --library=namespec Add the archive or object file specified by namespec to the list of files to link. This option may be used any number of times. "If namespec is of the form :filename, ld will search the library path for a file called filename, otherwise it will search the library path for a file called libnamespec.a." On systems which support shared libraries, ld may also search for files other than libnamespec.a. Specifically, on ELF and SunOS systems, ld will search a directory for a library called libnamespec.so before searching for one called libnamespec.a. (By convention, a ".so" extension indicates a shared library.) Note that this behavior does not apply to :filename, which always specifies a file called filename. The linker will search an archive only once, at the location where it is specified on the command line. If the archive defines a symbol which was undefined in some object which appeared before the archive on the command line, the linker will include the appropriate file(s) from the archive. However, an undefined symbol in an object appearing later on the command line will not cause the linker to search the archive again. See the -( option for a way to force the linker to search archives multiple times. You may list the same archive multiple times on the command line. This type of archive searching is standard for Unix linkers. However, if you are using ld on AIX , note that it is different from the behaviour of the AIX linker.

So from th man pages of ld , two important things can be figured out;

1) ld is searching the library path 

2) ld is searching for a file called libnamespec.a (in our case libjvm.a)

Library path can be found from the relevant make files..
In this case, as wer are buildint reports on EBS R12, makefiles are in $ORACLE_HOME(10.1.2.)/reports/lib32..
the key make file is env_reports.mk This make file declares the variables and included into the ins_reports.mk
In env_reports.mk; there is a variable called JVMSO_PATH.. Which is declared like ; JVMSO_PATH=-L$(JDK_HOME)/jre/bin/classsic... and JDK_HOME=$JAVA_HOME and JAVA_HOME=$ORACLE_HOME/jdk

So according to these variable declerations; if ld cannot find libjvm.a (i there is no file called libjvm.a) in $ORACLE_HOME/jdk/jre/bin/classic, the error " ld:0706-006 Cannot find or open library file -l jvm..." will be produced..

Fix: 

*Change JVMSO_PATH (for jdk 1.6 --> JVMSO_PATH= L$(JDK_HOME)/jre/lib/pcc/classsic)
or
*Copy libjvm.a to directory pointed by the JVMSO_PATH variable..

RAC - Error on changing public Ip address PRIF-33, CRS-02307, PRIF-32, CRS-02306

While deleting the ip configuration with "oifcfg delif -global ..." command on Rac, following errors may occur;

PRIF-33: Failed to set or delete interface because hosts could not be discovered
  CRS-02307: No GPnP services on requested remote hosts.
PRIF-32: Error in checking for profile availability for host ebsdb02
  CRS-02306: GPnP service on host "ebsdb02" not found.

The error messages are actually self explanatory : Gpnp services deamon of a remote node is not answering..

Brief explanation about GPnP;

The longname of the GPnP is Grid Plug and Play..
It s started before CSS.
This daemon plays a central role in the startup process.
If the name resolution is not working, the daemon wont start..
It defines cluster name, networks and storage.

Cause of the problem: 

A gpnp process failure on node ebsdb02. Gpnp is running on node ebsdb02 but does not answer the request coming from other nodes...


Workaround:

Run oifcfg command from the problematic node..  This action will be a workaround , as problematic gpnp will not be a part of the process, because the ip change command will be executed on the problematic node itself... So that node will communicate with the properly running gpnp processes on other nodes.

Fix: 

kill the gpnp daemon on the problematic node.
ps -ef |grep gpnp;
kill -9 {process_id}
This action will make the cluster software to restart the gpnp daemon..

Oracle Sun Servers ( i.e Oracle Database Appliance) changing ILOM ip address

There are two sun servers/nodes in Oracle Database Appliance Machine.

To change the network configuration of the ILOM interfaces;

1)Just type ip address of the  ILOM to your browser and login page of the web admin interface will appear.

2)Login (default user = root , default password= changeme)

3)Select Configuration => Network

4) Change the ip address and save..

No need to reboot for the setting to take effect. Just open a new browser session and using  the new Ip address..

Information about Network Settings:

MAC Address : The ILOM’s MAC address is set at the factory. The MAC address is a hardware address that is unique to each networked device. The ILOM’s MAC address is provided on a label on the ILOM, on the Customer Information Sheet included on the ship kit, and in the BIOS

Setup screen.

Configuration Method :Select one of the following radio buttons to configure the ILOM’s IP

address either dynamically or statically.

• Obtain an IP Address Automatically (Use DHCP) – Enables aDHCP server to configure the ILOM’s IP address dynamically.

• Use the Following IP Address – Enables you to configure theILOM’s IP address with a static IP. The IP Address, Subnet Mask,and Default Gateway fields will become editable when you select this option.

IP Address: Type the ILOM’s IP address. The IP address is a unique name that identifies the system on a TCP/IP network.

Subnet Mask : Type the subnet mask of the network on which the ILOM resides.

Default Gateway : Type the ILOM’s gateway access address

Screenshot of the Network Settings Page

So , this method is based on the ILOM Web Gui..

There are other methods to change ILOM configurations as well. Like CLI..

For details, you can check Integrated Lights Out Manager

(ILOM) Administration Guide  http://docs.oracle.com/cd/E19121-01/sf.x4600/820-0280-12/820-0280-12.pdf

ORACLE EBS R12 changing IP address

1)perl $AD_TOP/bin/adgentns.pl appspass=apps contextfile=$APPL_TOP/admin/<SID>_hostname.xml -removeserver
2)connect with sqlplus apps/apps and run the following
begin
FND_NET_SERVICES.remove_server('<SID>', '<hostname>');
end;
/
commit;
/
3) Run autoconfig on application tier

....


Click here to read the full article

Oracle RAC IP change -- including scan ip changes and operating system actions

In this case, we will change virtual, public and scan ip addresses..  Private ip addresses stays the same, as it is not needed in this scenario.
In this operation first of tall, we will change the ip addresses declared in RAC.
Second, we will make the ip address changes on the rac nodes(servers).
Finally, we will reboot the rac nodes, to make the new ip addresses assigned.
--Note actually, we dont need to reboot the server.. Only restarting the network service will do the job.. 
But, I write this post for a server migration operation.. I suppose the servers are moved from one system Root to other.. So a reboot is required anyways..


Click here to read the full article

UNIX/LINUX HOW TO KILL A ZOMBIE PROCESS

What is a zombie process? ( Z state process)

When a process ends/dies, it becomes a zombie. Process will be in zombie state, till its exit status and data will be received by its parent..

When a child process dies, its parent is informed by the SIGCHLD signal.. So when this happens, Parent process should execute wait() system call to receive its child's return.

Till parent process execute this wait system call, its child waits in zombie state.

In zombie state, there will be no resource consumption actually.. Only process information remain in memory, kernel tables..

Following is an example for a parent that leaves its processes in zombie state for a certain time...

Look; it forks, and then syncronously waits for its child processes..


#include <sys/wait.h>
#include <stdlib.h>
#include <unistd.h>
int main(void)
{
pid_t pids[10];
int i;
for (i = 9; i >= 0; --i) {
pids[i] = fork();
if (pids[i] == 0) {
sleep(i+1);
_exit(0);
}
}

for (i = 9; i >= 0; --i)
waitpid(pids[i], NULL, 0);
return 0;
}

How to kill a zombie process?

kill -9 will not kill a zombie process..

With kill -9 , SIGKILL signal is sent to the desired process. This signal cannot be caught by the process, because it s not even sent to the process, it s handled by the kernel..

For zombie processes, by design, kernel will not kill the process, as actually this process is not living, process is only appearing in ps ..

So wait system call is needed here and the wait system call is commonly invoked in the SIGCHLD handler..


static void sigchld_hdl (int sig)
{
/* Wait for all dead processes.
* We use a non-blocking call to be sure this signal handler will not
* block if a child was cleaned up in another part of the program. */
while (waitpid(-1, NULL, WNOHANG) > 0) {
}
}

So we use/try "kill -s SIGCHLD" .. By sending SIGCHLD signal to the parent process, we make the parent process to issue wait() and clear its zombie processes. .

If parent process can't process or ignores the signal, zombie processes will not be cleared..
In such situations, killing the parent process will make the init process to take over the child ownership

,issue the wait call and clear the zombie processes... As it is known that, Init process periodically issue the wait system call.. 

EBS 11i -- Add Custom Top Script

In EBS, we place our custom developments or customized objects under custom tops.
Following script was written to ease and automatize the dba work that is done under this process.

Click here for the script code..

Usage Example:

 sh add_custom_top.sh xxefa TEST oracle1 XXEFA XXEFA XXEFA_TBS TEMPPROD /u2/oradata/u2/xxefa_01.dbf 1000M

###ADD CUSTOM TOP SCRIPT BY ERMAN ARSLAN ### Created 6/18/2013

### This script should be executed by the application owner

### Before, executing this script, apps environment should be sourced. APPS{SID}_{hostname}.env

### 1. parameter ->  name of the custom top  (lowercase) example: xxefa

### 2. parameter- >  Ebs database SID 

### 3. parameter ->  System user password

### 4. parameter ->  Db user name that is going to be created for the new application 

### 5. parameter ->  Password of the db user  that is going to be created for the new application 

### 6. parameter ->  Tablespace name that will be created for the new  user.

### 7. parameter ->  Existing Temp tablespace that will be used by the new user

### 8. parameter ->  Datafile full path with datafile name . This datafile will be created while creating the new tablespace

### 9. parameter -> datafile size 1000M

OBIEE-- Business Intelligence Application, Component and Architecture Information

General Application Information: 

Oracle Business Intelligence Enterprise Edition (Oracle BI EE) is a comprehensive set of enterprise business intelligence tools and infrastructure, including a scalable and efficient query and analysis server, an ad-hoc query and analysis tool, interactive dashboards, proactive intelligence and alerts, and an enterprise reporting engine. 

The components of Oracle BI EE share a common service-oriented architecture, data access services, analytic and calculation infrastructure, metadata management services, semantic business model, security model and user preferences, and administration tools. Oracle BI EE provides scalability and performance with data-source specific optimized request generation, optimized data access, advanced calculation, intelligent caching services, and clustering. 

 Oracle Business Intelligence Main Components 

Oracle Business Intellıgence Publisher : 

This component provides an enterprise reporting solution for authoring, managing, and delivering all types of highly formatted documents to employees, customers, and suppliers. 

Oracle BI SERVER: 

This component provides the query and data access capabilities at the heart of Oracle Business Intelligence and provides services for accessing and managing the enterprise semantic model (stored in a file with an .RPD extension). 

Oracle BI Presentation Services: 

This component provides the framework and interface for the presentation of business intelligence data to web clients. It maintains an Oracle BI Presentation Catalog service on the file system for the customization of this presentation framework. 

Oracle BI SCHEDULER: 

This component provides extensible scheduling for analyses to be delivered to users at specified times. (Oracle BI Publisher has its own scheduler.) 

Oracle BI JAVAHOST: 

This component provides component services that enable Oracle BI Presentation Services to support various components such as Java tasks for Oracle BI Scheduler, Oracle BI Publisher, and graph generation. It also enables Oracle BI Server query access to Hyperion Financial Management and Oracle OLAP data sources.

Oracle BI JAVAHOST:

This component distributes requests to the BI Server, ensuring requests are evenly load-balanced across all BI Server process instances in the BI domain.

Java Components:

Deployed as JEE applications to service SOAP, HTTP, and other forms of requests.

ADMINISTRATION SERVER:

Deployed as a JEE container that runs in a dedicated Java virtual machine that contains Java components for administering the system. These components include Oracle WebLogic Server Administration Console, Oracle Fusion Middleware Control, and JMX MBeans.

MANAGED SERVER:

Deployed as a JEE container that runs in a dedicated Java virtual machine that provides the runtime environment for the Java-based services and applications within the system. These services and applications include BI Publisher and Oracle Real-Time Decisions.

NODE MANAGER:

Provides process management services for the Administration Server and Managed Server processes.

OPMN:

OPMN is a process management tool that manages the Oracle Business Intelligence system components such as BI server, BI scheduler, BI Presentation services, BI Javahost and Cluster Controller. OPMN supports both local and distributed process management, automatic process recycling, and the communication of process state (up, down, starting, stopping).

OBIEE Single Host Architecture

In single host architecture, single integrated set of manageable components called the Oracle BI domain is installed and configured to work together on a single host.

OBIEE Multiple Hosts Architecture

In multiple host architecture the Java components (WebLogic server domain) and the system components (BI instance) are clustered on each host as part of the single BI domain. The Administration Server exists on both hosts, but is only active on one host.

Added Weblogic and Applications page

http://ermanarslan.blogspot.com/p/weblogic-and-applications.html

In this page, I will share any kind of information about Weblogic and the Applications using Weblogic..

DATABASE - Premature Archivelogs -- Log Buffer, Redolog and Strand Concept

In some environments, in some database, you will see archivelogs are generated with smaller sizes than Redologs. Unless any log switches are forced , archivelogs should be almost  in the same size as their corresponding redolog files. Forcing a log switch can make archive log sizes different than redolog sizes, as it can occur before redolog files are completely filled. Forcing a log switch can be accomplished by manually running alter system log switch command, or by the built-in mechanisms like making oracle use archive_lag_target parameter..

But sometimes this will be not the case. Without any forcing , you will see archivelogs with smaller sizes , and also they will not be even in size.

These seems to be a bug , but actually it is caused by design…

This design is implemented with the following parameters;

·         Log Buffer ->

Strand 1

Strand 2

Strand 3

….

   

Log_buffer_size= Log buffer is formed by the strands.

 It has a default value,  derived by cpu_count actually..

Because log_buffer_size = strand count * strand size…Strand size = 128kb x # of cpus..  Strand count= # of cpus/16 , so log_buffer_size is derived by the cpu_count actuallyJ

Derived log_buffer_size is rounded up the granule size at the end;  

SGA size >= 128GB then granule size is 512MB

64GB <= SGA size < 128GB then granule size is 256MB

32GB <= SGA size < 64GB then granule size is 128MB

16GB <= SGA size < 32GB then granule size is 64MB

8GB <= SGA size < 16GB then granule size is 32MB

1GB <= SGA size < 8GB then granule size is 16MB

SGA size < 1GB then granule size is 4MB

   ·       #of Strands= #of Cpus/16  à number of cpus is actually the value of the cpu_count parameter.

   ·       Strand size= 128Kb * # of Cpus

The concept of strand is developed for performance reasons. The idea is like; as for writing to log buffer, latches should be acquired.A serialization is in question here.. But If we divide the log buffer in to parts(strands) and if we create a latch for each strand.. We will efficiently increase the parallelism in the of writing in to the log buffer..

Initially only one Strand is used/active for redo entries.. But whenever a contention occurs more strands will be used/active. So as contention increases, the number of active strands increases .. Active strands can be increased till it reaches the number of strands initially allocated in the log buffer.

·         Redolog Size = Decided by Dba.

The Log entries in the log buffer is written to redolog files. The design in here, is like aligning the Log buffer with the Redologs.

So , like in the above  figure;  Writes to the redologs are made strand by strand.. The redolog is divided into parts, sized with the strand size, and each strand is written to the relevant part of the redolog. In other words; the strands of log buffers, which are in memory, are mapped to the redolog  file.

If the lof buffer is smaller than redolog file; there will be an empty space in redolog file after the mapping. This empty space is called log residue.. So ,log residue is normal and expected as for most of the systems redolog files are bigger than the log buffers.

On the other this empty  space/log residue is not a garbage.. It s actually used.. It is used when one of the strands is filled..

The process is like the following;

Strand filled -> log residue ?

                                                  -> YES-> USE THAT EMPTY SPACE, NO LOG SWITCH

                                                 -> NO -> LOG SWITCH

So when a strand is filled,  if there is an empty space in redolog file; it is used .. That empty space is mapped by the strand and writes in to the redolog file will be continued.. But if there is no space left, there will be a log switch, and even if other strands are not yet filled, that does not matter.. The log switch will happen!

     

As the above figure explains; if one part(strand mapped) is filled faster than the others, and if their is no log residue, than a log switch will ocur. And as a result, a premature archivelog will be generated..

Based on these information,

Small Archive log files are generated.

This is because, Oracle prepare itself  to the battle through the parameters like, sga_size,cpu_counts etc..

It seems this particular design was developed by the idea of “If you have a lot of system resources, then you are expecting a high load. .”

So as a workaround; relevant parameters (like cpu_count, not recommended.. because there can be other things derived from cpu_count, even cbo’s cost calculations..) can be changed or redolog file sizes can be increased...