EBS 12.2 -- Solaris Sparc relinking reports, executing make -f from lib or lib32 directory? + an important info about the ins_reports.mk and the env settings

I'm writing this post, because I found a mismatch in the info delivered by Oracle Support documents.
According to some of those documents; reports should be relinked from lib32 directory( if that directory is present..) here ->

cd $ORACLE_HOME/reports/lib32
--Note: if this directory does not exist: --
cd $ORACLE_HOME/reports/lib
$ make -f ins_reports.mk install

However, even if you relink the reports binaries from lib32 directory, you may end up with the following error in your concurrent manager log files (when running reports based concurrent programs) ->

Program exited with status 1
Concurrent Manager encountered an error while running Oracle*Report for your concurrent request XXXXXXX

So, this error brings you to the following document:

E-Business Suite R12 : Oracle Reports (RDF) Completing In Error "Program Exited With Status 1" Specifically on Oracle Solaris on SPARC OS Installation (Doc ID 2312459.1)

Actually, what that is underlined in this document is the famous environment setting (LD_OPTIONS="-L/lib -L/lib/sparcv9").

However, look at what document 2312459.1 also says, 

Navigate to this directory $ORACLE_HOME/reports/lib and compile the reports executable make command "$ make -f ins_reports.mk install"

So, the document says relink your reports from the lib directory (not from the lib32 directory).

At the end of the day, it is not important to be in lib32 or lib directory for relinking the reports binaries in Solaris Sparc.

The important things are having a good/clean ins_reports.mk and the necessary a LD_OPTIONS environment setting.

What I mean by a clean ins_reports.mk is -> a default ins_reports.mk which is deployed by the EBS installer. No LD_OPTIONS inside that file!

You may recall that, I wrote a blog post for the error that we usually get in EBS 12.2 Solaris installation - > http://ermanarslan.blogspot.com.tr/2017/06/ebs-1220-installation-on-solaris-511.html

In that blog post, I was modifying the ins_reports.mk inside the stage. I wrote LD_OPTIONS inside the ins_reports.mk file and hardly fixed the errors in the runInstaller screens. This was the one and only solution as the runInstaller could not get the LD_OPTIONS env setting from the terminal where I executed it.

However, this was just for the installation.

So after the installation, we need to delete these modifications and use directly the LD_OPTIONS env setting for our future reports relink actions.

cd $ORACLE_HOME/reports/lib

make -f ins_reports.mk install

or

cd $ORACLE_HOME/reports/lib32

make -f ins_reports.mk install

Both works..

This is based on a true story and this is the tip of the day.

ODA X7-2 -- a closer look //Oracle Database Appliance X7-2 Family

The adventure of ODA started with the machine named Oracle Database Appliance in 2011, and almost each year a new model released since then.

ODA X3-2 in 2013, ODA X4-2 in 2014, ODA X5-2 in 2016 and ODA X6-2 in 2016.

Today, I m writing about the newest model, which was released recently.. the "ODA X7-2".

In ODA X6-2, we were introduced with the S (Small), M (Medium) and L (Large) types of ODA family. That is, the standard name of the ODA became ODA HA, and these S, M and L types were added to the familiy.

In ODA X7-2, Oracle decided to remove the Large model from the product family and released the new machine with 3 models: S, M and HA.

Actually, there are several enhancements delivered with the new ODA X7-2, but the most interesting enhancements of all is, "we can built standard edition RAC databases on ODA X7-2 machines". (most interesting new feature, in my opinion)

General enhancements:

• 3 new models ODA X7-2S, ODA X7-2M, ODA X7-2-HA
• Increased storage capacity
• Virtualization option for all the models.
• Standard Edition RAC support
• New Intel Xeon processors
• Increased core count
• Oracle database 12.2.0.1

General Specs:

Oracle Database Appliance X7-2S

Single-instance
SE/SE1/SE2 or EE
Virtualization
10 Cores
12.8 TB Data Storage (Raw)

Oracle Database Appliance X7-2M

Single-instance
SE/SE1/SE2 or EE
Virtualization
36 Cores
Up to 51.2 TB Data Storage (Raw)

Oracle Database Appliance X7-2HA

RAC, RAC One, SI

SE/SE1/SE2 or EE

Virtualization

72 Cores

Up to 128TB SSD or 300 TB HDD Data Storage (Raw)

NVME:

Like we had in ODA X6-2, we have Nvme storage for X7-2S and M models.

Appliance Manager and ACFS:

We have also appliance manager for the proactive management and we have ACFS as the filesystem.

Again, we have odacli as the command line management interface.

ODA X7-2 HA provides us the ability to build a high available environment. Again we are using ACFS for the filesystem, but this time we have SSD disks in the backend. (or SSD+HDD disks)

Supported Versions:

ODA X7-2 supports almost all the up-to-date Oracle Versions.

Enterprise Edition – 11.2.0.4, 12.1.0.2, 12.2.0.1

Standard Edition (SE, SE1, SE2)  – 11.2.0.4, 12.1.0.2

If you need the Database options, then you need to go with EE.

http://docs.oracle.com/database/121/DBLIC/options.htm Advanced Security Option, In-Memory, Mutitenant, …..

About Licensing:

Standard edition 2 RAC support is limited to the 2 socket db servers.

So, we have SE2 RAC support in ODA X7-2 HA (with Oracle Virtual Machine / OVM)
In order to run a SE2 RAC database on ODA X7-2 HA, we use max 18 cores in the ODA_BASE. The remaining cores can be used with the virtual application servers that can be built on top of ODA X7-2 HA. (using OVM)

In SE RAC licensing, on the other hand; we have no socket limit.
So, if we have SE RAC license, then we can have a SE RAC database on the virtualized environment or directly on Bare Metal.
If we want to have both SE RAC and SE2 RAC databases on ODA X7-2 HA, then we need to built a virtualized environment and we need to use max 18 cores in ODA_BASE.

Capacity-on-demand:

ODA X7-2 has the capacity-on-demand feature as well.
We use appliance manager for configuring our core counts.
Enabled core count must be an even number between 2 and 36 (2, 4, 6, 8, 10, 12, 14, 16, 18, 20, 22, 24, 26, 28, 30, 32, 34, 36)

ACFS snapshots:

In ODA X7-2, we have ACFS.. So, this new release of ODA offers ACFS snapshot based quick and space-efficient database copies and ACFS snapshot based rapid cloning of virtual machines.
This is also one of the exciting news..
It was actually there in ODA X6-2, but it wasn't documented clearly.

Here in the following blog post, you can see a demo that I did for ODA X6-2;

http://ermanarslan.blogspot.com.tr/2017/02/oda-x6-2-using-acfs-snapshots-for.html

Oracle VM Server and KVM (Kernel Based Virtual Machine):

The virtualization technology used in ODA X7-2 HA Virtualized environment is still Oracle VM Server.
However, we have KVM (Kernel Based Virtual Machine) for virtualizing ODA X7-2S and ODA X7-2M environments..

We can even use KVM to have a virtualized environment on top of ODA X7-2 HA Bare Metal..

There are some limitations for KVM though;

• You can run only Linux OS on top of them.
• It is not supported to run Oracle database on top of Guest KVM machines.
• There is no capacity-on-demand option for databases and applications that are running on Guest KVM machines. 

You can read my blog posts for getting the details about KVM and enabling the KVM..

http://ermanarslan.blogspot.com.tr/2017/07/oda-kvm-virtualization-for-oda-x6-2sx6.html

http://ermanarslan.blogspot.com.tr/2017/10/oda-x6-2m-virtualization-with-kvm-real.html

Note that, above links are for ODA X6-2M , but the process is almost the same..

Auto starting VMs , Failover for VMs:

In ODA X7-2 HA virtualized deployments, we still get the ODA_BASE based configuration for the Oracle VM Server.

This Oracle VM Server deployed with ODA, give us the opportunity for auto restarting the VMs and for using the standard failover capabilities for those VMs.

We may configure our VMs to be started on the same node or on the surviving node, in case of a failure. (without a manual intervention)

ASR, oakcli orachk and odaadmcli manage diagcollect:

In ODA X7-2, we have the opportunity to use ASR (Automatic Service Request), as well.. ASR monitors and creates automatic service request (Oacle SR) on any hardware component failure.

In addition to hat, we have the "oakcli orachk" command for checking all the hardware and software units of ODA. So, using oakcli, we can validate hardware and software components and quickly identify any anomaly or violation of best practice compliance.

Morever, we can collect the diagnostics by running a single command -> odaadmcli manage diagcollect.

So, at the bottom line, this new ODA X7-2 is not breaking the tradition... ODA is still easy to deploy, patch and manage.  It is still affordable and optimized.

I think, I will work on this new ODA at least in one of the upcoming projects this year.. So we will see :)

EBS 12.2 -- Solaris Apps Tier, "setting the Swap Space" , "/tmp on swap"

According to some administrators, swap should not be used in a production system.. They may say "if we are on swap(using swap at any time), we ' re screwed..

With this in mind, they may not configure a swap at all or they may give you a very small swap area for the use with your applications.

In case of EBS 12.2, this is especially true for the apps tier node. In apps tier, the swap area is generally kept in the minimum size.

On the other hand, I don't find it true.

Let's take Solaris for example..

In EBS 12.2 Solaris installation document, there is no info about swap space.

That is, in EBS 12.2 Solaris installation document, there is only one swap space related requirement and it is lead to the database tier..

Interestingly, in EBS 12.2 Linux installation document, we have a line saying "it is recommended that the swap space on the system be 16 GB or more

Well.. for Solaris actually, this info is more crucial..

In Solaris 10 and 11, there is a important enhancement which is made for the temporary file performance. That is, in Solaris 10 and 11, /tmp is mounted on swap.

Here is an "mount |grep tmp" output, which was get from a Solaris 11.3;

/tmp on swap read/write/setuid/devices/rstchown/xattr/dev=90c0002 on Wed Oct 11 13:42:55 2017

You see /tmp is on swap!

So, when you use /tmp for storing some files, your swap is used in the background.

This means, if you configure a small swap area, you may end up with application errors.

Programs like FNDWRR.exe will definetly crash randomly and even your OS commands like top, like ls will hang during these peaks.

When FNDWRR.exe crashes, you won't be able to see any concurrent output.

So, what I recommend is, to set the swap size to a value which is equal or greater then your /tmp directory size. 

Ofcourse, if you have unnecessarily big /tmp directory and if your EBS application code uses a very little portion of it, then you can arrange your swap size accordingly.

Bytheway, you may change this behaviour on OS level.. On the other hand; as this /tmp on swap thing comes by default, I don't want to make recommendation on Solaris OS layer..

Anyways.. In conclusion, we use swap and we use it very often in Solaris..

Lastly, the important point is ; using swap is not a bad thing for all the cases..  It doesn't always mean that we are screwed:)

Here we see an enhancement to store the temporary files on swap, rather than on disk and this increases the speed of accessing to those files.

ODA X6 -- my review of Oracle Database Appliance(ODA) published on IT Central Station

This is the third review that I did for ITCentralstation.com. This time for ODA X6!

I have given the valuable features of ODA, info about the initial setup, the things that Oracle can do for improvement, my advices to ODA customers and so on.

So, I think you will like it..

My review of Oracle Database Applicance X6 is now published and available for you to read at:

https://www.itcentralstation.com/product_reviews/oracle-database-appliance-review-46946-by-erman-arslan

Bytheway, ODA X7 released! (my followers probably know this, as it was already announced here earlier)

Today, I decided to write a more detailed blog post about it..

Stay tuned, an article about the new ODA Machine (ODA X7) is on its way :)