Sunday, May 2, 2021

Exadata X8M-2 & PCA X8-2 -- Part 1 Pre Installation / the process, my experiences and quick notes

Dealing with a new project, a big replatforming.. The target systems are 2 x Exadata X8M-2 (1 for primary site and 1 for disaster recovery site) and a PCA X8-2. This project involves installation, migration (readiness, SOPs , Runbooks, Project Management and execution) and improvements done during or after migration.
Just starting it and couldn't get my hands on yet, at least couldn't get my hands on Erman Arslan mode :), as we are currently in the pre-installation phase. But! I feel already excited, as these are the latest tech. enginereed systems of Oracle and although I did several projects (installation, upgrade, migration and so on) on engineered system like Exadata and ODA, PCA is a new engineered system for me and Exadata X8M-2 is the latest generation for the leading Engineered System of Oracle. 
Besides, we have ROCE (RDMA over Converged Ethernet) and PMEM on this new generation "M" model of Exadata. Note that that "m "in the X8M-2 is there for imlpying the changes in the memory (think about the persistent memory, the advantages it brings).

Remember, 

In 8th generation of Exadata, we have actually 2 main models. One of them is Exadata X8-2 and the other one is X8M-2, which has a brighter future.So if you decide to use X8M-2 on your POC, you'd better do your POC with Oracle Database 19C.
This is because; only 19.3 or greater has full support for the persistent memory commit accelerator. For instance; 18c will not directly use PMEM. It will send a message to the CellSrv, and this will find data in PMEM, so PMEM acts as an extra level of cache. On the other hand; 19c can access PMEM directly via RDMA.
So I think you know what I mean.. This is not a must, but in fact, this is necessary to unleash the performance of Exadata X8M-2.

Well, I want to share my experiences along this project, and we are just starting it with Part 1 - pre installation..

Let's first see the machines : 


You already know Exadata.. It is for Oracle Databases only. 
PCA, on the other hand; can be considered as a new engineered system for most of you, and it is basically the application machine.. We consolidate the applications and the middleware on PCA and get the countless benefit of it. 
We don't position ODA or Exalogic here.. Reason? 
Please see> https://ermanarslan.blogspot.com/2020/10/oda-as-weblogic-appliance-or-is-it.html (ODA as a Weblogic Appliance ? Or is it better to consider PCA?)

Okay, we start with the installation.. My part starts here.. Actually I have several responsibilites in this project, local installation coordination, tech lead for database migration , tech lead for application migration & consolidation and so on.

So the first part is the installation.
We have 2 different machines types and we have 2 different approaches for deploying them.

Let's start with Exadata X8M-2.. Actually the installation process is similar to the earlier generation of Exadata.. So it is a traditional installation method for those who have done Exadata installation.

Basically, we start by making some mettings with the customer and explain them the installation process. Then a coordinator is assigned to the installation process and he/or she makes the coordination globally (partners, ACS , HW team , Customer side and son).. Basically, coordination for HW installation and then software installation and so on :)

As the first thing, we start filling OEDA. (Oracle Exadata Deployment Assistant)
For the new Exadata customers, this is a little tricky.. So we help them.. We actually complete the OEDA process by taking the necessary inputs from the customer..

At this point, I want to mention the latest version of OEDA, because I liked it :) 

For the ones, who are not familiar with Exadata installation process; OEDA is a java tool to be used for gathering the information needed to deploy Exadata.


Once we download it, we unzip and run the installOedaServer.cmd (Note that, I prefer running the tool from a Windows client)

Once the command is executed, OEDA server starts running (Jetty and HTTP Sever), and we reach the OEDA interface by using our browser .. The url is : http://localhost:7002/oeda

It is web based now and it is quite stable.. (free from the most of the bugs)



Ofcourse, we will fill the OEDA with the information we gather from the customer . During the way, we give advices and we guide the customer through the form and finally we get our OEDA ouputs later to be used in the software installation.. 

We do that by leveraging our expertise and by reviewing the customer's order.. (what network card did they order? any memory expansion? how many cores -- considering capacity on demand / COD and so on)

Those OEDA outpus should be validated though. For this, we have a checkip script and we run it on a machine which has all the grants to reach the IP addresses used in the OEDA form.. That machine should reach NTP servers, DNS and etc..  (Note that, we have the option to directly execute the checkip from OEDA as well)

Before running the checkip script, we should be sure that all the requirements are met according to our inputs.. I mean those IP Addresses which are planned to be given to the Exadata nodes should be available, NTP and DNS servers should be reachable, hostname-ip definitions must be there and so on..

One more thing which should be ready before the installation is the network and power cables. These should be ready before the installation and a site visit can be done to check the customer site if necessary.

As we are having hard days because of pandemic, we work remotely.. Even in the case of an Exadata deployment.. Ofcourse for the hardware related part of the installation, we go there to the data center but after that we do the installation/reimaging/or reconfiguration remotely.. 

All we need is a remote connection (maybe a zoom session) and a NFS share which host the installation files to be used in the installation.

--Already curios about the installation? Can not wait for Part 2? Then search my blog with the keywords -> "Exadata installation", I have several articles about it..

Anyways; those installation files are easy to get.
We just make sure that the customer downloaded  "Appendix B: Required Files and Patches" patches and distributions. We also ask the customer to download the screen utility to avoid network connectivity issue which could impact on software installation process.

https://yum.oracle.com/repo/OracleLinux/OL7/latest/x86_64/getPackage/screen-4.1.0-0.26.20120314git3c2946.el7.x86_64.rpm

An example of the list of installation files used in Exadata X8M-2 deployment:

V982063-01.zip

V982068-01.zip

linuxamd64_12102_database_1of2.zip

linuxamd64_12102_database_2of2.zip

p32067171_190000_Linux-x86-64.zip

p32119956_121020_Linux-x86-64.zip

p32131231_121020_Linux-x86-64.zip

p32226239_190000_Linux-x86-64.zip

p32289783_1910000OCWRU_Linux-x86-64.zip

p32301133_1910000DBRU_Linux-x86-64.zip

p6880880_180000_Linux-x86-64.zip

ahf_setup (Autonomous Health Framework setup file)

download Autonomous Health Framework (AHF) setup file from My Oracle Support using patch number 30166242

Well, we make the customer download them and put them in a NFS share.. DVD ISO images/grid infrastructure/RDBMS files....

Some important notes here;

* We can place some inactive third party parts in Exadata Rack.. (such as patch panels) -- We still need to get approval from the Engineered System team.
* We should create at least one database from each Oracle Home that we specified in OEDA.. If we specify Oracle Homes and not specify any databases which should be created from them, this could lead unpredictable OEDA behaviors.
*Capacity on Demand installation requires something which can be used for remote monitoring.. Platinum Support Gateway for instance..  I said someting for remote monitoring.. This is my opinion, but there is a requirement like that..
* We don't need to use COD if we have the licenses required to support all the core that we have on Exadata nodes.. For instance, if we have 48 core database license for the machine, and if we are on a Exadata X82M-2 1/8, then we disable the COD and use the machine full core.. No need to enable COD and choose 24/24 cores per node.
* Core counts that we specify on OEDA are per node.
* Actually, it is better to use the latest Exadata Iso images (other than the one given in OEDA Appendix B). In order to do that;

we go to https://edelivery.oracle.com/

"All Categories" > "Download package": Exadata Storage Server Software 20.1.8.0.0 ( currently it it the latest version) -> then click search
"View items" -> "continue"




Well, after filling and validating the OEDA, having the data center ready, dowloading the installation
files and preparing the requirements for remote software installation, we can plan the hardware (HW) and software (SW) installation.. HW installation is a straight forward process, so I don't get into the details about it here, but SW installation is a little tricky, so I will explain it in the next blog post on this topic -- Part 2 -Installation..

As we are okay with the Exadata side, let's look at the PCA's pre installation phase.
PCA doesn't have a process like Exadata, I mean no OEDA and validation scripts, just a HW installation and provisioning.

After the provising I expect to have a VM Manager weui url and I also plan to register it with an Enterprise Manager and use these tools to create Virtual machines and etc.. 

PCA is quite new for the environment that I'm currently in. I mean for Turkey :) 
So we will see it once we get our hands dirty.. But it is basically a clustered Oracle VM Server environment for us.. We will use the VM Manager (and maybe EM) to configure our VM hosts on this new PCA and then migrate our applications mostly in as-is manner.. 
Details about the provisioning and managing the PCA will be on the next blog posts, especially on Part 2 ..

One thing about the PCA installation is that, the customer should review the following document and ensure the network is ready for the PCA installation -> 

No comments :

Post a Comment

If you will ask a question, please don't comment here..

For your questions, please create an issue into my forum.

Forum Link: http://ermanarslan.blogspot.com.tr/p/forum.html

Register and create an issue in the related category.
I will support you from there.