Sunday, November 25, 2018

Exadata -- Exadata X3 reimaging problem -- biosbootorder

This will be a quick post, because I 'am currently on-site and waiting to start a migration operation :)
Still, I couldn't wait to write about this :)

Recently, we needed to reimage an Exadata X3 system) with a newer image version. ( (X3 can be considered quite old)

We downloaded the Exadata images (18.1.7 and 12.2.1.1.8 versions) and configured our PXE server.

First we tried with 18.1.7...
We booted the compute node using PXE, but the imaging operation failed while validating the biosbootorder.. 

[ERROR][0-0][/opt/oracle.cellos/validations/init.d/biosbootorder- 247][main][247]  
Failed. See logs: /var/log/cellos/validations/biosbootorder

After the failed biosbootorder validation, we got a kernel panic (as a result of reboot).

That is, the boot problems ended up with a kernel panic ->

[ 70.944116] [<ffffffff816a9634>] dump_stack+0x63/0x81 
[ 70.949533] [<ffffffff816a757c>] panic+0xcb/0x21b 
[ 70.954618] [<ffffffff81086560>] do_exit+0xa70/0xa70 
[ 70.959946] [<ffffffff8106c53c>] ? __do_page_fault+0x1cc/0x480 
[ 70.966145] [<ffffffff816b5a6a>] ? page_fault+0xda/0x120 
[ 70.971820] [<ffffffff810865f5>] do_group_exit+0x45/0xb0 
[ 70.977494] [<ffffffff81086674>] SyS_exit_group+0x14/0x20 
[ 70.983284] [<ffffffff816b031a>] system_call_fastpath+0x18/0xd4 
[ 70.989605] Kernel Offset: disabled 
[ 70.993393] Rebooting in 60 seconds.. 

We thought that;
probably, during the boot, there were some drivers missing. They were probably not loaded so, we couldn't boot. ..

Following document was related with a virtualized env, but it had the same error stack and kernel panic.

OCI-C - Instance Fails to Boot Post Patching For L1TF Vulnerability ( Doc ID 2448058.1 )

As I already mentioned, we suspected from the drivers.. However, the real cause was the initramfs.. initramfs could not detect the boot disk using its boot label.

Anyways, we opened an SR and Oracle logged a bug for this.
Bug 28893408 - X3-2 : PXE IMAGING HANGS AND KERNEL PANIC " ?

The solution was as follows;

boot with diag.iso
chroot to /mnt/cell -> chroot /mnt/cell
change boot to DbSys1  (in file named i_am_hd_boot)
Install Grub on /dev/sda - > image_functions_grub2_install /dev/sda /boot force
rebuild initramfs -> dracut --force

I am not going into details of the actions above right now. But, I will :) in my next posts.

No comments :

Post a Comment

If you will ask a question, please don't comment here..

For your questions, please create an issue into my forum.

Forum Link: http://ermanarslan.blogspot.com.tr/p/forum.html

Register and create an issue in the related category.
I will support you from there.