ODA - Why ODA_BASE nodes are faster?

We have seen this in action... ODA_BASE performs much better in I/O, and it become more visible when you migrate a critical IO bound database from ODA_BASE to a Guest VM residing on ODA and compare its performance againts performance benchmark recorded when that database was running in ODA_BASE.
So, ODA_BASE is much faster than any other Virtual Machine which reside on ODA Virtualized environment, but what is the reason for that.

Well, the reason is,altough ODA_BASE is actually a HVM, it has paravirtualized drivers to enhance its operations.. Actually, it is not that :)  Paravirtualized drivers on ODA_BASE is used for eliminating the emulation for the Network devices, but for this devices there is another important thing..
so, ODA_BASE is a HVM with paravirtualized drivers, that 's for sure, but what makes ODA_BASE faster is that, in ODA_BASE domain, Oracle uses Intel Vt-d extentions and PCI passthrough to make ODA BASE virtual server to directly access to the underlying disk drives.

Look at the bios configuration of the hardware nodes, you will see;

-<IO_Virtualization>
<!-- VT-d -->
<!-- Description: Enable/Disable Intel(R) Virtualization Technology for Directed I/O. -->
<!-- Possible Values: "Disabled", "Enabled" -->
<VT-d>Enabled</VT-d>

Look at the dmesg output of ODA_BAE nodes, you will see;

Booting paravirtualized kernel on Xen HVM

Look at ODA BASE vm.cfg file residing in DOM0;

ODABASE VM.cfg:

vncunused = 1

kernel = '/usr/lib/xen/boot/hvmloader'

vnc = 1

name = 'oakDom1'

memory = 49152

timer_mode = 0

device_model = '/usr/lib64/xen/bin/qemu-dm'

builder = 'hvm'

vnclisten = '0.0.0.0'

cpus = '32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47'

on_crash = 'coredump-restart'

on_reboot = 'restart'

vcpus = 16

pci = ['30:00.0@15', '40:00.0@16'] -->PCI scsi controllers.

pae = 1

apic = 1

vif = ['type=netfront,bridge=priv1', 'type=netfront,bridge=net1', 'type=netfront,bridge=net2']

serial = 'pty'

disk = ['file:/OVS/Repositories/odabaseRepo/VirtualMachines/oakDom1/System.img,xvda,w', 'file:/OVS/Repositories/odabaseRepo/VirtualMachines/oakDom1/u01.img,xvdb,w', 'file:/OVS/Repositories/odabaseRepo/VirtualMachines/oakDom1/swap.img,xvdc,w']

acpi = 1

localtime = 1

Look at lspci output of ODA_BASE node, you will see;

30:00.0 Serial Attached SCSI controller: LSI Logic / Symbios Logic SAS2008 PCI-Express Fusion-MPT SAS-2 [Falcon] (rev 03)

40:00.0 Serial Attached SCSI controller: LSI Logic / Symbios Logic SAS2008 PCI-Express Fusion-MPT SAS-2 [Falcon] (rev 03)

Okay, when you look at the OS mount points of ODA_BASE , you will something different;

[root@odabasenode1 ~]# readlink /sys/block/xvda/device/driver

../../bus/xen/drivers/vbd

vbd driver indicates that ODA Base nodes are using paravirtualized drivers for accessing the OS mount points.

Conclusion;

ODA BASE is faster because, it is using Intel Vt-d extension + PCI passthrough for accessing the disk drivers on which it places ASM diskgroups and in turn Oracle Database files.
For OS mount points, ODA_BASE uses the same technology that any other Guest VMs on ODA use, the paravirtualized drivers.
So, this means if you place your database file on a standard mount point, then ODA_BASE is not so fast. This also means that, when you use OS mounts in ODA_BASE, then you will get the same performance that you get at any Guest VMs created on ODA environment. (I mean PVHVM Virtual Machines) , really?
Well, that s actually not true, because Guest VM are deployed on the Oracle Vm repository. So as the Oracle VM repository resides on ASM diskgroups of ODA_BASE and as the hypervisor is on dom0 and as Guest VMs are just user level processes on dom0 and as dom0 reaches these repositories through NFS exports, the Guest VMs residing on ODA are even slower.
Note that: oda base virtual machine files are located  in dom0.(locally)

Waiting for your comments.