Exadata -- Database initialization parameter : cluster_interconnects parameter

It is not recommended to use HAIP infiniband ip addresses as preferred cluster interconnects for the database. It is recommended to use static ib0 and ib1 (not ib0:1, ib1:1)
Making an Oracle database running in Exadata use static infiniband interconnect ip address relies on setting cluster_interconnects parameter.
If not set, Oracle database by default chooses HAIP infiniband addresses for the cluster interconnect and it is not recommended.

This recommendation can also be viewed by analyzing an Exachk report.
that is , if we don't set the cluster_interconnects parameter in the database and leave Oracle database to use the default HAIP interconnects, then Exachk  will report a failure saying "Database parameter CLUSTER_INTERCONNECTS is NOT set to the recommended value"

Also when we click on the Details link in Exachk report; here is what we see;
(pay attention to the sentence written in bold )

Database parameter CLUSTER_INTERCONNECTS should be a colon delimited string of the IP addresses returned from sbin/ifconfig for each cluster_interconnect interface returned by oifcfg. In the case of X2-2 it is expected that there would only be one interface and therefore one IP address. This is used to avoid the Clusterware HAIP address; For an X2-8 , the 4 IP addresses should be colon delimited

So use ifconfig to determine the ip addresses assigned for ib0 and ib1 interfaces (not the ib0:1 or ib1:1) on all the rac nodes, and set these ip address in a colon delimeted strings for all the instances and restart the database;

Ex:

alter system set cluster_interconnects='ib_ipaddress1ofnode1:ib_ipaddress2ofnode1' scope=spfile sid='SIDinst1';

alter system set cluster_interconnects='ib_ipaddress1ofnode2:ib_ipaddress2ofnode2' scope=spfile sid='SIDinst2';