Exadata -- Cisco Switch Firmware upgrade

In this post, I will explain upgrading the firmware of the Cisco Switch, which is delivered --built-in-- with the Exadata machines.
For explaning the process, I will go through a real life case, which was done in an Exadata X3-2 environment.

The Cisco switch version that I use for demonstrating this upgrade is Catalyst 4948e, which is the ethernet switch delivered with Exadata X3-2 machines. (In Exadata X7, we see Cisco Nexus switches..)

In Exadata environments, these cisco switches are used for systems management net interfaces access only. (ethernet based management network, ssh connection, ILOM and so on.)

So, during such an upgrade, no production traffic is affected, just consoles and node management...

The requirement for upgrading the firmware of these switches may arise after a security scan, which is usually performed regularly by the security teams in customer environments (enterprise customers..) 

Following is a list of vulnerabilities that were discovered in a customer environment.. These vulnerabilities were discovered on the cisco switch which was delivered with the Exadata X3-2. (cisco firmware version was : cat4500e-IPBASEK9-M Version 15.1(1)SG)

• Cisco IOS Cluster Management Protocol Telnet Option Handling 
• Cisco IOS IKEv2 Fragmentation DoS 
• Cisco IOS IKEv1 Fragmentation DoS 
• Cisco IOS Software DHCP Version 6 Server Denial of Service Vulnerability 
• Cisco IOS Software DHCP Denial of Service Vulnerability 
• Cisco IOS EnergyWise DoS 
• Cisco IOS Software Internet Key Exchange Version 2 (IKEv2) Denial of Service 
• Cisco IOS Software Smart Install Denial of Service Vulnerability 
• Cisco IOS Software RSVP DoS 
• Cisco IOS Multicast Routing Multiple DoS 
• Cisco IOS Multiple OpenSSL Vulnerabilities 
• Cisco IOS Software TFTP DoS 
• Cisco IOS Software DHCP Denial of Service Vulnerability 

These vulnerabilites are fixed in cisco firmware version "cat4500e-ipbasek9-mz.152-2.E8"  and here is the list of things that we did for upgrading this 15.2.2E8 target release;

• First, we connect to the cisco switch using telnet from db node 1 and check the current firware version;

[oracle@exanode1~]$ telnet <cisco_switch_ip_address>

exaswc0>show version

Cisco IOS Software, Catalyst 4500 L3 Switch Software (cat4500e-IPBASEK9-M), Version 15.1(1)SG, RELEASE SOFTWARE (fc3)

Technical Support: http://www.cisco.com/techsupport

Copyright (c) 1986-2012 by Cisco Systems, Inc.

Compiled Sun 15-Apr-12 02:55 by prod_rel_team

ROM: 12.2(44r)SG11

fbadmswc0 uptime is 4 years, 37 weeks, 2 days, 23 hours, 55 minutes

System returned to ROM by power-on

System restarted at 15:15:39 GDT Tue Jul 2 2013

System image file is "bootflash:cat4500e-ipbasek9-mz.151-1.SG.bin"

Hobgoblin Revision 21, Fortooine Revision 1.40

• Then, we download the new switch software from cisco -

https://software.cisco.com/download/release.html?mdfid=283027810&softwareid=280805680&release=15.2.2E8&flowid=3592

(Choose "IP Base Image" line from 15.2.2E8(MD) version.

File name : cat4500e-ipbasek9-tar.152-2.E8.tar)

• After downloading the new switch software , we create a tftp server and , and put the new cisco software bin (which comes out from the tar file) to a tftp directory like /tftpboot/switch_image.

[root@acs-vmmachine~]# mkdir /tftpboot/switch_image

[root@acs-vmmachine ~]# chmod 777 /tftpboot/switch_image/

[root@acs-vmmachine ~]# ls -l /tftpboot/switch_image/

total 0

-rwxrwxrwx 1 root root 0 Mar 19 09:16 new_image.bin

• Then again, in cisco switch; we list the files in the bootflash directory and check its size;

exaswc0>enable

Password: 

exaswc0#dir bootflash:

Directory of bootflash:/

    6  -rw-    25213107  Mar 19 2013 14:46:08 +04:00  cat4500e-ipbase-mz.150-2.SG2.bin

    7  -rw-    32288280   Jun 5 2013 20:04:54 +04:00  cat4500e-ipbasek9-mz.151-1.SG.bin

  
exaswc0>show file systems 
File Systems: 

Size(b) Free(b) Type Flags Prefixes 
* 60817408 45204152 flash rw bootflash:   --------> There are about 45 MB free space in bootflash. (Min 20 MB required.)

• We configure our cisco to boot from a specific firmware file. 

exaswc0#configure terminal
Enter configuration commands, one per line. End with CNTL/Z.
exaswc0(config)#no boot system
exaswc0(config)#boot system bootflash:cat4500e-ipbasek9-mz.151-1.SG.bin (current)

• Then, we save the running config and name it with the suffix "before-upgrade"

exaswc0#copy running-config startup-config all 

exaswc0#copy running-config bootflash:cat4500e-ipbasek9-mz.151-1.SG-before-upgrade

• Next, we copy this file to our tftp server. (for backup) -- we answer the prompts for the tftp-server name and the destination filename..

exaswc0#copy bootflash:cisco4948-ip-confg-before-upgrade tftp:

• After copying our running config to our tftp-server (installed earlier into our client machine), we copy the new image from tftp-server to our cisco switch by executing the following command on cisco.

copy tftp: bootflash:
Address or name of remote host []? acs-vmmachine
Source filename []? switch_image/new_image.bin
Destination filename [new_image.bin]?cat4500e-ipbasek9-mz.152-2.E8.bin

...

....

exaswc0# 

exaswc0# dir bootflash: 

Directory of bootflash:/

    6  -rw-    25213107  Mar 19 2013 14:46:08 +04:00  cat4500e-ipbase-mz.150-2.SG2.bin

    7  -rw-    32288280   Jun 5 2013 20:04:54 +04:00  cat4500e-ipbasek9-mz.151-1.SG.bin

25  -rw-    38791882  Mar 20 2018 15:24:24 +04:00  cat4500e-ipbasek9-mz.152-2.E8.bin -- this is the firmware that we are upgrading to.

• We verify the new image file;

exaswc0-ip#verify bootflash:cat4500e-ipbasek9-mz.152-2.E8.bin
File system hash verification successful.

• After our new image file is verified, we configure our cisco switch boot system to our new image bin and save the configuration into nvram.

exaswc0#configure terminal
Enter configuration commands, one per line. End with CNTL/Z.
exaswc0(config)#config-register 0x2102
exaswc0(config)#no boot system
exaswc0(config)#boot system bootflash:cat4500e-ipbasek9-mz.152-2.E8.bin
exaswc0(config)#
exaswc0(config)# (type <control-z> here to end)
exaswc0#show run | include boot
boot-start-marker
boot system bootflash:cat4500e-ipbasek9-mz.152-2.E8.bin
boot-end-marker

exaswc0# copy running-config startup-config all
exaswc0#write memory 

Note that: 0x2102 instructs the boot process to ignore any breaks, sets baudrate to 9600 and boots into ROM if the main boot process fails for some reason.

• Lastly, we boot our cisco switch with the new firmware and save running config.

exaswc0# reload 

exaswc0-#copy running-config startup-config all 

exaswc0#copy running-config bootflash:cat4500e-ipbasek9-mz.152-2.E8-after-upgrade

exaswc0#write memory 

• At this point, we can continue enabling SSH access and disabling telnet access. (although, this action is optional, it is highly recommended. Check the below references for the instructions.

References:

Upgrading firmware / Configuring SSH on Cisco Catalyst 4948 Ethernet Switch (Doc ID 1415044.1)
How To Update Exadata Management Network Switch Firmware (Doc ID 1593004.1)