Friday, April 3, 2015

Exadata-- ZFS adminstration for DBAs

Nowadays, we see ZFS storages in the Exadata environments..
The Sun ZFS Storage Appliance can be connected to the Exadata Database Machine using InfiniBand or 10GigE infrastructure.
We see ZFS in Exadata environment, because there are advantages of using it..
The key benefits of using ZFS in Exadata environments are ;
  • Implementing highly available and highly performing backup and restore solution 
  • Implementing a Cost effective backup environment
  • Eliminating the configuration and operational guess work to achieve the above backup and restore rates  
  • Offloading backups from Exadata Database Machine so more DATA space is available for database growth or for consolidating additional database on to the Exadata system 
  • Fast backup and restore times that can meet most RPO and RTO requirements

Please read the following for detailed info:
Backup and Recovery Performance and Best Practices using the Sun ZFS Storage Appliance with the Oracle Exadata Database Machine
So , after giving the introduction lets jump to our topic.. As you may guess by its title; this blog post focuses on giving an introduction about the ZFS management in Exadata environment..
If you are an experienced Dba having the general knowledge about the SANS, then managing the ZFS storages is not a very big task for you.
When we examine the environment of a new release ZFS storage like ZFS ZS3-2, we see that we have a Webuser─▒ interface called BUI, and a CLI for managing the whole storage..
Though, the storage uses Solaris as Operating System, we cant login  to the Operating system, it seems to be not supported..
We ofcourse we have ILOM to manage the device, as almost all the latest sun devices have one.
We also have management interfaces to manage the components of the appliance. That is we are managing  our ZFS storages using BUI or CLI..

ILOM is a well known interface, I dont need to see a need to go in the details of it..


In general, ILOM has an IP associated with it. ILOM is working on a Web Server..We reach ILOM interface using the ILOM IP address with https and supply our root user and its password to login.. We change check the memory, CPU , fans, cooling, network devices , pci devices.. We also have the opportunity to manage the power of the storage, and open a remote console.



Note that: We can also reach the ILOM using ssh..

ssh root@192.168.0.8
Password: 
Oracle(R) Integrated Lights Out Manager
Version 3.1.2.18 r81429
Copyright (c) 2013, Oracle and/or its affiliates. All rights reserved.
Warning: password is set to factory default.

Okay, lets continue with the BUI..
I can say that BUI is our friend, it is a user friendly user interface and it is not clumsy (like the Oracle Vm console).. We login to the BUI using our Storage root account (its default password is changeme as for the all sun systems) ..

URL:  https://management_ip_address_of_ZFS:215



Once we login , our dashboard is populated using the information related with components and services such as CPU, Network, NFS, Storage Capacity , Services, Disk performance, ISCSI, SMB , FTP etc..


So The ZFS console welcomes us with status information.. (status tab)
As you in the picture above, we have configuration, maintanence, shares and analytics tabs in the console page, and in every tab there are a several subtabs in which we can take bunch of actions for the associated components..
We can manage shares, changing the ip address make the IPMP configuration, see the logs, display the system info,check the storage health and so on..


For the details about the BUI please read the following doc:

Okay, lets talk about CLI.. It is also an alternative way to manage the ZFS Storage.. It is a admin friendly CLI, so as dba 's and OS admins we easily get used to it.
To connect to the ZFS storage using CLI, we use ssh to connect to the ZFS management ip, we supply the root user and its password and we are in...

We can use the help in CLI to display the available commands.. 

help
Subcommands that are valid in this context:
   configuration        => Perform configuration actions
   maintenance          => Perform maintenance actions
   raw                  => Make raw XML-RPC calls
   analytics            => Manage appliance analytics
   status               => View appliance status
   shares               => Manage shares
   help [topic]         => Get context-sensitive help. If [topic] is specified,
                           it must be one of "builtins", "commands", "general",
                           "help", "script" or "properties".
   show                 => Show information pertinent to the current context
   get [prop]           => Get value for property [prop]. ("help properties"
                           for valid properties.) If [prop] is not specified,
                           returns values for all properties.
   set [prop]           => Set property [prop] to [value]. ("help properties"
                           for valid properties.) For properties taking list
                           values, [value] should be a comma-separated list of
                           values.

The logic in the cli is like the logic in directory tree structure.. 
In order to choose a command or subcommand, we just type its name; 
Like the following example:
exayedek:> configuration

Like the way we use the "ls" command to list the contents in a directory in Linux/Unix systems, we use "ls" command to see available subcommands of a command..

exayedek:configuration> ls
Children:
                              net => Configure networking
                         services => Configure services
                          version => Display system version
                            users => Configure administrative users
                            roles => Configure administrative roles
                      preferences => Configure user preferences
                           alerts => Configure alerts
                          cluster => Configure clustering
                          storage => Configure Storage
                              san => Configure storage area networking

exayedek:configuration> net
exayedek:configuration net> ls
Children:
                        datalinks => Manage datalinks
                          devices => Manage physical devices
                       interfaces => Manage IP interfaces
                          routing => Manage routing configuration

exayedek:configuration net> interfaces

exayedek:configuration net interfaces> ls
Interfaces:

INTERFACE   STATE    CLASS LINKS       ADDRS                  LABEL
ipmp1       up       ipmp  ixgbe1      10.10.10.10/24        exayedek
                           ixgbe2                             
ixgbe0      up       ip    ixgbe0      192.168.0.10/22       Mgmt_Interface
ixgbe1      up       ip    ixgbe1      0.0.0.0/8              ExaYedekInterface1
ixgbe2      up       ip    ixgbe2      0.0.0.0/8              ExaYedekInterface2

help
Subcommands that are valid in this context:

   help [topic]         => Get context-sensitive help. If [topic] is specified,
                           it must be one of "builtins", "commands", "general",
                           "help" or "script".

   show                 => Show information pertinent to the current context

   done                 => Finish operating on "interfaces"

   select [interface]   => Select the specified interface to get its
                           properties, set its properties, or run a subcommand

   list                 => List all interfaces

   destroy [interface]  => Destroy the specified interface

   ip                   => Create an IP interface

   ipmp                 => Create an IP multipath interface

Sometimes when we need choose something , we use "select" command to choose..
We need to use select if the object which is our next move is not a command or subcommand..
For example: 
Here, we choose the ipmp1 in the configuration/net/interfaces

exayedek:configuration net interfaces>  select ipmp1
exayedek:configuration net interfaces ipmp1> 
exayedek:configuration net interfaces ipmp1> ls
Properties:
                         state = up
                      curaddrs = 10.10.10.10/24
                         class = ipmp
                         label = exayedek
                        enable = true
                         admin = true
                         links = ixgbe1,ixgbe2
                       v4addrs = 10.10.10.10/24
                        v4dhcp = false
                       v6addrs = 
                        v6dhcp = false
                           key = 1
                      standbys = 

We can use help command anywhere in the CLI to see our options..
For example: in ipmp1, we can show the details about the ipmp1 configuration.. Also, we can set a property and get a property of it.

exayedek:configuration net interfaces ipmp1> help
Subcommands that are valid in this context:

   help [topic]         => Get context-sensitive help. If [topic] is specified,
                           it must be one of "builtins", "commands", "general",
                           "help", "script" or "properties".

   show                 => Show information pertinent to the current context

   commit               => Commit current state, including any changes

   done                 => Finish operating on "ipmp1"

   get [prop]           => Get value for property [prop]. ("help properties"
                           for valid properties.) If [prop] is not specified,
                           returns values for all properties.

   set [prop]           => Set property [prop] to [value]. ("help properties"
                           for valid properties.) For properties taking list
                           values, [value] should be a comma-separated list of
                           values.

CLI can also be connected from ILOM interface..
We may login to ILOM and start /SP/console to connect to the CLI ..

ssh root@192.168.0.8
Password: 

Oracle(R) Integrated Lights Out Manager
Version 3.1.2.18 r81429
Copyright (c) 2013, Oracle and/or its affiliates. All rights reserved.
Warning: password is set to factory default.
-> start /SP/console
Are you sure you want to start /SP/console (y/n)? y
Serial console started.  To stop, type ESC (
exayedek:configuration net interfaces> 

Okay.. Enough for now.. My next blog post will be based on a real life example  of a Production ZFS environment, which we had a problematic NFS share.. We 'll walk through the concepts used in ZFS networking and use BUI to correct the problem..

3 comments :

  1. Awesome. Thanks for sharing.

    ReplyDelete
  2. Nice piece of work. Thanks for this share.

    ReplyDelete