Wednesday, May 13, 2015

Linux -- ISCSI, mounting ISCSI disks

          ISCSI is a abbrevation for Internet Small Computer System Interface, and it is a transport layer protocol which works on top of TCP.

ISCSI is a protocol that make us able to use SCSI command over an IP network.. In other words, by using ISCSI ,we use IP switches and Ethernet Cards to use the SCSI commands, to reach devices located in the SAN systems and NAS systems . So it is a cheaper solution than Fibre Channel.

Using ISCSI , we use the shared disks without having to use FCP , FC Host Bus Adapters and SAN switches.. ISCSI does not provide you mount point in the basis of Network filesystems. It actually makes the remote disks , block devices available to you as local disks.. It works via network, but it is not like NAS.
As ISCSI devices are like a local hard disk to you, you can use any filesystem on top of them.. (Ext3,ex2,reiserfs etc..)

In detail , In the standart SCSI protocol, the initiator sends a SCSI command information unit to the target device. ISCSI takes it to a higher level, as ISCSI transports the block level data between the Server and the Storage using TCP packets...

In this context, Server side called as ISCSI initiator and the Storage side called as ISCSI target..

ISCI protocol puts the SCSI commands and data into TCP packets which are sent over the network between the initiator and target..

The ISCSI protocol also put out the commands from the TCP packets when they are reached to their destination.. This way OS sees the storage disks a local SCSI devices.

Things to keep in mind about ISCSI:

CIP,iFCP,FCOE and AoE are aother alternatives to ISCSI .. They all transmit storage data over the IP networks

iSCSI adds extra load to the CPU of the server. Although hardware iSCSI HBAs, which takes this load from the CPU exist, I have never seen they are used in our clients..

Okay.. Enough for the introduction for noew.. I want to write about ISCSI Speed comparisons , as well.. But it is another story.. Today I will write how to mount an iscsi Disk in Linux..


1. For the first time usage, you need install iSCSI Software with below command.. I suppose you have yum confiugured..

yum install iscsi-initiator-utils

2. Make sure iscsi.conf node parameter is set to automatic. This makes iscsi targets to be discovered and connected after rebooting the Linux machine .

[root@erman ~]# vi /etc/iscsi/iscsid.conf

node.startup = automatic

3. Create the disks on NAS storage interface as in the image below. Here we are using a Seagate for our tests..

4. After creating the disks and configured them accessible from ISCI, we start our discovery in the Linux machine to see the iscsi targets..

[root@erman~]# iscsiadm -m discovery -t st -p >>>>>> NAS STORAGE IP,1,1,1,1,1,1

5. Next, using the iscsiadm with the proper arguments, we attach the storage disk in to our Linux machine..

[root@erman ~]# iscsiadm -m node --targetname "" --portal "" –login

Logging in to [iface: default, target:, portal:,3260] (multiple)
Login to [iface: default, target:, portal:,3260] successful.

We check the disk after attaching to it... In this example it is /dev/sda

[root@erman~]# fdisk -l

Disk /dev/cciss/c0d0: 299.9 GB, 299966445568 bytes
255 heads, 63 sectors/track, 36468 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System
/dev/cciss/c0d0p1 * 1 13 104391 83 Linux
/dev/cciss/c0d0p2 14 4356 34885147+ 82 Linux swap / Solaris
/dev/cciss/c0d0p3 4357 36468 257939640 83 Linux

Disk /dev/cciss/c0d1: 899.8 GB, 899898718208 bytes
255 heads, 63 sectors/track, 109406 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/cciss/c0d1p1 1 54703 439401816 83 Linux
/dev/cciss/c0d1p2 54704 109406 439401847+ 83 Linux
Disk /dev/sda: 1099.5 GB, 1099512676352 bytes
255 heads, 63 sectors/track, 133674 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk /dev/sda doesn't contain a valid partition table

6. Next we mount the disk with a classic mount command. In this example, the disk was already formatted with ext3 filesystem.. If it would not be so, we could format it and then mount it..

Note that: there is an important note here when you use Iscsi on linux:

When you restart the machine or ISCSI service, your physical device names can be changed . This can lead a problem with your mount. So, too make the mount points to be stable you have to add the ISCSI disk names to /etc/fstab with their corresponding UID.

To find the UID of an ISCSI disk, follow the action plan described in step7

[root@erman /]# mount /dev/sda /yedek2

[root@erman /]# df –h
Filesystem Size Used Avail Use% Mounted on
/dev/cciss/c0d0p3 239G 88G 139G 39% /
/dev/cciss/c0d0p1 99M 22M 73M 24% /boot
tmpfs 32G 0 32G 0% /dev/shm
/dev/cciss/c0d1p1 413G 388G 4.6G 99% /data1
/dev/cciss/c0d1p2 413G 156G 236G 40% /data2
/dev/sda 1008G 77G 931G 8% /yedek2

7. To find UID use the command below..

[root@erman /]# blkid

/dev/sr0: LABEL="OL5.8 x86_64 dvd 20120229" TYPE="iso9660"
/dev/cciss/c0d1p2: UUID="11c9bd31-46db-43eb-ae12-2c6ecab9d3f1" TYPE="ext3"
/dev/cciss/c0d1p1: UUID="37cb3250-e7ce-4f2a-b21a-dda2acbe650c" TYPE="ext3"
/dev/cciss/c0d0p3: LABEL="/" UUID="0acaad8c-6542-4bd5-ae16-674b4ddfaebe" TYPE="ext3" SEC_TYPE="ext2"
/dev/cciss/c0d0p2: TYPE="swap" LABEL="SW-cciss/c0d0p2"
/dev/cciss/c0d0p1: LABEL="/boot1" UUID="0b5563e1-8467-4e00-bca3-119449036729" TYPE="ext3" SEC_TYPE="ext2"
/dev/sda: UUID="56659ab7-76f6-4da5-9281-e23fa1bcfe47" TYPE="ext3" SEC_TYPE="ext2"
/dev/sdc: UUID="1e92b1d2-c67f-4f44-b4dd-d67653db9e6a" TYPE="ext3"
/dev/sdd: UUID="76f943d9-ff21-4483-a387-da57126a0fc9" TYPE="ext3" SEC_TYPE="ext2"
Once you find the UUID , add the necessary line to /etc/fstab file as follows;

[root@erman/]# cat /etc/fstab
LABEL=/ / ext3 defaults 1 1
LABEL=/boot1 /boot ext3 defaults 1 2
tmpfs /dev/shm tmpfs defaults 0 0
devpts /dev/pts devpts gid=5,mode=620 0 0
sysfs /sys sysfs defaults 0 0
proc /proc proc defaults 0 0
LABEL=SW-cciss/c0d0p2 swap swap defaults 0 0
/dev/cciss/c0d1p1 /data1 ext3 defaults 0 0
/dev/cciss/c0d1p2 /data2 ext3 defaults 0 0
UUID="56659ab7-76f6-4da5-9281-e23fa1bcfe47" /yedek2 ext3 _netdev 0 0

That 's all. I hope you will find this post useful..

No comments :

Post a Comment