How to configure software RAID on Dell Express Flash PCIe SSDs

Applies to: 
---------------------------------------------------------------------------------------------------------------------------
Operating System(s) - Oracle Linux 6.x, RHEL 6.x

Server Platform(s) - PowerEdge R720, R820

Author:
---------------------------------------------------------------------------------------------------------------------------
Naveen Iyengar

Problem: 
---------------------------------------------------------------------------------------------------------------------------
How to configure software RAID on Dell Express Flash PCIe SSDs

Solution: 
--------------------------------------------------------------------------------------------------------------------------

1. Identify the Express Flash block devices– Dell’s Micron Express Flash drives show up as the following block devices in EL6.x OS

$> cat /proc/partitions

Major minor #blocks name
251    256     341873784  rssda
251    512     341873784  rssdb

2. Create a Partition – Use the fdisk linux utility as follows to create an 'fd' type partition on the Flash drives

$> fdisk –u /dev/rssda

Command (m for help): n
Command action
  e extended
  p primary partition (1-4)
p

Partition number (1-4): 1
First sector (56-683747567, default 56: 128
Last sector, +sectors or +size…, default 683747567: <hit Enter> 

Command (m for help): t
Selected partition 1
Hex code (type L to list codes): fd
Changed system type of partition 1 to fd (Linux raid autodetect) 

Command (m for help): wq
The partition table has been altered!
$>

3. Create software RAID– Use the Linux utility tool called mdadm as follows to create the software RAID array on the Express Flash drives. The following example create a RAID1 using two Flash drives /dev/rssda1 and /dev/rssdb1.

$> mdadm --create /dev/md0 --level=mirror --raid-devices=2 /dev/rssd[ab]1

4. Viewing the details of the array: View the status of the multi disk array md0.

$> mdadm --detail /dev/md0

5.  Make the array persistent across reboots: To add md0 to the configuration file so that it is recognized next time on boot, do the following:

$> mdadm -Es | grep md0

Check if the above command displays the details of the md0 array created. If not, try

$> mdadm -Es | grep “md/0”

Depending on which of the above two command works, run the appropriate command below to add info to the mdadm.conf file

$> mdadm -Es | grep md0 >> /etc/mdadm.conf

Or

$> mdadm -Es | grep “md/0” >> /etc/mdadm.conf

6.  Check for resync complete: Run the --detail option to make sure that the two SSDs in the array are not in the Resync Status, or wait until it finishes resyncing before you run a test against them.

$> mdadm --detail /dev/md0

A.    Deleting the array:

1. To halt the array:
   $> mdadm --stop /dev/md0
2. Delete the superblock from all drives in the array
   $> mdadm --zero-superblock /dev/sd[abc]1
3. Edit /etc/mdadm.conf to delete any rows related to the deleted array(s)

 At the command prompt, type:

1.   #> fdisk /dev/<block_device>

   where <block_device> is the name of the block device that you are adjusting. For example, if the block device is /dev/sdb, type: fdisk /dev/sdb

   The system displays the following message:

   The number of cylinders for this disk is set to 8782. There is nothing wrong with that, but this is larger than 1024, and could in certain setups cause problems with:
   1) software that runs at boot time (e.g., old versions of LILO)2)booting and partitioning software from other OSs 
   (e.g., DOS FDISK, OS/2 FDISK)
   
       a. Command (m for help): x # To enter the expert Mode
           b.  Expert command (m for help): b # To move beginning of data in a partition
           c.  Partition number (1-4): 1 # The partition number to be aligned
           d.  New beginning of data (128-xxxxx, default 128): 128
   
   
   

   NOTE	128 blocks or 64KB is the default Stripe Element Size of EqualLogic PS Series and PowerVault MD Line of storage arrays. Hence we adjust the beginning of the data partition to 128 blocks.

   
       e. Expert command (m for help): wq # write and quit
   

2. Repeat step 1 for all the disks that need to be aligned 
   

3. Run the following command to re-scan all the partitions:

   #> partprobe
   

4. Verify that the partition has been aligned by running the following command:
   

   #> fdisk -ul /dev/<block_device>

   where <block_device> is the name of the block device that we aligned the partition of. Below is an example of a sample output of the above command on a block device that has been aligned. If you partition is properly aligned then you will see 128 under the Start column against your partition.
   
   

   Disk /dev/mapper/mpath70: 53.6 GB, 53697576960 bytes
   255 heads, 63 sectors/track, 6528 cylinders, total 104878080 sectors
   Units = sectors of 1 * 512 = 512 bytes
   Device Boot      Start         End      Blocks   Id  System

   /dev/mapper/mpath70p1  128   104872319    52436096   83  Linux

5. If required, reboot the system for the kernel to be able to re-read the new partition table or if you do not see your newly created aligned partition.