As it should be evident from my recent posts, I have been playing with iSCSI and ZFS a lot lately. And the only OS in my home that I work on is OS-X on my laptops. And so started my quest to find a perfect iSCSI solution (based on OpenSolaris) for my OSX environment.

Apple does not provide any iSCSI initiator with Leopard, it’s been rumored that with Snow Leopard Apple might release iSCSI initiator. But till then there are at least 3 iSCSI initiators that exists for OS X users.

I have been told that iSCSI is a very complex specification and the success with a particular (initiator / target) combination depends upon how comprehensive they are (and how lucky you are). And that’s exactly what I found.

As my first test I just downloaded the globalSAN iSCSI initiator and used it to connect to the “iscsitgt” based target from my OpenSolaris 2008.11 Workstation. I was able to connect to the target the read performance was great, write kinda sucked but this was the first time I was playing with iSCSI and i liked the idea of block level access over Wi-Fi

Then I started becoming inquisitive (greedy), and wanted to share multiple thin provisioned iSCSI devices to be used as TimeMachine, iTunes Library, iPhoto Library across my home network to be shared between multiple laptops. And for achieving that i started playing with advanced features like “Access Control List”, “CHAP Authentication” et. al.

So globalSAN iSCSI initiator was able to scan the target’s but as soon as I tried connecting to the target’s, the iSCSI initiator went into some sort of infinite loop and hung.

I came to know about this latest iSCSI implementation by Sun Microsystem which is written from scratch and is blazing fast. I had to try it and I did. Unfortunately with globalSAN iSCSI initiator I was unable to format the block devices.

Then I concentrated the next week getting access to different iSCSI initiators available for OSX and came across three:

• globalSAN (www.studionetworksolutions.com)
• Small-Tree (www.small-tree.com)
• Atto Xtend SAN (www.attotech.com)

I have not performed extensive tests (as of now) so I will just present my finding in form of a small table.

I would also like to state that compatibility table only represents the compatibility between these initiators with OpenSolaris based targets and does not talks about performance only compatibility.

As it’s visible from the table above Atto’s Xtend SAN was the only initiator that has so far been able to connect and access both the iSCSI subsystem’s with all variations (that I have tested so far). It’s also the one that costs the most ~200$ per seat, but hey it’s the only one which works so can’t complain.

If the requirements are not for using ACL’s or COMSTAR globalSAN has the best interface integration with the OSX preferences pane. On the other hand ATTO’s interface show’s age but also has many tunable options available.

Disabling ZIL (Intent Log)

Before actually investing in some kind of Solid State Device, it would be nice to actually quantify how much improvement in write performance is possible. That is if I separate the ZIL to a dedicated, fast, transactional device.

A quick and clean test would be to run a similar IOzone benchmark on the iSCSI device and turning off the Intent Log completely from the ZFS file system. This should give me the extreme best case as it would take away the entire overhead of ZIL.

Note: Keeping the ZIL disabled is not at all recommended in any production system as it could severely damage the file system. And the file system is prone to corruption in the event of any crash.

In order to turn off the ZIL dynamically the following command can be executed. I personally feel it’s a nice idea to restart the machine after making such change, especially if I am benchmarking. But I am too lazy so I followed the following steps:

Step I: Disable the ZFS Intent Log

# echo zil_disable/W0t1 | mdb -kw

Step II: Unmount the ZFS file system in the concerned pool

# zfs unmount /tank

Step III: Export the zpool

# zpool export tank

Step IV: Import the zpool back, which would also mount all the file systems

# zpool import tank

Step V: Execute the benchmarking applications

# iozone -ac -s8G -i0 -i1 -y4k -q1M

The results were very interesting, as thought there was a big improvement in writes. But I didn’t expect the performance to increase by 26x on average. The following graph shows the numbers that I got after turning off the ZIL.

Also notice that the Read performance also increased big time, ~110MB/s which is an improvement of around 12%. But it’s hard to say that the bottleneck is the back-end (solaris/zfs/iscsi) or the network itself since the performance is near-wire speed. The results from the perevious tests are mentioned in this entry

Summary

Summarizing all the pervious tests the following graphs shows the improvement I achieved with the two changes:

1. Moving the file system block size from 8k to 128k
2. Running the tests with ZFS intent log disabled (case for SSD)

There is still a lot to work on getting a perfect working system. The next step would be to insert a USB Flash Disk and see where does the performance lands (esp. on writes)