OpenSolaris (2008.11) can serve iSCSI target’s easily. I have been using this capability for serving an iSCSI target which is being shared between my Apple Mac’s for TimeMachine Backups. I have been using this for a few months, and now I can backup my laptops using my home 802.11n Wi-Fi.
There are many advantages, like I don’t have to use an external USB disk which always required extra efforts and was not safe (redundant). But I have to be careful so that I don’t double mount the shared device. This is not exactly a problem as I can always create multiple iSCSI targets and bind them to the laptop’s exclusively, this way the host’s would only see and mount the devices for which they have access to.
I will eventually detail out all the steps I followed for configuring the OpenSolaris / iSCSI / ZFS / OSX / TimeMachine setup, in my subsequent blog entries. The basic purpose of this entry is to actually show some performance numbers of iSCSI shared volume to an Apple Mac client.
So first some details about the setup, The OpenSolaris host is configured as follows:
- Intel Core 2 Duo (3.00GHz) with 4GB RAM
- OpenSolaris 2008.11
- Zpool (RAID-Z) consisting of 5 x SATA-II HDD
- Broadcom GbE Adapter
On the client side I am using iSCSI driver for OSX from SNS (Studio Network Solution). All of these machines are connected using a cheap DLINK DIR-655 router. Since this a very small setup with limited number of disks and non enterprise hardware, the performance numbers might not be anything near what it could be with more robust hardware.
What I am trying to achieve is to somehow improve the performance as much as I could with the setup that I have by first identifying the bottlenecks and then optimizing the setup.
Baselining Performance
In order to identify the bottleneck which might exists (and most probably would) on the network subsystem, I started by getting the raw performance of my ZFS file system locally. The tool that I am using for all these test is “iozone ” and I am only limiting my tests to sequential read/write tests. A more exhaustive tests would take a lot more resources and would not be appropriate for the very small setup that I have.
On a five disk RAID-Z, I am getting a very decent performance, around 112MB/s for Writes and 145MB/s for Reads for 512Kb IO size. The IOZone command that used for the benchmark was:
iozone -ac -s8G -i0 -i1 -y4k -q1M
I was using an 8GB file for the benchmark to ensure that i am reading and writing enough data to bypass the caching effect which can spoil the results; 8GB is twice the size of the RAM, so no chances for read or write from cache.
iSCSI Performance
Then I tested a block device powered by the same zpool and exported as an iSCSI target using the default iSCSI subsystem (iscsitadm). The RAW volume was formatted as a HFS+ Journaled File system from the Apple Mac. Following graph shows the bandwidth that I got on my Apple Mac.
I am really disappointed by the write performance that I am getting out of my iSCSI setup. Read performance is actually not bad ~98MB/s on a cheap DIR-655 GbE switch is good. There are couple of different suggestions that I have received from various different places, for improving the performance:
- Adding a separate dedicated fast block device for storing ZIL “Intent Log”. Typically this is supposed to be a Solid State Disk or something similar like i-RAM. But that would defeat the purpose of a cheap and reliable storage.
- Trying out the COMSTAR based iSCSI implementation instead of the default “iscsitadm” subsystem. Apparently COMSTAR is much more optimized and offer’s better performance. Unfortunately it’s not provided with Open Solaris 2008.11 distribution and I’ll have to upgrade my installation to the development release of 2009.06.
ZFS uses the Intent Log for all the synchronous writes, so I assume that the performance of writes would improve dramatically if I can get a fast transactional block device (like an SSD) for separating the ZIL. But I am not going to be able to get an SSD disk for these tests as the cost of the SSD disk alone would constitute for more than 50% of the entire setup. I will be writing more about my optimization efforts and results, in the meantime I would greatly appreciate any suggestions or feedbacks.
