NFS, iSCSI oh boy!

net-managerThe joys of storage!  Recently, I decided to dabble again in iSCSI.  With the VMware farm for “itinthedatacenter.com” getting larger, my small time NFS set up was starting to show its limits.  Even with 4 drives, and spreading out the I/O, the Linux NFS server I use (OpenFiler BTW) was trashing more and more with increased activity.

OpenFiler includes iSCSI, but I decided to update the drivers and the iSCSI software myself.  OpenFiler uses a rather old iSCSI version, and I wanted to be current.  So, off to compiling and hacking at kernel modules I went.  After about 2 days of effort, the filer was converted over to running iSCSI for my VMFS volumes form NFS.  The performance so far has been pretty good, and the trashing has been reduced.  This is most likely due to the lack of page cache that the iSCSI server uses.

My next task is to upgrade the storage controller and array.  Internal disks and standard SATA quite frankly blow chuncks on consumer and even enthusiast boards.  So, I have decided to pick up a new eSATA RADI card, along with a new 8 bay storage array.  It is a couple months away from implementation, but the plan is done.

I expect that I’ll be able to get 70-90MB/s from the new array with my existing drives, versus the 45-60MB/s I get now.  Reads should be quite improved with a controller having between 128MB and 256MB RAM.

I’m thankful I moved away from NFS to iSCSI.  NFS works, but, the lack of features and problems with NFS server reboots forced me to make the change.  ESX4.0 handles iSCSI much better than 3.5 ever did.  I even have created boot floppy for W2K8/W2K3/W2K8R2 for the VMware PVSCSI devices so that they can be booted from.  Oh, and yes, Virginia, you *CAN* boot from VMware PVSCSI disks in Windows.  It works nicely once you figure out how VMware has implemented it.