In August 2008 I built a high performance, high reliability FTP/Samba/CIFS server out of commodity PC parts to run OpenSolaris and ZFS. It's worked out very nicely - it's remained extremely fast, it's flexible enough to split off filesystems for friends and family, and I haven't lost a single bit of data. The one down-side is that it's a little noisy and eats a good bit of power - up to 113 watts while it's spinning up the drives and 80 watts at idle. And that's why I was very excited when a friend pointed me to the SuperMicro X7SPA-H. While most 2nd generation Atom boards are built around the NM10 chipset which supports 2 SATA devices, the X7SPA-H uses the standard ICH9R chipset and supports 6 devices making it a perfect candidate for a low-power server duty.
The D510 Atom CPU and Pineview claims just 13 watts max TDP, so it ought to run very cool and consume very little power. The earlier Diamondville Atom chipset just supported PCI, but Pineview supports PCI Express for fast I/O if you want to add a second SATA card for even more ports. Right above the SATA ports there's an internal USB port, ideally placed for a small USB boot drive. The only thing that doesn't make sense is the Molex plug next to the SATA plugs. According to page 16 of the manual it's a "Power Connector for Add-on devices", but it's a male plug which means add-on devices can't plug into it. Weird.
I ordered mine from Acme Micro and it arrived very fast. Since the board only uses laptop SODIMMs, not ECC RAM, I wanted to be extra careful so I ordered two Hynix HYMP125S64CP8-Y5 chips from Ebay which at that time were the only chips on the tested and approved list, and passed MemTest86+. Good. The 2009.06 CD boots without problems and installs with out-of-the-box support for the chipset, graphics, network card, etc. Very good.
I added a 16GB 16gb Patriot Xporter thumb drive to the USB port and was able to install and boot OpenSolaris without problems. The diskless, fanless system makes zero noise while it's running which is kinda the idea, and 16GB will be more than enough for the operating system. Initial power consumption rates are not quite as low as Intel advertises but not bad either: 27 watts at idle, 30 watts under full CPU use (no drives) compared to 63/66 watts from my old system (no drives). I was careful to 'zfs export' my pool when I shut down on the old board, so I was able to 'zfs import' and start serving from the new system without problems. With four drives the entire system consumes 80 watts peak, 64 watts during heavy drive access, and 41 watts at idle.
I was concerned that my new benchmarks using dd were a lot lower than my previous benchmarks until I realized a couple of things. First, this ZFS pool is a much later version 22, which means I'm comparing apples to oranges. Second, if I just increase the block size I'm writing the throughput actually becomes quite respectable:
soren@sbox:/tank/soren/test$ dd if=/dev/zero of=tmp.bin bs=1024 count=1048576
1048576+0 records in
1048576+0 records out
1073741824 bytes (1.1 GB) copied, 42.1124 s, 25.5 MB/s
soren@sbox:/tank/soren/test$ dd if=/dev/zero of=tmp.bin bs=16384 count=65536
65536+0 records in
65536+0 records out
1073741824 bytes (1.1 GB) copied, 4.91285 s, 219 MB/s
soren@sbox:/tank/soren/test$ dd if=/dev/zero of=tmp.bin bs=65536 count=16384
16384+0 records in
16384+0 records out
1073741824 bytes (1.1 GB) copied, 2.94921 s, 364 MB/s
Version 1.03 ------Sequential Output------ --Sequential Input- --Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
sbox 8G 47441 79 105408 68 49655 43 57557 98 138997 42 168.3 2
------Sequential Create------ --------Random Create--------
-Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
16 8679 99 +++++ +++ 8147 93 7083 99 +++++ +++ 8381 99sbox,8G,47441,79,105408,68,49655,43,57557,98,138997,42,168.3,2,16,8679,99,+++++,+++,8147,93,7083,99,+++++,+++,8381,99
Minor problems were very minor. The video card is VGA-out only and the default screen size seems to be 1024x768. Sometimes the "poweroff" command would only suspend the machine, not power it off. Either one of these problems are not especially problematic.
Major problems started with this error on the console. "Device is gone", in reference to sd07, the USB drive. The drive itself was properly seated and the machine rebooted without problems, but after a few hours' uptime I got the same message again. I tried disabling the hal and rmvolmgr services but all that did was make the failures happen silently. I thought that maybe my USB drive or the USB port was flaky so I tried reinstalling and booting from a new 2.5" 80gb laptop drive. This problem gradually disappeared, possibly due to a combination of BIOS settings and the b133 system update, which brings me to problem #2…
About once every 12-24 hours the machine would just drop off the network. I couldn't get into the shares via CIFS and I couldn't get in via SSH either. I could log in via the console and everything seemed fine, but I couldn't ping other machines on the network. Eventually I ran Wireshark and noticed something really strange - Solaris was sending ping packets and the remote machines were responding them, but Solaris wasn't seeing the responses. I got pretty deep into this problem with the help of a Sun employee who was able to verify that the driver was accepting the packets and passing them up into the IP stack, but we were never able to resolve the problem.
The X7SPA-H is potentially a very nice board. It's very small, reasonably priced, dead silent, and consumes half the power of a desktop system. (It would be even nicer if it accepted ECC RAM.) Unfortunately it's just not stable, and after two weeks of tinkering I gave up and put my old DP35DP based system back together and listed the SuperMicro board on eBay. I've got several friends running this board with Linux and FreeBSD, but until the OpenSolaris bugs get worked out I do *not* suggest building a ZFS server with this one.