my face

Test Audiences

Richard Matheson's "I Am Legend" was a clever commentary on moral relativism and the banality of evil, and the movie did a pretty good job of trying to be faithful to the book; setting up the scene and the characters and giving you just enough information so that you could see where the twist at the end came from when it finally happened.

But then the reports from the test screenings came back. Audiences hated the ending. Either they didn't get it, or they didn't like it, or they weren't comfortable with its implications. They didn't like the idea that the guy they'd been cheering for turns out to actually be a world-class jerk; that we might want to spend a little more time questioning our assumptions of justice and morality. So there was a mad scramble at the end to rework the movie and give it a conventional Hollywood ending: Robert Neville holds off the evil zombies while the girl and boy escape, bad guys are punished, good guys are safe, the end. Which is too bad. The original ending was not only very good, it's the whole point of the movie. It's the reason why the movie is titled "I Am Legend".

"Inception" made a billion zillion dollars not just because of the trailers and the posters but because it had really good buzz. You get buzz (according to some people anyway) by appealing to fanatics. By appealing to the people who recognize good stuff and get so excited that they tell all their friends. And those people aren't necessarily the ones you'll get corralling strangers on the 3rd Street Promenade in Santa Monica. I wonder whether the whole concept of test audiences is fundamentally flawed. Obviously this has been happening for years, but it's especially frustrating to see it unfold in front of you.
my face

New QR-Code Business Cards

I designed myself some new business cards and they arrived today. Human readable on one side, machine readable on the other.

My New QR-Code Business Card

I generated the QR code with the "Barcode Scanner" app, which comes standard on Android phones. Select a contact, select "share", select "Barcode Scanner", and there's a QR that you can save. This online generator will do the same thing but with much more options.
my face

Kill the Motorola 2210-02 ADSL2+ Modem with Fire

When my ISP and I started trying to fix the DSL connection the first thing they did was send me a new replacement modem. I swapped out the old one for the new replacement, but it didn't fix the problem so we figured "must not be the modem" and we went off on a snipe hunt involving data caps, fast/interleaved rates, two AT&T visits, and an rather expensive internal rewiring of the house.

The AT&T guy visited this week to test the lines and reported that, if anything, my phone lines are exceptionally good. He also added that "I've seen a lot of problems with this silver Motorola modem". I told him about the impossible "-17.-20 SNR" errors and he said yeah, you might as well just replace the modem with something like 2Wire, which is pretty reliable. I went on Craigslist and found a guy selling a 2Wire 2701HG-B. Plugged it in, spent about 15 minutes figuring out how to change its settings to work with our network, and suddenly we had total stability. I had two bad modems. says that they've been aware of the problem for a while but are continuing to use this model because Motorola fixed the problem. They say that serial numbers beginning with 157 don't have the bad capacitors. Except that the replacement that they sent me has a 157 serial number, and it was also broken, but I don't care, because I've got my own 2wire which is very stable and reliable now.

In conclusion, if you see this modem, kill it with fire.

Update: just sent me a third replacement. And it's also a 157 serial Motorola 2210-02 modem. And it's also busted. I am barely surprised.
my face DSL Woes

I've had DSL service at my house and it's been really great for about five years, but recently the connection started dropping intermittently. Nothing too unmanageable, I'd just occasionally notice my AIM window lose all its contacts and they'd pop right back as the connection dropped and reconnected. The drops kept getting worse, and things got pretty unmanageable by about two or three weeks ago. I'll get drops and reconnections every minute or two, some of them lasting a few hours or more.

I thought that the problem was the wiring in my house, which is very old, so I had my electrician and had him pull new Cat5e wires. That didn't work, so I tried taking the wires out of the problem by plugging the modem directly into the line's point of entry on the side of the house and running a long ethernet cable to the router. That didn't work either, which told me that the problem is somewhere outside the house either on the dry loop that AT&T provides or the network that provisions it. has pretty good customer service. You call their number and you get someone with a clue, frequently a CS student from the local college who will give you straight answers. They sent me out a new modem, but swapping it didn't work. They sent AT&T out to inspect the line, but AT&T said it was good. Then they tried tinkering with the transmission and speed settings to see if they could keep things stable. For the last few years I've been getting 12Mbps down, 1100kbps up, and for the last week or so we've been trying 9, 10, and 11Mpbs down and 800-1000kbps up, but we could never find settings that were stable. One of the techs suggested that maybe I was too far away from the central office for a fast connection, and I assured him that my house was in the exact same place it's been for the last five years when I was getting a great signal. On Friday I suggested that we take a different track. Rather than try to get the fastest stable connection, what if we tried setting it to the slowest stable connection? They said OK and capped my line to 3Mbps down, 300kbps up, but the connection remained completely unstable. While I've been writing this post the line has been capped at 1.5M/160k and has gone up and down three times in 20 minutes. Whatever the problem is, it's not related to line quality, although right now the modem is reporting that my upstream SN Margin is -17.-20 which is impossible, since SNR can't be negative and that isn't even a number. has pretty good customer service, and they're giving it a good try, but none of us can figure out what's going on. As of this morning my best explanation was the outside my house. They've been growing for the last few years, and they're finally up high enough that they're hitting the wires. I figured this might also explain why the outage was intermittent - it's fine for a while, and then the wind kicks up and the branches move and the lines stretch. But this afternoon a guy came out and looked at the trees and said they were fine.

There's an AT&T truck roll scheduled for my house on Wednesday morning, which I consider a last ditch effort. If that doesn't work there's a Comcast installer scheduled for Thursday morning, so either way I'll hopefully have this fixed soon. Until then I'm stuck playing this ridiculous game of internet "red light / green light" while feeling like as the boyfriend, the resident geek, and the landlord/homeowner I've basically failed my duties three different ways.
my face

Big O Tire Patch Scam

The last time I needed tires for my Prius I got them at Big O, where one of the benefits is "free flat repair". If tires installed by Big O ever get a nail they'll patch the tire for free. This week I noticed a tack in the right front tire causing a slow leak, so I went in to Big O for my free patch.

At the counter I started realizing that this "free offer" might be a scam. There was a big laminated graphic glued the counter, outlining tire manufacturers' repair guidelines which forbade Big O from repairing anything on the sidewall or the "shoulder" of the tire. The only place they could patch a tire was the belt, around the middle of the tread. Fortunately, that's where the tack was, but there was a problem.

"Your tires are pretty worn, and the manufacturer's repair guidelines prevent us from patching a tire this worn."
"I'm OK with that. Can you patch it anyway?"
"It's too risky to do it."
"It was risky for me to drive here with a nail in my tire in the first place. Won't a patch be less risky?"
"We still can't do that, but we can install some Big O brand tires today, or we can special order some others for tomorrow."
"I'll order the Goodyears. Can you patch my tire in the meantime?"
"No, it's too risky. We'll just mount the spare tire."
"But isn't it risky to drive on the little rubber donut?"
*shrug* "We'll hold onto your rim until your tires come in."
"I'd rather hold onto my own rim, thanks."

On one hand they were probably right. I'd been eyeing my tires for a month or so, and they'd been getting pretty worn, and I really did need to replace them. On the other hand they seemed to be making it hard for me to have any other option. Their "free flat repair" offer seems designed to get you into their store, where they come up with narrow circumstances where their flat repair can actually occur, and you become a captive audience for their sales pitch. Don't want to buy their tires? Good luck driving away. They've already removed your old tire from the rim, thrown it away, and want to hold your rim hostage until you return to get your new tires put on.

What bugged me about this approach is that I was ready to be a repeat customer. I had a positive experience last time; the service, work, price and product were excellent. I knew I needed new tires again and I was happy to return to their store for more of the same. But I felt icky getting railroaded into purchasing something that I was planning to purchase anyway.
my face

The VFX Economy and Me

The visual effects business model is a lot like the Las Vegas construction model. We're hired to build a big fancy casino. We bid on the work, and we're paid the same whether the casino makes billions of dollars or fails completely. This gets us off the hook if a movie flops, but it's frustrating when you don't see a penny when something you've put your heart and soul into becomes the highest grossing films of all time (adjusted for inflation). Avatar, Star Wars, Titanic, Lord of the Rings - people went to see these films as much for the visual effects as for Sam Worthington, Mark Hamill, or Elijah Wood.

Producers, directors, and stars walk away from these successes paid incredibly well. Visual effects production facilities have a much harder time. Our profit margins are extremely narrow and operating costs are extremely high. A year's worth of profit from a well-run production can be consumed by a few weeks of "overhead" between projects, leaving studios in the difficult position of laying off the talent largely responsible for creating the latest blockbuster so that the entire facility doesn't go out of business. I can't count the number of times my friends, colleagues, and I have heard "I'm sorry, you did excellent work but there's no new work right now. Hopefully we'll find some and hire you back before someone else does." Sometimes a whole studio goes out of business. Laika shut down its computer animation department 7 months after "Coraline" was released. CafeFX are "downsizing and retooling" after the commercial successes of "Shutter Island" and "Alice in Wonderland". Disney closed Image Movers Digital throwing away a massive amount of IP, infrastructure, and personnel just 3.5 years into a 10 year contract.

Artists on the production side feel like they too much value to the century's dominant art form to deserve this. Pipeline engineers spend a lot of time reinventing the wheel and re-training artists on these new wheels as companies start up and collapse. Industry churn means more money gets wasted on the 'churn' of starting up and shutting down. High overhead reduces the money available to pay employees' salaries. Clients receive higher bids and less consistent product. It's in everyone's long-term interest to build an efficient, stable industry.

The above is mostly my editorialized summary of the Open Letter to James Cameron, posted at the Huffington Post a month before the Academy Awards by Lee Stranahan. He asked James Cameron to "speak out on behalf of visual effects artists and how they are being treated unfairly." It didn't happen, but the article caused a lot of chatter in the VFX community that resulted in an online town hall meeting between Warner Brothers VP Chris deFaria, VES chair Jeff Okun, and former ILM/DD CEO Scott Ross.

For anyone looking for a major shake-up the discussion was underwhelming. All three basically agreed that the industry doesn't owe artists anything. Artists owe it to themselves to negotiate better terms, and if studios or artists accept bad terms or agree to do work at a loss that's their fault, not the industry's. Some work has become commodified, particularly work like roto requiring physical correctness rather than artistic control. It's still possible for top-level VFX studios to be paid top dollar for top-quality work, but top-quality work requires a lot of expensive R&D, prices drop off pretty fast, and "state of the art" is a very swiftly moving target. It's not uncommon to see last year's million dollar effect become this year's ad gimmick. Companies like Pixar do well because they produce and sell their own content, but if you're in the business of making VFX for someone else's movies you're in a tight spot.

Since before I had a job anywhere I've looked up to groundbreaking visual effects pioneers like Ray Harryhausen, Dennis Muren, Tony Apodaca, and Joe Letteri. Their talent, genius, dedication, and research enable them make amazing images, but all that is secondary. The primary thrust of their research and trailblazing is to keep their studios and employees competitive, in business, and making money. At this point the industry as a whole is pretty good at stylized or photorealistic rendering. What we're not so good at is getting paid for it. The VFX industry needs right now is less technical R&D and more business R&D. And that's why I've been thinking of going back to school, getting an MBA or postgraduate economics degree, and seeing what I can do about it. I've spent over a decade optimizing workflows, disk access, and render times. I'd like to try to optimize the business itself.
my face

Opensolaris/ZFS on the Supermicro X7SPA-H (Update)

I've got some good news and some bad news about Opensolaris on the Supermicro X7SPA-H.

The Good News

Whatever was causing my server every few hours on Opensolaris snv_133 was fixed in snv_134. Without any further intervention on my part I now enjoy a reasonably fast, reasonably robust, problem-free experience with the 134 distribution.

soren@sbox:~$ uptime
11:35am up 10 days 11:35, 2 users, load average: 0.01, 0.00, 0.00

Performance-wise the board isn't as fast as the old board was, but it's completely acceptable for home NAS use:

soren@sbox:/tank/soren/test$ dd if=/dev/zero of=tmp.bin bs=1024 count=1048576
1048576+0 records in
1048576+0 records out
1073741824 bytes (1.1 GB) copied, 42.8177 s, 25.1 MB/s

soren@sbox:/tank/soren/test$ dd if=/dev/zero of=tmp.bin bs=16384 count=65536
65536+0 records in
65536+0 records out
1073741824 bytes (1.1 GB) copied, 5.63977 s, 190 MB/s

soren@sbox:/tank/soren/test$ dd if=/dev/zero of=tmp.bin bs=65536 count=16384
16384+0 records in
16384+0 records out
1073741824 bytes (1.1 GB) copied, 3.55965 s, 302 MB/s

Version 1.03d       ------Sequential Output------ --Sequential Input- --Random-
                    -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
sbox             8G 33094  55 49144  31 35313  30 58180  98 220054  71 543.9   5
                    ------Sequential Create------ --------Random Create--------
                    -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
                 16  8939  97 +++++ +++  9521  99  8827 100 +++++ +++  9937 100

The Bad News

Oracle purchased Sun three weeks ago, putting the fate of OpenSolaris in doubt. On one hand the license is still free. On the other hand most of the contributors to Opensolaris have been Sun employees, who are now Oracle employees. What was a very nice idea under the old Sun management but serves no useful purpose under Oracle, which might be why some developers are leaving. It certainly doesn't help that 2010.03 hasn't been released yet either.

I'm too uninformed to know what I'm talking about. I am not heavily involved in the open source community generally or Opensolaris specifically, I'm just a guy who was very impressed with ZFS's ability to keep my bits where I left them. I'd love to stick with Opensolaris if it remains a viable option, but I don't want to be stuck in a dead end OS with bugs that don't get fixed.

If things stay bad I guess there are alternatives. A friend who set up her storage recently ended up going with FreeNas, which is fine for her, but the stable version uses ZFSv6 and the experimental version still uses ZFSv13 which isn't good for folks like me using dedup on ZFSv22. I won't be able to re-point my spool, I'll have to make a whole new pool with new disks, copy everything over, and live without features like dedup until FreeNAS catches up. There's also NexentaStor Community Edition, which allows up to 12TB of free storage - more than enough for my needs. And I suppose there's also non-ZFS options like BTRFS or hardware RAID or something, but the amount of time I've spent enjoying ZFS while being impressed with its stability, performance, and ease of use really makes me hope things work out and OpenSolaris stays viable.
my face

Data recovery after 'dd'

Various disk formatting utilities will let you specify "secure erase" options that let you write several passes of random data to the drive for extra security. Is there any real precedent for this? I realize it's supposedly theoretically possible, and perhaps it is given an electron microscope and the funding of a state intelligence agency, but is it practically possible for whatever random human who purchases my used disks on eBay?

Here's my challenge: I will take a disk and write a repeating single ascii character to the entire disk. Then I will use "dd if=/dev/zero of=/dev/sda1" to zero out the device. Is there anyone who thinks they'll be able to tell me what ascii value was written to the disk before I erased it? If it's possible to recover a whole file (or a whole disk) it certainly ought to be possible to recover one byte.
my face

Daily Show pullout from leaves deaf fans with no captions

My partner Kathryn is deaf. I'm a big fan of the Daily Show, but we can't watch it together when it runs on DirecTV because the closed captioning is pretty terrible. The fresh content every night requires a live captioner who struggles with limited success to keep up with fast-paced dialog filled with unfamiliar words. The closed captions frequently drop major parts of the spoken content and have so many major spelling and transcription errors that the show is difficult to understand at least to our deaf audience of one. The captions also lag behind by 10-30 seconds, making any of the visual jokes fall flat as well. It's frustrating enough that she just leaves the room and finds something else to do when I turn on the TV. Shows are archived and viewable online at, but without subtitles.

Fortunately, for the last year or so we've had Hulu, which streams the Daily Show and Colbert Report with its own captions. The Hulu captions are high quality and complete with few if any spelling errors, and they're well-synchronized with the video. Hulu has been a godsend for deaf fans of TDS/TCR, which why we've been very sad to learn that the Daily Show and Colbert Report were pulling out of Hulu. Comedy Central is unintentionally cutting off the one place we've been able to reliably find deaf-friendly content.

I understand that The Daily Show probably don't want Hulu to compete online with their own website for advertisers and eyeballs, but we're very sad that Comedy Central is cutting off the one outlet that deaf fans have for reliable captioned content.
my face

Opensolaris/ZFS on the Supermicro X7SPA-H Atom D510 Pineview

In August 2008 I built a high performance, high reliability FTP/Samba/CIFS server out of commodity PC parts to run OpenSolaris and ZFS. It's worked out very nicely - it's remained extremely fast, it's flexible enough to split off filesystems for friends and family, and I haven't lost a single bit of data. The one down-side is that it's a little noisy and eats a good bit of power - up to 113 watts while it's spinning up the drives and 80 watts at idle. And that's why I was very excited when a friend pointed me to the SuperMicro X7SPA-H. While most 2nd generation Atom boards are built around the NM10 chipset which supports 2 SATA devices, the X7SPA-H uses the standard ICH9R chipset and supports 6 devices making it a perfect candidate for a low-power server duty.

SuperMicro X7SPA-H


The D510 Atom CPU and Pineview claims just 13 watts max TDP, so it ought to run very cool and consume very little power. The earlier Diamondville Atom chipset just supported PCI, but Pineview supports PCI Express for fast I/O if you want to add a second SATA card for even more ports. Right above the SATA ports there's an internal USB port, ideally placed for a small USB boot drive. The only thing that doesn't make sense is the Molex plug next to the SATA plugs. According to page 16 of the manual it's a "Power Connector for Add-on devices", but it's a male plug which means add-on devices can't plug into it. Weird.


I ordered mine from Acme Micro and it arrived very fast. Since the board only uses laptop SODIMMs, not ECC RAM, I wanted to be extra careful so I ordered two Hynix HYMP125S64CP8-Y5 chips from Ebay which at that time were the only chips on the tested and approved list, and passed MemTest86+. Good. The 2009.06 CD boots without problems and installs with out-of-the-box support for the chipset, graphics, network card, etc. Very good.

I added a 16GB 16gb Patriot Xporter thumb drive to the USB port and was able to install and boot OpenSolaris without problems. The diskless, fanless system makes zero noise while it's running which is kinda the idea, and 16GB will be more than enough for the operating system. Initial power consumption rates are not quite as low as Intel advertises but not bad either: 27 watts at idle, 30 watts under full CPU use (no drives) compared to 63/66 watts from my old system (no drives). I was careful to 'zfs export' my pool when I shut down on the old board, so I was able to 'zfs import' and start serving from the new system without problems. With four drives the entire system consumes 80 watts peak, 64 watts during heavy drive access, and 41 watts at idle.


I was concerned that my new benchmarks using dd were a lot lower than my previous benchmarks until I realized a couple of things. First, this ZFS pool is a much later version 22, which means I'm comparing apples to oranges. Second, if I just increase the block size I'm writing the throughput actually becomes quite respectable:

soren@sbox:/tank/soren/test$ dd if=/dev/zero of=tmp.bin bs=1024 count=1048576
1048576+0 records in
1048576+0 records out
1073741824 bytes (1.1 GB) copied, 42.1124 s, 25.5 MB/s

soren@sbox:/tank/soren/test$ dd if=/dev/zero of=tmp.bin bs=16384 count=65536
65536+0 records in
65536+0 records out
1073741824 bytes (1.1 GB) copied, 4.91285 s, 219 MB/s

soren@sbox:/tank/soren/test$ dd if=/dev/zero of=tmp.bin bs=65536 count=16384
16384+0 records in
16384+0 records out
1073741824 bytes (1.1 GB) copied, 2.94921 s, 364 MB/s

Version  1.03       ------Sequential Output------ --Sequential Input- --Random-
                    -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
sbox             8G 47441  79 105408  68 49655  43 57557  98 138997  42 168.3   2
                    ------Sequential Create------ --------Random Create--------
                    -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
                 16  8679  99 +++++ +++  8147  93  7083  99 +++++ +++  8381  99


Minor problems were very minor. The video card is VGA-out only and the default screen size seems to be 1024x768. Sometimes the "poweroff" command would only suspend the machine, not power it off. Either one of these problems are not especially problematic.

Major problems started with this error on the console. "Device is gone", in reference to sd07, the USB drive. The drive itself was properly seated and the machine rebooted without problems, but after a few hours' uptime I got the same message again. I tried disabling the hal and rmvolmgr services but all that did was make the failures happen silently. I thought that maybe my USB drive or the USB port was flaky so I tried reinstalling and booting from a new 2.5" 80gb laptop drive. This problem gradually disappeared, possibly due to a combination of BIOS settings and the b133 system update, which brings me to problem #2…

About once every 12-24 hours the machine would just drop off the network. I couldn't get into the shares via CIFS and I couldn't get in via SSH either. I could log in via the console and everything seemed fine, but I couldn't ping other machines on the network. Eventually I ran Wireshark and noticed something really strange - Solaris was sending ping packets and the remote machines were responding them, but Solaris wasn't seeing the responses. I got pretty deep into this problem with the help of a Sun employee who was able to verify that the driver was accepting the packets and passing them up into the IP stack, but we were never able to resolve the problem.


The X7SPA-H is potentially a very nice board. It's very small, reasonably priced, dead silent, and consumes half the power of a desktop system. (It would be even nicer if it accepted ECC RAM.) Unfortunately it's just not stable, and after two weeks of tinkering I gave up and put my old DP35DP based system back together and listed the SuperMicro board on eBay. I've got several friends running this board with Linux and FreeBSD, but until the OpenSolaris bugs get worked out I do *not* suggest building a ZFS server with this one.