Hardware from hell
It's not easy to do bleeding edge development


Every year I end up refreshing my hardware to save on taxes stay on the bleeding edge. It’s getting hard to spend a lot of money on anything but I had had a small cash injection, so I tried to find hardware that I would actually need for the coming year.

Nearly all my other main hardware I left behind in Sweden, notably my hyperfast octa-core build/storage box, “snapon”. I basically returned home to san francisco with my laptop, and what I had in my storage unit - and as the kind of testing and work I do requires at least 5 machines to do right, I went shopping for new hardware. This is what happened.

Embedded boards

I picked up an odroid C1+, Raspberry Pi B2, and a firefly.

I still haven’t got the firefly board to boot. It requires some weird flashing utility that I have yet to get to work. I’d got it specifically because it had 8GB on board, came with case, and had wifi…

The odroid C1+ appears to be quite fast, but the ethernet driver gave me trouble, it couldn’t crack 400Mbits on a download. They are also stuck at kernel 3.10.80 for some reason. A few more details here.

The raspberry pi story is pretty amazing, inside of a few minutes I’d installed kodi, and was delightedly using it via the tablet as a remote. I went through a brief flurry of finding all my old flac files on various old hard disks to play back on it. It does have some trouble playing back higher rate videos, and I’ve crashed kodi a couple of times - but it’s a darn nice box.

I still have a beaglebone black, which I’d hoped had got reliable enough to use for something. I flashed a new OS on it, it refused to boot, I gave up.

Of course, today, a mere matter of weeks after I got this gear, the latest 64 bit raspberry pi 3 and odroid C2 started shipping, so I’m hopelessly out of date! However given that the raspbian folk are not shipping a 64 bit OS yet, and my experience so far with the C1+, I think I’ll wait for others to make it more robust. Ah, hell, I just bought a pi 3 anyway.

It is still simply amazing that I can get a quad core 1.2ghz box for 79 bucks, in a world where the HDMI cables alone cost 20 bucks, and my preferred keyboard costs 100 bucks… I still keep hoping that someday I can get a box rated to 70C, with flash life measured in decades, with a battery, that I could rely on to keep a few terabytes around on… but so far, nothing meets my standards for reliability on these hacker style boards.

I have some hope that the latest Pi or odroid could become a desktop replacement. I’d really like to see the reliability get as high as intel, the drivers get fully opened, etc, etc.

One thing that bugs me about the form factor is I’d like to have flash drives I trust (msata), rather than these itty bitty little emmc cards, that sort of, kind of, snap on.

X86 boards

My desktop platform of choice for years now has been the nuc series. I love these little guys - very quiet, reasonably fast, great linux support, great ethernet drivers, they bolt on the back of the monitor, etc. I have not a lot of reason to want to change, as even the i3 models are “fast enough” for nearly all my day to day use except kernel/openwrt compilation (which I do on snapon). I used the nucs extensively as small, portable, reliable test platforms, also, where the only limitation was the lack of two ethernet ports.

For x86 based routers over the past 5 years I went through a terrible string of atom based boards, and a rangeley which worked out in the end but would overheat. I really wanted something that booted from flash and never wrote to it again, also, and took less time to boot than your typical x86 box did.

So I need something to be a “faster” x86 based router, capable of shaping to 1Gbit, that I could debug stuff on easily, So I picked up the first dual ethernet nuc I’d seen, ever.

I really liked the design - a serial port, 2 ethernet ports, dual dvi 4 usb3 - nearly hermetically sealed case, room for a sata drive - so I figured that would make a good router substitute for me - as an x86 platform it’s easy to build kernels for and I could slam an ath9k or ath10k card in it to use for wifi testing. Maybe even fiddle with pfsense.

… but - after a few days of operation, the box stopped booting reliably. I’ve reseated everything in it a couple times, and can no longer even get it to install an OS. I swapped out the hard disk, no dice.

I’m pretty hosed without a box like this in my testbed, and I’d kind of planned on getting two (one to emulate delay).

Home routers to hack on

Got a tp-link archer c5 v2 - which had firmware totally unsupported by openwrt. From v1 to v2 was a complete hardware swapout from QCA to Broadcom without changing a word on the box, which I am still quite POed about. I’d return it if I didn’t want to benchmark it eventually.

Got a tp-link archer c7 v2 - which had firmware locked down by the manufacturer. I’m even more POed about that…

David Lang gave me a Linksys 1200ac - which, aside from the wifi, seems too flaky to use day to day, as yet. There was some progress on adding BQL support to back in December, and the wifi got a lot better, but as I don’t know how to make the wifi better, it sits there, acting as a switch, glowing…

I tried to get a mt76 box from alibaba - sold out - this is the new hotness in the OpenWrt 802.11ac world.

So I am still using an older archer c7v2 with the latest openwrt to test cake on, on my main cable gateway to the Internet. The ipv6 support on that thing is amazing and it’s been quite reliable - however it peaks out at 75mbit down on my comcast “blast” service.

A turris omnia will probably become my next eval platform unless I can get a mt76 up and running.

SSDs

I have SSDs everywhere except where I need terabytes of storage. It turns out I don’t need terabytes of storage much anymore, except in the cloud, where it costs too much for me to want to pay for it. Snapon was hosted, for free, by ISC.org for years….

Got a bunch of msata, sdcard, and emmc cards instead, and thought about exposing my vpn to the cloud instead…

There are teeny little msata to usb converters now, too.

I also got a raid 0,1,jbod sata to dual msata converter board. It looked kind of cool (and I don’t trust flash storage as far as I’d like) - but what I’d wanted to use it for was jbod, and use software raid on it - and that didn’t work, as it insisted on presenting it as one drive to Linux. So I ended up hardware mirroring on it - and my experience with doing that in hardware is that when one drive fails, both tend to fail, and either “disk” is unreadable on other controllers entirely.

Meh, it’s there, it’s on, I will regret it, or not, later. And it’s in the new nuc that doesn’t work, also. Maybe for a similar reason.

Tablets

I picked up a Best Buy “inspire plus” tablet with a high resolution display for 99 dollars, with android lollipop. It was wonderful for the 3 weeks I had it before I sat on it and broke it. A replacement for it had gone up to 149 dollars - and it was out of stock.

Honestly I liked the original nexus 7 best - the slight curve and the smaller size fit better in the hand - and I didn’t break the screen in 3 years of trying. I do not like the supersizing of tablets, 7 was “good enough”.

I also picked up a kindle to read on… which doesn’t have a backlight. It turned out I liked the backlight on my old kindle. Maybe I’ll figure out how to fix the old nexus 7.

Laptops

I still love the weight, battery life, and typing on the macbook air I have used as my main laptop for a few years now. If it wasn’t for the fact I do nothing in OSX except browse the web, libreoffice and aquamacs, and I find running linux in virtualbox maddening due to the differences in alternate key handling, I’d have never tried another laptop. My second choice for a good laptop remains the keyboard + touchpoint mouse equipped old-style lenovos, running linux, which I’ve been using for decades, now.

But they are soo heavy!

And: Lenovo foolishly discontinued touchpoint support last year, and went chicklet keyboard. (they have added touchpoint back into new models - which I didn’t even know they had resumed shipping as no store had any on display.)

So I went looking for new laptops. I must have fiddled with dozens, giving the new windows 10 ones a shot, also. I hated all the new chicklet keyboards out there. After long and careful deliberation I picked up a cheap chromebook and a puri.sm laptop.

I have to admit that I’d bought into the marketing claims behind pur-ism before I delved deeply into it, and had I realized how far from “pure” it was would probably have looked into something more air-like.

(All I really wanted was a laptop with an ath9k in it that ran Linux well)

I dislike the chromebook thus far also, but I suppose if I work at it, I might use it for road work, as it seems fast enough for web work. Of course, in my case, all I need it to do is run netperf over wifi, and I’ll be damned if I know how to go through all the rigamarole needed to get an old fashioned shell on chromeos, much less my vpn software, or other oddities of my own setups. I got it specifically, however, to play with the mediatek chipset in it. Mediatek seems to be an up and coming player on multiple fronts.

I have buyers remorse on the purism laptop. It’s heavier than the Air by a lot, doesn’t have a lit keyboard, the touchpad is hard to control, and I’m tempted to rip the pure-os off of it in favor of something I understand better. But I’ve only had it a few days, and I decided having OSX in front of me all the time was not the most productive thing when I needed to sit down and hack more than anything else.

GF has lust for the Air…

Ethernet over powerline

There is no way to run ethernet to my “office” (which is a repurposed closet) in SF, and wifi was too flaky (over 30APs I can now hear). So I use ethernet over powerline. These little Netgear AV/1200 converter boxes cost about 89 dollars a pair, and while they claim AV1200 - slightly more than ethernet speeds, they peak out at 200mbit down, 3mbit up in my environment.

They have horrible latency under load - over 90ms at 200mbit - random delays, and some weirdness now and then that totally disconnects them for seconds at a time - and I’m concerned about the “crypto” feature because it seems like they half work without it with IPv6 enabled.

I keep hoping the ethernet over powerline people catch on to fq_codel, it would be a much nicer and less annoying product if it had it on there. I wonder what OS is on the things? Anyway, they are faster than my internet downlink so I don’t notice the latency under load except when I’m doing a big transfer locally. I DO notice when things get flaky for mosh or ssh elsewhere and have yet to get to the bottom of that… and they are still way better than wifi for most work.

Wifi Mini-pcie cards

got an ath10k full length card, and an ath9k half length card - have plugged in neither yet.

Summary

Let’s see: pieces of totally non-working hardware - many. working testbed setup - don’t have. Days expended - more than I care to think about. Money expended - a tiny fraction of what stuff like this would have cost 2 years ago, and still, well over half wasted. Any given one of the embedded boards I got has sufficient complexity for what could have been a lifetime of enjoyment - I mean, I got something like 5 years of life out of my old cbm64 for 1200 bucks back in the day…

I’ve got good variety in wifi chipsets, with the exception of not having an iwl card to play with. Maybe I’ll get my old lenovo t400 back.

Still key to having a working setup is having a good router and a good delay box between my test driver boxes and test clients. Ah, well, back to trying to fix the new nuc. Or digging through the atoms in my storage unit.

In this orgy of post-christmas shopping, what did I like best? The Sennheiser HD 280 headphones, hooked up to my old nuc at 96khz, playing “Wish you were here”.

Find me elsewhere.

Best of the blog Uncle Bill's Helicopter - A speech I gave to ITT Tech - Chicken soup for engineers
Beating the Brand - A pathological exploration of how branding makes it hard to think straight
Inside the Internet Mind - trying to map the weather within the global supercomputer that consists of humans and google
Sex In Politics - If politicians spent more time pounding the flesh rather than pressing it, it would be a better world
Getting resources from space - An alternative to blowing money on mars using NEAs.
On the Columbia - Why I care about space
Authors I like:
Doc Searls
Jerry Pournelle
The Cubic Dog
David Brin
Charlie Stross
Eric Raymond
Anonymous
WikiLeaks
The Intercept
Chunky Mark
Brizzled
Dan Luu's rants about hardware design
Selenian Boondocks
Transterrestial Musings
Callahans

March 1, 2016
2287 words


Tags
hardware