The stopping point fallacy

  link   print

Saturday 6 December 2008 @13:15

This Geek Hero comic by Salvatore Iovene reminded me of a recommendation I made about programming a few years back. Even with the best documentation and planning, effective programmers must keep a lot of state and context in their heads. That’s why it’s so difficult to get back into the flow the next day, why even a 10-minute interruption can cost hours, and why — above all else — we can’t call it a day until we reach ‘a good stopping point’.

The trouble is, most people are terrible at predicting how long it will take to get to that stopping point. That gut feeling that we’re “really close to fixing this bug” (or finalizing that class, or finishing this refactoring) can last for 4 or 5 hours — until either we do finish it, or we determine for certain that it’s an order of magnitude more work than we thought. In grad school, my office-mate’s girlfriend once asked me what was going on when her beau said he’d be home in an hour, but three hours later she’d phone and he’d still be there. I’m sure many hacker widow(er)s are familiar with this scenario.

I don’t claim to have a solution to the problem of restoring context after an interruption; however, I have come to believe that insisting on a good stopping point is counterproductive.  Let’s say you get there: bugs fixed, test suites passed, clean code committed, and you go home happy.  I claim that the context problem will be even worse the next day.  You sit down, type ‘make’, and it builds cleanly.  You type ‘make test’ and it succeeds spectacularly.  You type ‘svn status’ and your working copy is clean.  Now what?  Meh.  Fire up the feed reader and ease into your day.

Imagine this alternative.  You’re implementing a new function you’ll need to fix a bug, but it’s getting late.  You hit the build button, ready to try it out, but instead you get 21 compiler errors.  Perfect — go home now. WHAT!?  I know, it pains the hacker’s brain even to imagine it.  That’s not a stopping point!  But think about tomorrow.  You sit down, build it, and suddenly you are pulled right back into the code, fixing the errors.  Sure, it will take you a bit longer without the immediate familiarity you had last night, but you’re already engaged in a simple, low-level activity that will help you restore context quickly.

Morals vs. Skillz

  link   print

Friday 21 December 2007 @14:43

A few months back, I was at a workshop on campus where some other staff member noticed my Mac and remarked (disapprovingly) on how quickly some hackers had gotten the latest OS X release running on stock (non-Apple) x86 hardware. “Why do people do that… why not just buy a Mac?” he asked. “They’re good computers.” I offered that I might like to have such skilled hackers as students (or employees). This seemed to surprise him, but I think it’s actually true.

For the moment, let’s set aside arguments supporting the morality of digital restrictions circumvention (legal or not). Just assume that the hack is ethically dubious. Assume also that it’s non-trivial. Then, I maintain that I would aim to recruit the hacker. Why? Ethics can be instilled, but raw technical talent is so rare that it’s still a net win.

New home PC

  link   print

Thursday 9 August 2007 @11:41

I think I was a tinkerer even before I encountered computer programming or computer science. And even though my research is pretty theoretical, I still enjoy breaking out the screwdrivers and anti-static wrist strap.

Our main home PC was a midrange Dell Optiplex that Art bought when he started med school in 2000. About two years later he switched to a Mac laptop and I converted the PC to a GNU/Linux workstation. We mainly use it as a home file and backup server, but it’s also my desktop when I’m working at home. Over the years I added substantially more disk capacity, bought a DVD writer, swiped a video card from an older machine for a dual-head display, etc. By this summer, I was itching for a more substantial upgrade. Software builds are pretty slow, firefox was struggling with memory limitations, and occasionally I had trouble helping out friends with USB backup drives because the system didn’t support USB2.

One of the great things about the PC architecture (as opposed to laptops and small form-factor consumer systems like the Mac mini and iMac) is that it’s entirely possible to upgrade it piecemeal. I had two high-capacity disks and an optical drive that were newer than the base system — no point in replacing those. So I went onto Newegg and did some research on the latest specifications. I generally don’t like buying the very latest stuff because the price/performance ratio is too high. The economical sweet spot on the curve is usually a generation or two back.

I went for the AMD Athlon 64-bit X2 dual-core processor. I got a compatible ASUS mini-ATX motherboard, 2G RAM, and a new mini-tower case. One thing I knew I needed out of the motherboard was two IDE (PATA) buses: one for the optical drive, another for the two legacy disks. The newer I/O bus is called SATA. The prices on drives are so good that I bought a 320G SATA disk too. The new PC will have well over half a terabyte of storage over the 3 disks.

Putting it all together went okay. I scraped my fingers to bleeding twice. The case seemed roomy — I chose it because having four hard drive bays was fairly rare for inexpensive cases — until I started putting the components inside. Before installing the disks, I booted an Ubuntu live CD to check that all the other components would work.

It turns out that three disks in a case this size is not ideal, even though it physically would hold four. After a few days of use, the disks were running hot. Really hot — the SMART temperature sensors reported 54°C (129°F)! Manufacturers are not always very precise about max operating temperatures. According to some numbers I was finding, this was on the high end, but not out of range. Also, the lifetime of the drive seems to depend more on the ambient case temperature, and ACPI was reporting 40°C on the motherboard.

Ultimately, I found a small fan in an older, unused system that I’ve been too lazy to take for recycling. I managed to secure it in between two of the drives and bring some air across them from the vents in the front of the case. Now the drive temperatures are in the range 45-47°C and reach 50°C only during heavy use (such as backups with rsync). I guess I’m satisfied with that, for now. But next time I think I will choose a case with more drive space, and with better front cooling facilities.

This is the first time I built a PC from the motherboard up, and overall, it has been a good experience.

And the livin’ is easy

  link   print

Tuesday 26 June 2007 @8:47

Hm, this space has been quiet for a while, but for justifiable reasons: I have two journal manuscripts submitted since the summer break began.

I’m never thrilled about writing for journals, because it often means that the key problem is already solved, and I usually would prefer to work on new problems than to “dot all the i’s” on old ones. On the other hand, it’s liberating to escape the strict space constraints of a conference paper. On the third hand, constraints are sometimes cited as catalysts for creativity. I’m reminded of the proverb “I wrote you a long letter because I didn’t have time to write a short one.”

I have also been ‘sharpening the saw’, also known as… Emacs hacking! Version 22.1 was finally released, and I took it as an opportunity to run through the manual and look for all the great little features and tweaks that have become available since the last time I studied the manual so intently. For example, just one thing that I adore for Java programming is glasses-mode (o^o). On-screen, it inserts some customizable little character in between LongCamelCaseWords so that you see them as Long·Camel·Case·Words. Ha!

Now I’d like to ‘sharpen my shell’ too. Zsh has lots of great stuff that I’m not currently using. I learned shell scripting in the early 90s on straight Bourne shell and tcsh, and only recently learned I could do concise parameter-frobbing things like ${file/foo/bar} rather than `echo $file | sed ’s/foo/bar/’` or whatever. Tab-completion for sub-commands (of svn, darcs, etc.) and host names (for ssh) would be great, and I know there are some directory-hopping features (beyond pushd/popd) that would help me. But one thing I’m grappling with is that I currently use zsh both in regular xterms and inside Emacs shell-mode. In the latter case, a lot of the fancy stuff in zsh won’t work. So do I avoid running shells inside Emacs, or hack shell-mode, or get term-mode working instead? Or, maybe forget zsh and do everything with eshell? Am I prepared to run always in Emacs, even when logged in to remote machines? I’m stuck.

Meanwhile, I cleaned up /usr/local/ on most of my machines. I try to avoid installing anything that’s not managed by apt, even if I have to backport it myself (such as with emacs22 on Debian etch). But sometimes it’s inevitable: either it’s something impossibly obscure, or I need a newer version than what’s available already, or it’s something I have hacked on myself and I need my version installed. So now what I do is keep a branch in /usr/local/src/, install it to /usr/local/stow/, and everything else in /usr/local/ is a symbolic link managed by GNU Stow. This should solve the problem of discovering some problematic file or library in local that I make-installed six years ago, and can’t remember what package it’s from or why it’s there.

Vitreousness

  link   print

Friday 16 March 2007 @10:27

I’m a latecomer to this fad, but here’s my entry into the transparent screen phenomenon that was all the rage. It’s far from perfect. But it’s true that the final image is not a composite; photoshop trickery is needed only to create the desktop background image. And even that was fairly simple-minded in my case, because I don’t know how to use any of the perspective tools in Gimp.

This reminds me: I can’t stand when someone observes an act of amateur creativity and says “he has too much time on his hands.” I guess it reflects the Protestant work ethic: idle hands are the devil’s tools and all that. But in my estimation, it tends to be said by people who spend much of their leisure time parked in front of the boob tube. Who are you to judge that reruns of Friends are more worthwhile than making an Earth sandwich? Personally, I want to live in a world where everyone has the time and inclination to pursue whatever creative outlets strike their fancy.

Bootstrap magic

  link   print

Tuesday 20 February 2007 @8:07

I’m giving a talk on Thursday for our CS club (a student ACM chapter). Our compilers course is hardly ever offered, because it ends up being a fairly arcane topic considering the career goals of the majority of our students. There are ways to make it more relevant of course, but I don’t want to argue either way on that today.

Instead, I decided to put together a fun little talk for the club on some of the ‘big ideas’ in the area. Here’s the abstract:

One of the more profound concepts in computer science is compiler bootstrapping: very often, the compiler for a programming language is written in that language itself. This begs an almost mystical question: what compiles the compiler? (And what compiled that compiler, and so on…) The first part of this talk is an adaptation of the famous Turing Award speech “Reflections on Trusting Trust” by Ken Thompson, co-inventor of the UNIX operating system. We explore the bootstrapping concept, and how to exploit it to devious ends. The second part is a very brief introduction to program analysis and compiler optimization, using static single assignment form.

I’ve been experimenting with some code to show that technique of teaching the compiler once, and then removing it from the source. Ken Thompson used the example of control codes, with a fragment of code that essentially said case '\n': return '\n'; but I found an enlightening post on the topic that cites this trick, from a Pascal compiler:

   insertSymbolConstantBinding("INTEGER_MAX", INTEGER_MAX)

The value of INTEGER_MAX was ‘taught’ to the compiler at some point in its evolution, but since the compiler compiles itself, it is no longer needed. Pray you don’t lose all the binaries!

I’d like to turn the talk into a screen-cast, just because I’d like to experiment with that as a pedagogical format, and (I think) I have the tools. We’re starting to see lots of ‘Web 2.0’ tool-builders publish video tutorials. I doubt I’ll record any audio/video directly during my talk, but rather use that as a trial run of the script. Watch this space!

GeekOS in VGA

  link   print

Saturday 20 January 2007 @16:11

The first week of classes seemed to go okay. I’m excited about exchanging code with students using Subversion. Not for group projects, mind you, but in place of downloading my support code and stubs, then emailing back their solutions.

Today I’ve been toying with GeekOS, trying to increase my understanding, and look for other projects to do with it. When I was taking and teaching OS at Maryland, I recall doing much lower-level projects: video and keyboard drivers, getting the task switching code to work, etc. Finally understanding the task switching mechanism was a big eureka moment, I believe. I kept trying to figure out how to jump to the next task in the queue, when really what you do is return to it. From the point of view of each task, it makes a call to the Yield() function, and then sometime later, returns from it. The whole trick is to do the context and stack switch and make the call from one task return to another.

Anyway, GeekOS seems to be designed for higher-level projects now, because the video, keyboard, and task-switching code is all in place from the start. But of course, that doesn’t prevent me from clearing it away, down to just the bootstrap code, and taking my students along on that journey.

Next week, I’m going to talk about PC video modes, and how to write characters and attributes (colors) to video memory. So today I started playing with basic VGA modes, the simplest standard one being 320×200 with 256 colors, also known as mode 13h. I imported some bitmap font code from the Linux console drivers, hacked on the screen.c driver a bit, and now I have GeekOS running in VGA:

That’s a 6×11 font, and seemed to be the best-looking choice with the limited resolution and non-square pixels of mode 13h. But it’s parameterized well enough that I can change a few definitions and employ other fonts stolen from Linux. Here’s a more typical 8×8:

Kind of deliciously chunky, and takes me back to those early days of the PC, when many graphical programs had text that looked about like this. Linux also has a 4×6 font, which is pretty much unreadable, but at least you get 80 columns:

(All of the screen-shots are shown here at 67%, so click through to see the actual size.)

A perverse level of virtualization

  link   print

Tuesday 16 January 2007 @18:36

So here I am installing a VNC client on my Powerbook so I can connect to my desktop Linux at work and control a VMware installation running Windows XP. And on that virtual XP? I’m running a GeekOS kernel on Bochs.

Just thought I’d share. Although I’d never personally choose XP for anything, I must admit that it has been convenient to be able to run it in VMware, just so I can see what kind of environment the majority of my students are using, and what problems they may run into. All the software I require for my courses is cross-platform, because I don’t want to be tied to anything. I even can cross-compile GeekOS on my PPC Mac and run it on Bochs there.

I managed to get a virtual XP running on my Linux desktop at work, but so far it doesn’t work at home… and that Linux machine at home is so underpowered at this point, I’m not sure I’d want it on there anyway. So using VNC to connect to it from elsewhere made sense.

More on GeekOS later, but so far hacking it is definitely fun. Learned more about segmentation registers on Intel this week than I ever needed to know.