As a project, a student wants to implement a parser for a small language in C/C++. I considered reaching back into my ancient experience with YACC/Bison, but instead we decided to try ANTLR. This is an LL(*) parser generator that produces a recursive-descent (top-down) implementation. YACC/Bison, in contrast, are LR (bottom-up).

Let me interject here that I never considered parsing a very interesting problem in language implementation. I'm open to arguments to the contrary; maybe it just isn't to my taste. As a programmer, I appreciate clean and regular syntax, but as a language implementer I see the appeal of Lisp's non-syntax – just give me the AST directly.

ANTLR is written in Java, but supports several other languages for its output. The documentation for these other languages is a bit sparse, so it may be best to learn its capabilities in Java and then port them. Ultimately, I was reading the antlr3.h files to figure out the capabilities of tree and string types.

ANTLR has some useful operators for structuring and rewriting the resulting AST, which is especially helpful given the restriction against left-recursive grammars. Some examples also use a separate specification for a tree grammar that interprets the output of the parser in some way. We definitely want a grammar that produces an AST, but for further processing we'll stick to straight C/C++ without needing the tool.

ExprCppTree.g

Here is the grammar file. One of the tricks for later post-processing is that you want to define as many named tokens as possible: replacing something like ('+'|'-') with (PLUS|MINUS) is beneficial because you can switch on the token type using case PLUS in the interpreter.

ExprCppTree.cpp

One other thing I ought to point out here is that I'm using the tilde ~ as the negation operator, as in SML. This avoids having to figure out conflicts between subtraction and negation.

Demo

Given the text file:

This program outputs:

Update

This works with ANTLR version 3.2 (debian stable), but I also learned today there are some significant C API changes in the latest, version 3.4. For one, antlr3AsciiFileStreamNew must be replaced with antlr3FileStreamNew(filename, ANTLR3_ENC_8BIT).

I have mixed feelings about the conviction of Dharun Ravi. Obviously the death of Tyler Clementi broke my heart, and Ravi’s actions were pretty vile. But it’s a mistake to presume a direct link between a few incidents of boorish badgering and a suicide.

I don’t think there’s any doubt that Ravi would not be in this circumstance if Clementi were still alive. So here’s the awful message the conviction sends to beleaguered kids: the way to get back at your bully is to throw yourself off a bridge.

Please don’t. Find a safe zone, call the Trevor Project. Find a way to stay with us. The world will learn to love you as you are, I promise. It gets better.

Every day on my way to the office, I pass this large animated display on Flatbush Avenue for the Brooklyn Academy of Music. It's a good illustration for computing novices of how pixels in an image are made up solely of red, green, and blue lights. On this sign, it's easy to see the subpixel configuration when you are up close:

      GRB
      BRG

Below it is another sign, with differently-shaped pixels, that I saw recently in Orlando. It has one blue light in the center, with two red and green lights at diagonals.

I decided to write a program I could use to demonstrate what any image would look like when rendered by the BAM sign. Each pixel in the input image is expanded into 4×3 pixels in the output image, including a black margin between each group of subpixels.

You can see the result to the right. Bart's eyes still look relatively white, but if you look close or zoom in, they're made up only of red-green-blue dots. Below is the code, using the Magick++ library. See also the subpixelize repository on github.

A recent email exchange with a curious student that brought back memories. Posted without further comment.

Subject: Zork
Are you fimilar with this text based video game of the 80s? If so can you explain to me how it was made, programming-wise?

OMG definitely. Spent many hours solving Zork 2 and/or 3, many of them with my high school buddy in the office of our chemistry teacher, who was also a fan.

I really don't know much about how they were programmed, although I did try to reproduce some of the basic ideas on my own. There were some listings in magazines of programs in BASIC that implemented simpler text adventures, I might have just modified those.

Here's a reference to one from Wikipedia:

Pirate Adventure, was an original game in a similar style to Adventure—its source code, written in BASIC, was published in the December 1980 issue of Byte magazine.[41][44]

but I can't find the article online.

I almost made my peace with Blackboard for the upcoming semester. I recently discovered that they support WebDAV for transferring files in and out of the ‘content collection’. Although WebDAV largely sucks, this was a revelation. It meant I could use real tools like rsync instead of clickety-clickety-uploading files one at a time. I pictured using Blackboard just as a shell, with its discussion fora and gradebook, but with most of my content (notes, assignments) in uploaded HTML. I don't want that kind of content managed directly by Blackboard, which makes it way too hard. There are plenty of great tools to generate HTML from simpler sources – with syntax highlighting for code – such as pandoc.

Of course, while experimenting today I found not one but two showstopping misfeatures. The first is that, once in a while, Blackboard decided not to display the HTML content within its frame, but rather to download it. I couldn't for the life of me figure out on what it based this decision. In one case I opened an editor over WebDAV, added one line, and thereafter it would switch from download to inline display.

The second problem was that, after an rsync, Blackboard would decide that, because the file I had linked into the content area had changed, it wouldn't link to it anymore, and instead display the title with the message “(Invalid File)”. Helpful. Clickety-clickety to fix them all again.

(It's conceivable that some of these issues are due partly to buggy WebDAV on either end, but I'm unwilling to investigate too far just to exonerate Blackboard.)

So I'm back to linking to the course web site (which I'll probably build with yst) on an external server. I think what I most need in an LMS is for it to be a (very) thin layer on top of static HTML that can be generated however I want. I'm inspired by how easily you can add something like Disqus to static HTML. An LMS could just be an integrated collection of similar tools to provide discussions, gradebooks, assignment drop boxes, student profiles, etc.

I started to hack on something like that, but I'm not happy enough with it to use it for Fall 2011. (And if I resume hacking on it, I'll probably start over and scrap Python.)

I switched back to GNU/Linux full-time (Ubuntu on ThinkPad x201) almost a year ago, but it was just this week that I found the will to meander away from the default Gnome 2 desktop and window manager.

In the world of X-windows, the way your desktop looks and behaves is controlled by an independent program called a window manager, and there are many such programs. I burned my way through several in the old days before Mac OS X. The first one I remember explicitly was twm, the default on our DEC Ultrix setup at the time.

A persistent, though minority, approach in some window managers has been tiling. This idea abandons the typical (and once revolutionary) desktop metaphor of overlapping windows, like pages on a desk. Instead, your windows fall into particular patterns called layouts, covering whatever screen space is available.

The upshot is that – in most cases, with many types of apps – you don't have to think much about moving and resizing windows to see what you want. You also use the mouse a lot less.

I tried a few tiling WMs over the years in my pre-Mac Linux days, but none ever stuck for more than a day. They were hard to configure and, at times, clunky and awkward to use. Recently, though, I had been hearing about Xmonad, a window manager written in and configurable with – get this – Haskell!

Having a full-blown programming language in which to configure things makes a huge difference. Previous WMs required heavily repetitive but unrefactorable configs. (I guess I'm lucky that I knew Haskell already – it can be somewhat impenetrable to the uninitiated.) What's really remarkable about writing your Xmonad config in Haskell is that it's a statically-typed, compiled language. Most configuration languages are dynamic, loosely-typed: Emacs Lisp, Guile, Lua. And yet Xmonad can reload your config at any time, seamlessly. (It's fast, and doesn't lose state.) Now I'm having fantasies that one day we could have an Emacs configured in Haskell. Blasphemy?

My Xmonad configuration is available on github if anyone is interested, along with other ‘dot files’. I did end up writing my own layout. I wanted one with a centered master window, big enough for 960-pixel web pages, and tiled windows along the side margins that the master actually overlaps. The side windows can be brought forward when focused, without having to swap them with master. It's kind of a mashup of the CenteredMaster and the Circle layouts in contrib.

If you'd like to explore tiling and can spend some time on configuration, I definitely recommend Xmonad!

This morning I whipped up my first (tiny) hand-coded HTML5 app, using the canvas tag for 2D drawing. The first unit of my CS0 course is about binary representations of data, including numbers, text, and graphics. The graphics are of course the most fun — see also hex nostalgia.

There’s no shortage of color-choosing widgets on the web, but I wanted one that would clearly illustrate the additive color model used by computer video displays and projectors. Most of the students are not media arts majors, and so they’re still attached to the subtractive primary colors they learned in elementary school. Despite their familiarity with displays that rely on it, “red + green = yellow” is not natural to them.

With the HTML5 canvas, additive color is a simple matter of:

Once I got it working with the three 8-bit RGB sliders, I decided to have some fun with 4-bit color — call it CGA nostalgia. I recall my early days with PC BASIC (did color on TI and Apple work this way too?) when I first understood the pattern to the mapping of colors to the numbers 0–15.

Intarweb, je vous présente: 4-bit color, 24-bit color. Feel free to view source.

Every semester I have about 22 brand new students in an intro course. I try pretty hard to learn their names quickly, without sacrificing significant class time to the endeavor. There are always some I get right away (the ones that are outspoken from the first lecture), and others that take weeks (usually the ones that tend to be quiet and/or absent).

Our classrooms are amazingly diverse, and I truly enjoy that. It means I personally have a hand in broadening participation in computing, to use the NSF phrase. Plus it’s just darned interesting to work with people with varied backgrounds and stories.

To learn names, I notice myself doing something that I believe is perfectly harmless, natural, and useful, but could seem somewhat dubious from the outside. That is, I cluster the names and faces by perceived ethnicity, race, and gender. It means I don’t really have to match 22 names to 22 faces; I just have to distinguish names within the perceived cluster. Not every face is an equally likely candidate for the name Mahesh or Mahmud, Matsumoto, Miguel, or… Alice.

The embarrassing consequence, however, is that as I’m still learning, my mistakes appear to be indicative of my hillbilly heritage, namely, “durr… all y’all [ETHNIC] folks look alike!” By week three, I’m specifically focusing on distinguishing between (for example) the three quiet Indian women or the two absent Turkish guys.

I don’t believe there are any clusters for which I’m particularly better or worse at learning names, though I could be wrong. There probably are clusters for which I can make finer distinctions than others: Bangladeshis vs. Pakistanis vs. Hindu South Indians; or Chinese, Japanese, Thai (certainly by name, sometimes by face or dress). But I’m useless at the varieties of Caribbean Islanders or Eastern Europeans.

Also, once in a while, my perception doesn’t match reality, such as when I guess that Bob Smith is Angelo Santos. But as a result of such a mistake, I manage to learn Bob’s and Angelo’s names indelibly! So perhaps intentionally making cross-ethnic mistakes would be an even better learning strategy.

In general, I’m a believer in what is known in software development as “eating your own dog food.” That is, you use the product you create, as you create it. A project to build project management software is managed using that software. Bugs in the bug tracking system are tracked in that system. A new distributed version control system is eventually hosted in its own repository. And so on.

The principle is most obvious, of course, for tools related to software development, but conceivably it can be pushed further, as long as the developer is a member of the target audience for the system.

Recently I began thinking about ‘dogfooding’ in the context of my primary responsibility: not software development, but teaching. I noticed a troubling inconsistency. I truly love learning; my life is structured so that I can explore and experiment with new ideas and techniques just about every day. However, I no longer have any interest in learning in the context of a formal course. It seems I don’t like the taste of my own medicine!

This isn’t even about the much-maligned lecture format or the traditional classroom. I’m happy to attend independent lectures about topics that interest me. And a flexible environment doesn’t make a course any more tolerable for me as a student. I think I just rebel at the prospect of following someone else’s syllabus.

Maybe this, as much as anything, is what a Ph.D. is about. It signifies a dedication to life-long learning, but also endows the scholar with the ability (and responsibility) to construct her own curriculum. Can one stand on the shoulders of giants without having to sit through their classes?

Naturally, I wouldn’t wish heart disease on anyone, but it is kind of nice to read accounts of others going through the same thing at approximately the same time. Especially when they have the wit and wherewithal of biology professor and noted skeptic blogger PZ Myers. His posts on My lost weekend and A fistful of stents were familiar echoes. The latter one, about his PTCA, is hilarious.

I don’t believe I was nearly that out of it during mine. Sure there’s some kind of relaxing agent in that IV (not bothering to look up what exactly it was), but I recall talking lucidly to the interventionist before, during, and after. And watching the radiography monitors as best I could, from a poor angle. I’m pretty sure I didn’t nap.

Of course, it’s super annoying that I’m not just twenty-some days ahead of PZ on this matter, but more like 16 years — he’s 53, I’m 37. And this wasn’t my first time. The good news is that, 42 days out, I still feel great, I’m adjusting dietary habits effectively, and I’m shedding weight steadily. You never know what’s going to kill you, or when. If there’s one positive spin on chronic disease, it provides a clear, likely threat and thus something to manage.

Nothing is built on stone; all is built on sand, but we must build as if the sand were stone.
— Jorge Luis Borges

P.S., unlike PZ, I like cabbage. But I’m terrified of CABG.

Yeah, so I spent some time in the hospital this summer. I’m not going to get into the gory details at the moment, but suffice to say that I’ve known I had premature heart disease for some time, and on 20th July, it struck again. For the moment, I’m feeling pretty much normal. I’m on a bunch of new medication. I’m walking every day. I’ll know a lot more after 3 months out (around Hallowe’en). For now I want to focus on one aspect of the aftermath: food. And because I’m such a nerd, I have plenty of data, and charts!

We love food. The tremendous number, quality, and variety of restaurants in NYC is really one of the top perks of living here, just as the average apartment kitchen size is not! So although I enjoy cooking, particularly for special occasions, lately we had been eating dinners out almost every day.

Thanks to my obsessive use of ledger, I can compute average dining and grocery expenditures over the past several months. In the graph below, the blue line indicates restaurant expenses per day, averaged over the preceding seven days. The red line is the same computation for grocery expenses. (For any chromatically challenged readers, the blue line is the one on top, for most of the graph.) You can clearly see the inverse relationship in several places; when there is a sharp drop in restaurant expenses, there is usually a corresponding hump in grocery. (The large blue spike corresponds to a several-day getaway at the Borgata Water Club, Atlantic City.)

The gray rectangle on the X axis indicates the days I spent in the hospital. Something remarkable happens after that: restaurant expenses drop to nearly zero, and grocery stays somewhat high. In short, I’ve been cooking up a storm since being home! I started working through the New Mayo Clinic Cookbook, which a good friend gave me several years back. I like that it has complete nutrition facts for every recipe.

While munching on hospital food, I came to think of restaurant food as inherently unhealthy. Even if you make reasonable choices regarding cuisine and dishes – which, for the most part, I thought I did – restaurant food is always going to be saltier and fattier than what I would prepare at home. I imagine I’ll eventually find some balance, where I won’t see restaurant food as poison, but rather something to be enjoyed in strict moderation; i.e., not six nights a week.

I also started tracking my weight, and everything I consumed, using the LoseIt iPhone app. It actually doesn’t seem that hard to drop some weight just by honestly counting calories; the awareness alone helps with willpower, and seeing some results is motivating too. Below is a graph of my daily weight measurements (blue diamonds) and trend (red line) since hospital. It was produced with the Hacker’s Diet online tools. (LoseIt should incorporate the trend computation too – it’s very helpful.)

I started the trend at two pounds heavier than the first sample. That was the last measurement I took before hospital, and having the graph reflect that there had already been some loss there was more motivating! Of course, this is just the beginning. I’d like to periodically post these graphs here or on Flickr, to put the evidence of my progress out there in public.

Monadologie: Professional Help for Type Anxiety from Nathan Hamblen on Vimeo.

The code is available at github.com/league/scala-type-anxiety

I believe I’m a reasonably objective grader, but if I wasn’t, would I know? Most everyone thinks they’re an above-average driver too.

It helps that, in my field, many responses are right or wrong; programs work or they don’t. Of course, I award partial credit for programs that partly work, or responses that are “on the right track,” so then it becomes fuzzy. I often use what one could (charitably) call a rubric, to help keep track of criteria and standards. But still, is it fair?

I’d like to think so, but one thing raises alarms. I notice myself doing it from time to time. In the midst of grading an exam, I’ll encounter a response so mystifying (or so brilliant – it can happen!) that I’ll flip to the front page as I ask, “who is this?” Does the score I award depend at all on the answer to that query?

Maybe in the future I will try blinding exams and significant assignments in some course, to deny myself the answer.

So, in the midst of finals, I decided to install the new Ubuntu 10.04 GNU/Linux OS on my MacBook Pro. A wellspring for procrastination, no doubt, but also I was again becoming concerned about my dependence on proprietary software. Combine that general orientation with some of Apple's recent policies and I was looking to escape – or, at least, to map out a possible escape route.

The Ubuntu Apple/Intel installation directions were very helpful, although things didn't go exactly as advertised. First, neither of the Mac tools (Disk Utility or Boot Camp) were able to shrink my partition. The got about halfway and then gave up and said it couldn't be done. That means I'd have to nuke and reinstall MacOS to make this work. It's a testament to my resolve – and my trust in rdiff-backup – that I did just that.

So after reinstalling Snow Leopard on a shrunken partition, installing rEFIt as the initial boot loader, the Ubuntu install went smoothly. The only other thing that went wrong was that grub (the secondary boot loader) did not configure itself properly. Fortunately, grub has a command line and it's possible to guess the right incantations to get it to boot anyway.

It has been years since I used any sort of GNU/Linux desktop. From 1996 to 2005, my office machine was always Linux – at three different offices. In 2001 I bought my first Mac laptop (an iBook) but I did not start using Mac exclusively (or nearly so) until 2005, with my beloved 12” PowerBook. Even then, my two most well-worn apps are definitely emacs and zsh.

Anyway, a lot evolves in five years, especially thanks to the hard-working and well-funded folks at Ubuntu/Canonical. It has been a learning experience, but mostly positive. I love having all software managed by apt, once again. Still think the Mac platform could benefit from this kind of an app repository — as long as unsanctioned sources are permitted. The Mac apps that may be the hardest to replace are: Keynote (OpenOffice Presentations are fine compared to Powerpoint, but Keynote is a cut above), Skim and BibDesk (PDF annotations and bibliography tracking that work well together — both open source, but reliant on Mac GUI), and ScreenFlow (really great for capturing and editing screencasts).

The first thing I felt I had to get sorted, though, was the keyboard! Mac apps specialize in the so-called ‘command’ key (⌘), with control and option as distant cousins. Linux/Gnome apps prefer the control key, with ‘alt’ as a close second. So I wanted two control keys, and in better positions than the existing one. Below is the layout I came up with. It has left and right control, left and right alt, a ‘super’ [win] key (which is great for mapping window and desktop management functionality so it doesn't conflict with emacs) and the ‘multi’ key (aka compose character) for typing accented and other “§peçìâł” characters.

I did not rearrange the alphabetic key-caps as shown in the picture (by doommeer on Flickr, used with permission – CC:by-nc-sa) but I do use the layout shown. I've been typing Dvorak every day since 1994 or so.

Below is the Xmodmap configuration that achieves this modifier-key layout on the MacBook Pro. X key configuration is a little wonky, to say the least. The key codes I explicitly unmap here (6c, cc, etc.) seemed to interfere with the ones I wanted; there can be only so many syms for each modifier. This still isn't perfect; the right control key auto-repeats unless I xset it not to, and I have to semi-manually switch to a different configuration (via a .Xmodmap symlink) when I dock the laptop with my Kinesis QD contoured keyboard.

More later on other applications and tricks, but for now I'm pretty happy. The machine dual-boots just fine with rEFIt, and although I have all my stuff (except for music – too big for a partitioned drive) synced on both sides, when that chooser screen pops up, I always go with the penguin.

So we’ve started more seriously looking into the housing market in Queens, New York, the most diverse county in the nation. So far, it’s mainly about orienting ourselves to the styles, conditions, and neighborhoods here, as we struggle with the concept of becoming slaves to that mortgage payment.

One thing I’ve noticed wandering around is that you can pass along the perimeter of a neighborhood thousands of times without ever noticing the treasures (and, to be fair, the dumps) within. Partly this is just because of geometry. The number of lots, blocks, and houses is proportional to the square of the length of the edge. To visit every meter of curb makes for a considerable journey!

To honor the quadratic, I illustrated this post with two square-cropped photos – though neither from New York!

Applications must be originally written in Objective-C, C, C++, or JavaScript as executed by the iPhone OS WebKit engine, and only code written in C, C++, and Objective-C may compile and directly link against the Documented APIs.

Not good news for programming language enthusiasts. Presumably the purpose of this measure is to combat Adobe. I have no love for Adobe either these days, but as a software developer I don't want to be a pawn in some corporate feud. Why should Apple care what languages and tools we use? Publish the API spec and we'll follow it.

Annoying developers is no way to promote your platform. There's a certain class of developers that will build for whatever platforms are dominant – in order to reach the most users. There's another class that will primarily target platforms they themselves use and love. Which class produces the most compelling, innovative, game-changing applications? We leave that as an exercise for the reader.

Here's an experiment you can try at home, in your favorite editor, browser text box, etc.

1. Type “one”
2. Do some backward cursor motion – this forces a separate action in the undo list
3. Type “two”
4. More cursor motion
5. Hit undo, to remove “two”
6. Type “three”
7. Fiddle in vain for a combination of undo/redo that will bring back “two”
8. Curse redo as the most useless concept in user interface design – evar!!!
9. Start searching for ways to do every last bit of text editing in emacs.

The design of the site is my own hand-coded CSS creation, but based on the very helpful 960 grid system. I'm very curious to hear feedback. I'm pretty confident in my coding ability, but less so with design.

It seems like I go through this relaunch process every couple of years. Each time the goal is to make it easier to maintain. But then within two years I declare it unmaintainable and try something different. Either I'm not good at making something that is actually maintainable, or else I change my mind every couple of years about what that concept means. ;)

When I wrote last October in defense of strong AI, I promised to take on the ‘uncharitably narrow’ notion of algorithmic used by Ian Tattersall to argue that machines will never achieve consciousness.

In The Monkey in the Mirror (2002), he writes “Computers by their nature are algorithmic, rapidly applying a fixed set of rules to the solution of well-defined questions” [page 71]. He refers here to IBM's Deep Blue playing chess, the canonical example of a machine accomplishment that, while impressive, is surely not conscious. From this example, he extrapolates:

Human consciousness is very clearly non-algorithmic. Even if we have to conclude [...] that subjective awareness in ourselves and our relatives is due ultimately to a mass of electrochemical events in the brain [...] it is equally clearly not the product of a mechanism that dutifully clicks through a listing of tasks and ‘if-then’ choices.

It doesn't take very much code to produce a system whose precise behavior is practically unpredictable, even by the person who wrote it. We may use strategies such as functional decomposition to design a system to target specific requirements, but as the system evolves complex behaviors emerge. If they didn't then debugging would be easy.

Yes, ultimately any deterministic program is expressed as a list of simple instructions and branches – millions of them, coupled with sophisticated data structures in an infinite number of configurations! Tattersall makes it sound like little more than a flowchart you could hang on the wall.

I don't claim that our current programs are conscious, or intelligent in any strong sense. But don't rely on a naïve notion of algorithmic to argue that machines cannot be conscious.

Growl comes with a script, ‘growlnotify’, that can trigger notifications from the command-line. You could, for example, type something like ‘make; growlnotify done’ – the problem is that I never remember that I would like the notification until after the command is already running.

So here is an alternative: this bit of zsh code will notify me upon completion of any job that takes longer than 5 seconds.

Note that this is specific to zsh, which is not the default shell on OS X, although it is supplied with the system. Possibly it can be made to work with other shells; I really don't care. :)

Ideas for improvements: filter out commands that are obviously going to be long, interactive jobs, such as ssh sessions, vi, command-line emacs, etc. Also, is there some way not to rely on ‘/bin/date’?