When is a programming feature 'helpful'?

There was a kerfuffle in my twitter feed this morning which highlighted some issues about language design. Don Stewart retweeted this conversation about funny JavaScript behavior. Reproduced in a JavaScript  console, it looks like this:

    > ['10','10','10','10','10'].map(parseInt)
    [10, NaN, 2, 3, 4]

It looks completely mysterious to a Haskeller, and by "mysterious" I mean "punishable by life in prison." I'd bet there's many a functional head listing on a sprained neck out there in Haskell-land,and that co-functional parents are explaining to their children what adult words are. I set about looking for an explanation, because I could not imagine how map could behave this way. I got the exact same answer in my browser, which made it unlikely that it was due to uninitialized memory. So first I tried using the identity function:

    > ['10','10','10','10','10'].map(function(x) {x;})
    [undefined, undefined, undefined, undefined, undefined]

Oh, right, JavaScript requires a return:

    > ['10','10','10','10','10'].map(function(x) {return x;})
    ["10", "10", "10", "10", "10"]

Great! Sensible behavior. Next, I looked up parseInt and found that it takes two arguments: parseInt(string, radix). JavaScript code is full of helpful functions that take variable numbers of arguments and guess what you mean based on their type, so this seemed a likely candidate for trouble.

    > ['10','10','10','10','10'].map(function(x) {return parseInt(x,10);} );
    [10, 10, 10, 10, 10]

Much better. Presumably then, the broken version supplies a second argument to parseInt that it pulled out of its ass. The tail of ..2,3,4] made me suspect there was some accumulation happening, so I tried this:

    > [1,2,3,4,5].map(function(x,y) {return (x+y);} )
    [1, 3, 5, 7, 9]

Prison's too good for them, I thought. Apparently this sums adjacent elements, except...wait, there would have to be a zero prepended for that to work. I observed that addition being commutative, it was projecting out valuable information, so I tried something a little more revealing:

    > [1,2,3,4,5].map(function(x,y) {return ('('+x+','+y+')');} )
    ["(1,0)", "(2,1)", "(3,2)", "(4,3)", "(5,4)"]

Still looks like adjacent sums with a magical zero. At this point I had some coffee and realized I was confused about the types. I had two lists of small integers, one from a mysterious source, not one list being zipped with itself.

    > [1,2,3,4,5].map(function(x) {return x*10;}).map(function(x,y) {return ('('+x+','+y+')');} )
    ["(10,0)", "(20,1)", "(30,2)", "(40,3)", "(50,4)"]

Okay. The second argument is clearly not a function of the array contents and now looks like the array index of the current argument. Going back to the original problem:

    > parseInt('10',0)
    10
    > parseInt('10',1)
    NaN
    > parseInt('10',2)
    2

Great, this sort of makes sense now. A radix of zero is assumed to be 10, a very C-like idiom, and the second example fails correctly because '10' is not a unary number. So, parseInt is not the villain, nor really is map. It is perfectly reasonable to have a variant for arrays that would operate on an element and its index. For me, the villain here is the handling of optional arguments. The map function is overloaded silently and that's the kind of help I don't want.

'Helpful' means different things to different people. To a Haskeller, the definition is something like: publish your promises and keep them. Explicit type signatures, referential transparency, static type-checking are all there to make those promises come true. In JavaScript, optional arguments are considered helpful, because they let you reuse the same function name in slightly different contexts.

Steadily, monotonically, over the years, I have moved away from convenience in the specification of a problem whenever it introduces uncertainty. When I first learned C, I memorized the precedence of operators so that I could write expressions with a minimum number of parentheses. Then I learned other things and forgot the precedences and just started parenthesizing everything until it was unambiguous. A friend in graduate school used Modula-3, which had no automatic casting between numeric types. C programmers were horrified, but I felt it was a breath of fresh air.

These conveniences should be part of interactive programming environments, not programming language specifications. Fuzziness, forgetfulness, disorder are part of the human condition and should be acknowledged, but they should not make it into the code. Say what you mean to the programming environment, by all means, but have the resulting program be completely unambiguous.

Which reminds me, the JavaScript map actually accepts a function of three arguments...

GHC Talks to the Scarecrow

I have a strange sense of humor. It's essential that you know that before reading on.  'Cause really, this is a pretty dry little post about an obscure error in a Haskell program.

Today, I got a message from GHC that always amuses me. Why?  Because the message makes feel like GHC is poor little Dorothy, stuck at the fork in the road, confused by the Scarecrow's advice that some people go both ways.

@ConalElliott re: What resources & practices (teaching Haskell)

Conal Elliott asks: What resources & practices would you recommend for helping a group of Java & C++ programmers learn to work in Haskell? I saw the question first on Twitter, but my compression skills were not up to the task of answering there.

I have two recommendations: teach them the simplest definitions of the fundamentals; read programs with them, out loud, like children's books, skipping nothing.

These steps are often passed hurriedly, by teachers so steeped in a subject they don't realize how different their worlds are, by students eager to build things as quickly in a new programming language as they do in the old. Most likely, the students will assume that they just need to learn new syntax to be off and running. Indeed, I am sure they are capable, self-motivated software professionals and that they would succeed in learning Haskell, just without the deeper understanding that can make it such a joy.

As a test, ask your students to write down the definition of 'type'. I'll bet that their answers will be longer than 'set' and include some mention of bytes, words and big-endian. We were all trained to be mechanics instead of drivers, because all we had were go-carts.

C and its children will have left clutter in their minds. Types will be obstacles, efficiency a drug, correctness assured by machismo. They will gloss over code without really reading it, trying to fit it into an existing model. Saying things out loud helps slow things down, forces them to make connections between symbols, words and definitions so fundamental they are rarely written down. (I backtracked through Paul Hudak's book for hours trying to find out how to say '::'; newcomers ask pretty regularly in #haskell.)

Make sure they know all the places that patterns can appear in code and how they work. Make sure they know that data constructors are functions. Have them verify that with :type in GHCi. And that type constructors are functions, verifying with :kind. They should be familiar with the syntax and semantics of higher-order types before they meet IO, so that they understand it is not magic.

Lists, recursion, functional algorithms, none of these will be challenges, but constructs they have not seen before will give them much more trouble if they cannot deconstruct them to primitives. I cannot imagine Conal omitting such fundamental semantics from his lessons, but I can easily imagine both teacher and students, excited to build new stuff, skimping on the exercises necessary to flush out the old foundations and cement the new.

Functional Programming and the Universe

In Lee Smolin's TED talk How science is like democracy, he describes how our understanding of space and time has evolved. He describes three stages of cosmology:

1. Aristotelian, which is hierarchical and all properties are defined with respect to that hierarchy;
2. Newtonian, where properties are all defined with respect to an eternal absolute background of space and time;
3. Quantum Theory or Relational Universe, where the universe is nothing but an ever evolving network of relationships, and all properties are about relations between subsystems.
   

Two quotes from the talk that jumped out at me:

According to Newton, space is just the way that god knows where everything is.

And this one, regarding the modern relational model of the universe:

Space is just one aspect, so there's no meaning to say absolutely where something is, there's only where it is relative to everything else that is, so we call it a relational universe.

In other words, even the physicists don't use an object-oriented model to describe objects any more.

Two-dimensional zip

I've been doing battle with things like CSS recently (the W3C will be first against the wall when the revolution comes,) so it was a pleasure to run into a little Haskell puzzle and an elegant solution.

I was working with a matrix represented with a list of lists: [[a]]. I wanted to add the row and column indices to each element, giving: [[((Integer,Integer),a)]].

This isn't a pattern I remember seeing before, but it seemed simple enough, so I just hacked away. Here's my first try:

> index grid = zipWith indexrow [0..] grid
>     where indexrow rn rs = zipWith (\cn r -> ((rn,cn),r)) [0..] rs

Serviceable, but neither pretty nor general. The one dimensional case is so elegant; for any list, you can add indices with:

> index1d xs = zip [0..] xs

The zip function is simple and the infinite list of indices covers all cases. So it occurred to me I was looking for two things: a two-dimensional version of zip; and a grid of index tuples, semi-infinite in both directions.

Both turned out to be simple enough. Here's the grid of tuples using list comprehensions:

> indices2d = [[(r,c) | c <- [0..]] | r <- [0..]]

And the two-dimensional zip:

> zip2d = zipWith zip

And it's easy to reproduce the "With" version:

> zip2dWith f = zipWith (zipWith f)

So now my function looks like this:

> index2d grid = zip2d indices2d grid

or just:

> index2d' = zip2d indices2d

This version is clearer, simpler and easily generalizable to higher dimensions.

Eschewing flip

I realized recently that I've developed a small dislike of the Haskell function flip, which reverses the order of arguments of a binary function:

mod 3 5 ==> 3
flip mod 3 5 ==> 2

flip is generally used in place of a lambda expression, when constructing a specialized function:

map (\x -> mod x 17) [1..] -- with lambda
map (flip mod 17) [1..] -- with flip

Unfortunately, my brain really doesn't like that reversal. When reading any non-trivial expression using flip, I have to stop and think about which argument is now where.

I've found that I prefer sections. You need to have one of the arguments handy, but I find that's often the case:

map (`mod` 17) [1..] -- with section

I like this form better because it preserves the visual ordering of the arguments, which seems to be deeply seated in my reasoning process. Because of Haskell's flexibility with switching between prefix and infix application, it's easy for me to adopt a style to suit this preference.

Rewinding the Mind

Jeremy Shaw sent me this bit about a DVD rewinder. Very humorous, of course, but there's also a nice little moral about programming in there.

When I was at Linspire, we often got questions from people looking to switch to Linux for the first time: How do I defrag my disk? Where do I buy anti-virus software?

We would explain that Linux filesystems didn't need defragmenting and that the virus problem was virtually non-existent. But sometimes, there was just no reassuring them. IT people sometimes said they simply had to have virus protection software, bought and paid for, because the company had made it mandatory for all computers. We felt a little silly, but we licensed one, shipped it and then they switched.

I wonder how many people learning a functional language wondered how to allocate and deallocate memory like they did in C? Whether the answer is obvious or not depends on one's understanding of the semantics of the language. If you have a model of C semantics in your head and you're looking at Haskell code, you're predictions about what's going to happen are bound to be wrong.

All these are examples of essentially the same problem. If we can't see (or otherwise sense) how a technology works, then a change in the underlying mechanism will leave us confused because our mental models are out of date.

I tend to prefer designs that show a technology at work, such as record players as opposed to most CD or DVD players. Unfortunately, miniaturization and digitalization only make the problem worse. I suppose that's why Hollywood always has been at the forefront of computer graphics, because the audience has no idea what's going on without all the flashing lights.

I will pursue these ideas further in some upcoming posts about functional programming and user interface design.

The Programming Grapevine

Two non-Haskell programmers from Linspire went indoor rock climbing last night. One was wearing a Firefox t-shirt and someone next to him asked if he worked for Mozilla. He explained that he actually worked at Linspire, to which the guy said, "Oh, you guys use Haskell there, don't you?"

It's nice to know that the word has gotten around.

Well, David Fox and I left last summer and Jeremy Shaw has one foot out the door, so the Haskell presence at Linspire is just about over, but we continue to use and maintain many of the tools and libraries we built there. David and Jeremy have been cleaning up the version number policy in the Debian package builder (called the autobuilder) so that we can build Debian repositories of Haskell software. There is still some work to do, but you can look at our darcs repositories to see if there is anything that might interest you.

To see what we're up to now, see Section 7.1.6 of the Haskell Communities and Activities Report.

Mebbe

My friend David has always had a knack for brevity and wit, the most recent example being this snippet of Haskell:

mebbe x = maybe x id

Not as clear as fromMaybe, perhaps, but much funnier.

Slouching Towards Bethlehem

After five years of discussions about programming languages, there are signs that my friend Joe has started to grok the benefits of functional programming. He's listened to me talk about it and agreed that it sounds good, but he only recently did some reading that really got the message across. Read how a book on Erlang got him excited about FP.

Last weekend I went tide-pooling with a friend and two of her kids. Half way there, her son Will (7) noticed that we all had coats and complained that his mom hadn't told him to bring one. She said, "I did tell you, but only twice. I have to repeat things five times before you hear me."

We all have trouble absorbing information that seems counter to our experience. In Will's case, he couldn't imagine the cold, wet ocean breeze from the comfort of his living room. For Joe, the functional approach was just so foreign, especially given that he was solving problems so well with imperative languages, that he could never really feel any benefit.

At CUFP this year, many people speculated that the need to exploit multi-core processors would be a motivating factor for people to move towards functional programming. (Intel is betting on it.) This proved true in Joe's case. He's currently doing server work in Ruby and he mentioned a few weeks ago that he'd been running into limitations of the Rails/DB design. Reading about the ease of threaded programming in Erlang suddenly made things click.

Patience and persistence pay off. People will hear when they are listening, understand when they have the need, switch when they can no longer contain their excitement.

Data: Singular or Plural?

My brother Sam was visiting recently and corrected me when I used the word "data" as a collective singular rather than the plural. He made some huffy comment about the difference being important to scientists. Back to that in a moment.

A few days ago I discovered that professional wrestling was older than I thought while watching a film from the 1940's. Well today I discovered that the use of data as a singular noun dates from at least that far back as well. This time my source is "The Philadelphia Story," also filmed in the 1940's. Jimmy Stewart, playing an author working as a journalist, says, "Our research department didn't give us much data."

I was about to send a snide note to Sam, when I started to wonder about the 'data' statement in Haskell, used to construct new types. There is also a 'type' statement and a 'newtype', both apparently singular, but then, each constructs exactly one type. So I have to wonder, is "data" in this context also singular?

If Haskell used "datum" instead, and there was more than one element in the type, it wouldn't make sense, I suppose. Of course, no sane language designer would force you to use 'datum Singular = One', but 'data Plural = One | Two', would he? Details like that are important in the semantics of Haskell, though perhaps not the syntax. Still, perhaps I should be more careful about my usage.

Don't you want a title that's not boring?

I've been meaning to create a blog for some time, but have been waffling between using blogspot and making my own blog technology in Haskell.

Well, today my friend Ellie (8) is visiting. She has her own blog (lala090909.wordpress.com) and has been bugging me to start mine. Clearly, she was right, so I had her create me an account at blogspot (she recommended wordpress.com, because everyone at Club Penguin uses it, but all the people I know use blogspot, so she acquiesced: "Fine, if you want to be boring.")

("Wow, that's a lot of writing," she just said.)

After typing in my name, she asked for the title. "Clifford's Posts," she suggested? "Ruminations," I replied. "Oh no, more boring," she moaned. When I insisted, she said, "Fine, how do you spell that?" Halfway through the letters, she said, "All capitals?" "No, just the first letter." "How about one capital, one lowercase, and so on." "Hmm...," I said. "You don't want it all boring, do you?"

Clearly, we have different audiences. The current point of contention is that she doesn't know what "ruminatons" means. I've delayed telling her for some time and she finally said, "Ruminations isn't that bad, I guess."

So here's the rub. I'm involved in starting a new business, using functional programming (Haskell) to make a website that will both teach math skills and be fun. Seems like I've got the boring part of the math down pat. I'm going to have to work on the fun part.

Addendum: After all that fuss, blogspot capitalized the title anyway.

Ellie just read this and said, "You're not boring, it's just that adults tend not to use colors and other things that make it exotic."