The Failure of Lisp? A Reply To Brandon Werner

Brandon Werner has written an excellent, thought-provoking post on his blog entitled “The Rise Of Functional Programming: F#/Scala/Haskell and the failing of Lisp”.

He currently works for Microsoft, but I think it’s clear that this posting reflects his own opinions rather than being the corporate voice of Microsoft. He clearly has lot of real experience with Common Lisp and knows whereof he speaks, and I take him very seriously. I started to write a comment for the blog, but it got too big for that. So, here’s my open reply to Brandon:

Hi. I am one of those 50 year old men. (Well, I’ll be 50 in January, and I don’t use IRC.) I was one of the designers of Common Lisp and one of the co-founders of Symbolics. I wrote the second Emacs ever, in Lisp, following along as RMS developed the first Emacs (in TECO), when I was a teenager.

I currently work at ITA Software, a.k.a “the 800-pound gorilla of Common Lisp”. If you use Orbitz and ask “how do I get from Boston to Chicago on 10/4/2000 at 2pm …”, we provide an excellent set of choices of the cheapest routes and fares for which seats are available. This program, known as QPX, is written in Common Lisp.

I am working on our new product, an airline reservation system. It’s an online transaction-processing system that must be up 99.99% of the time, maintaining maximum response time (e.g. on www.aircanada.com). It’s a very, very complicated system. The presentation layer is written in Java using conventional techniques. The business rule layer is written in Common Lisp; about 500,000 lines of code (plus another 100,000 or so of open source libraries). The database layer is Oracle RAC. We operate our own data centers, some here in Massachusetts and a disaster-recovery site in Canada (separate power grid).

There are currently a total of eleven implementations of Common Lisp being actively maintained; see my survey. I see you are clearly much more interested in the seven free ones than the four for-money ones. At ITA Software, QPX uses SBCL, and the airline reservation system uses Clozure Common Lisp (formerly known as OpenMCL). There are a bunch of reasons we use different dialects, including historical and business ones that don’t apply to anyone but us. They’re both great. SBCL takes longer to compile but generates better code, which is more important for QPX (totally compute-bound) than for the airline reservation system (TP). Personally, I’d recommend these two, but reasonable people differ.

I am distressed and sad to hear that the community is judgmental and unfriendly to newcomers and thorny and un-inspiring. I have heard this same criticism from other people than you, and at this point I assume it must really be true. My own point of view is, of course, entirely different from that of newcomers, so it’s probably harder for me to see that this is going on. Indeed, to me it seems that people do get answers on comp.lang.lisp and LispForum, and the tone doesn’t seem so nasty to me, usually. Maybe I’m just not “getting it”.

I would truly appreciate if you could let me know more specifically what kind of incidents you have encountered. (Was this mainly on IRC?) Putting people off like that is, in my opinion, rude, unethical, and obviously very bad for Lisp.

Please send me side mail about ASDF being incompatible in the two implementations. It’s certainly not supposed to be, and I’m not getting this from your installation writeup. Thank you. I agree that installation could be made more friendly and beginner-oriented for all these implementations. It should be much more “Batteries Included”. There has been at least one serious attempt to deal with this: Google for “Lisp in a Box” (and Peter Siebel has plans to improve it even more) . But your general point is still right; we need much more of what “Lisp in a Box” is doing. The fact that too little effort has been put into this is part of a greater issue about the attitude of the community; more below.

You say “.. Common Lisp showed its failure as a community by sitting out this enthusiasm …” and I agree completely. In my opinion, this is part of the same issue about the attitude of the community.

Here’s another one you didn’t mention. Look at www.python.org and www.ruby-lang.com. They immediately tell you what the language is and what’s good about it (and they’re attractive). Now look at www.lisp.org. This will be vastly improved shortly, but the point is that it is yet another reflection of the same attitude issue.

The problems are that there is hardly any organized “Lisp community” at all, and that few people have been putting serious effort into trying to publicize and promote Lisp, and get new people involved.

The only companies with a direct financial interest, as far as I know, are the four for-money Lisp vendors. Franz has made some serious efforts, but what I understand now is that they’ve had so much trouble making money from selling Lisp that they’re spending less effort on that, and more effort on developing new products (many built on Lisp underneath). The other companies are basically too small to be effective.

Open source communities, like those around Python and Ruby, have been very effective. Those of us who’d like to see Lisp promoted need to understand more about how new languages have been so successful at this, and then rouse people to undertake the work of making it all happen. Certainly part of that is to provide a friendly community who can help and encourage new users.

I hope we can address these issues and get some real work going at the next International Lisp Conference. It’s at MIT, Mar 22-25; I’m general chair. (There’s not much info at the web site yet, but it’s coming soon.)