Dan Weinreb’s blog

Registration for the International Lisp Conference, 2009, is now open.

The conference will be from Sunday, March 22 through Wednesday, March 25, at the Stata Center, MIT, Cambridge, MA.

It features:

• Gerald Sussman, Shriram Krishnamurthi, and David Moon as invited speakers
• Five tutorials (no extra charge)
• Fifteen technical papers
• Seven demonstrations
• Lightning talks (you can give you own; first-come, first-served)
• An all-star panel on the future of Lisp
• The Great Macro Debate
• Birds of a feather sessions, and other informal discussions
• Banquet at the Hyatt-Regency, with special entertainment (no extra charge)

Come meet the top Lisp experts and practitioners in the world. Learn how to get the most out of Lisp. Find about about the latest developments from research and industry. The conference is a rare opportunity for face-to-face interaction, sharing knowledge and ideas with the experts of the worldwide Lisp community. Students are especially welcome. Everyone will have a great time!

Here’s where to find all the information.

Sponsored by:

• ITA Software, Inc.
• Franz Inc.
• LispWorks Limited
• Clozure, Inc.
• Ravenbrook Limited

Brandon Werner has written an excellent, thought-provoking post on his blog entitled “The Rise Of Functional Programming: F#/Scala/Haskell and the failing of Lisp”.

He currently works for Microsoft, but I think it’s clear that this posting reflects his own opinions rather than being the corporate voice of Microsoft.  He clearly has lot of real experience with Common Lisp and knows whereof he speaks, and I take him very seriously.  I started to write a comment for the blog, but it got too big for that.  So, here’s my open reply to Brandon:

Hi.  I am one of those 50 year old men.  (Well, I’ll be 50 in January, and I don’t use IRC.)  I was one of the designers of Common Lisp and one of the co-founders of Symbolics.  I wrote the second Emacs ever, in Lisp, following along as RMS developed the first Emacs (in TECO), when I was a teenager.

I currently work at ITA Software, a.k.a “the 800-pound gorilla of Common Lisp”.  If you use Orbitz and ask “how do I get from Boston to Chicago on 10/4/2000 at 2pm …”, we provide an excellent set of choices of the cheapest routes and fares for which seats are available.  This program, known as QPX, is written in Common Lisp.

I am working on our new product, an airline reservation system.  It’s an online transaction-processing system that must be up 99.99% of the time, maintaining maximum response time (e.g. on www.aircanada.com).  It’s a very, very complicated system.  The presentation layer is written in Java using conventional techniques.  The business rule layer is written in Common Lisp; about 500,000 lines of code (plus another 100,000 or so of open source libraries).  The database layer is Oracle RAC.  We operate our own data centers, some here in Massachusetts and a disaster-recovery site in Canada (separate power grid).

There are currently a total of eleven implementations of Common Lisp being actively maintained; see my survey.  I see you are clearly much more interested in the seven free ones than the four for-money ones.  At ITA Software, QPX uses SBCL, and the airline reservation system uses Clozure Common Lisp (formerly known as OpenMCL).  There are a bunch of reasons we use different dialects, including historical and business ones that don’t apply to anyone but us.  They’re both great.  SBCL takes longer to compile but generates better code, which is more important for QPX (totally compute-bound) than for the airline reservation system (TP).  Personally, I’d recommend these two, but reasonable people differ.

I am distressed and sad to hear that the community is judgmental and unfriendly to newcomers and thorny and un-inspiring.  I have heard this same criticism from other people than you, and at this point I assume it must really be true.  My own point of view is, of course, entirely different from that of newcomers, so it’s probably harder for me to see that this is going on.  Indeed, to me it seems that people do get answers on comp.lang.lisp and LispForum, and the tone doesn’t seem so nasty to me, usually.  Maybe I’m just not “getting it”.

I would truly appreciate if you could let me know more specifically what kind of incidents you have encountered.  (Was this mainly on IRC?)  Putting people off like that is, in my opinion, rude, unethical, and obviously very bad for Lisp.

Please send me side mail about ASDF being incompatible in the two implementations.  It’s certainly not supposed to be, and I’m not getting this from your installation writeup.  Thank you. I agree that installation could be made more friendly and beginner-oriented for all these implementations.  It should be much more “Batteries Included”.  There has been at least one serious attempt to deal with this: Google for “Lisp in a Box” (and Peter Siebel has plans to improve it even more) .  But your general point is still right; we need much more of what “Lisp in a Box” is doing.  The fact that too little effort has been put into this is part of a greater issue about the attitude of the community; more below.

You say “.. Common Lisp showed its failure as a community by sitting out this enthusiasm …” and I agree completely.  In my opinion, this is part of the same issue about the attitude of the community.

Here’s another one you didn’t mention.  Look at www.python.org and www.ruby-lang.com.  They immediately tell you what the language is and what’s good about it (and they’re attractive).  Now look at www.lisp.org.  This will be vastly improved shortly, but the point is that it is yet another reflection of the same attitude issue.

The problems are that there is hardly any organized “Lisp community” at all, and that few people have been putting serious effort into trying to publicize and promote Lisp, and get new people involved.

The only companies with a direct financial interest, as far as I know, are the four for-money Lisp vendors.  Franz has made some serious efforts, but what I understand now is that they’ve had so much trouble making money from selling Lisp that they’re spending less effort on that, and more effort on developing new products (many built on Lisp underneath).  The other companies are basically too small to be effective.

Open source communities, like those around Python and Ruby, have been very effective.  Those of us who’d like to see Lisp promoted need to understand more about how new languages have been so successful at this, and then rouse people to undertake the work of making it all happen.  Certainly part of that is to provide a friendly community who can help and encourage new users.

I hope we can address these issues and get some real work going at the next International Lisp Conference.  It’s at MIT, Mar 22-25; I’m general chair.  (There’s not much info at the web site yet, but it’s coming soon.)

Thank you very much!

– Dan

Friends have asked me why I signed up to work at ITA Software, and how I feel about it.

For the last four years, I had been working at BEA Systems, in their Burlington MA office. I had joined BEA to work on an exciting new product (a message broker), and because I was very psyched about working for Adam Bosworth. Unfortunately it didn’t work out. There were too many problems with how the message broker would fit into BEA’s overall architecture, which bogged us down for a long time. Eventually the message broker was made part of the WebLogic Integration product, and moved to San Jose. Meanwhile, Adam Bosworth left BEA to go to Google. BEA found a new position for me, as architect of the “Operations, Administration, and Management” aspect of WebLogic Server. Unfortunately, the WebLogic Server group was in the middle of a very long release cycle, so all of my ideas had to be on hold until the next cycle. Also, WebLogic Server’s technical strategy was changing every quarter or so, and I’d have to start all over again. I learned a lot at BEA, studying the WebLogic Server, but finally the work was too frustrating.

Meanwhile, I had been keeping in touch with my friend Scott McKay, who had been one of the senior software engineers at Symbolics. Scott was architecting and implementing a new product at BEA, and was very enthusiastic about it. I had several other friends working at ITA, from both Symbolics and Object Design. So I thought this might be a good place to be. In January of 2006, I had lunch with a group of these friends, and Sundar Narasimhan, the CTO, and they convinced me to join.

I love working at ITA. My own primary criterion for an employer is that I get to work directly with extremely good software engineers who work well together. Symbolics and Object Design were like that, and ITA has the same kind of environment. My co-workers have lots of knowledge and experience. I learn new things from them continually.

I didn’t have any a priori interest in the airline software field, but I was fascinated by transaction processing systems. I had been studying operating systems, database management systems, and application servers for years, but I had never had a chance to contribute to a real-world, high-performance, high-availability system. Airline reservation systems were the first transaction processing systems. Most major airlines still use (some modified fork of) the original system, originally known as SABRE. It is written in assembly language on IBM mainframes, and the original creators are mostly retired or deceased. After forty years, these systems are still in operation, since they basically work and are so hard to replace. But the airline industry has changed, and has pressing new requirements. When our system becomes operational, it will be a major innovation for the industry.

It’s a very big piece of software, with a whole lot of moving parts. It has to be, because of the problem definition and the customer requirements. Fortunately, we have a great customer: Air Canada. We work very closely with them. Their CEO has realistic expectations and a very good attitude about the way the development process works. Air Canada has stepped up to be the pioneer, and it’s our job to minimize the arrows in their back. Eventually we’ll have other airlines as customers, after they see the success at Air Canada.

Scott, Sundar, I, and several others are improving the overall architecture of the reservation system. Among many other things, we’re working on high availability: making the system stay up all the time, despite any kind of failure that we can reasonably anticipate. I have been focusing specifically on the problem that we call “hot upgrade”: how to install new versions of components of the system, while it’s running, without impacting latency. This is very challenging and a lot of fun.

ITA has been voted one of the twenty best places to work for in Boston by the Boston Business Journal two years running. We work in offices rather than cubies. There are free drinks and snacks, and the company serves lunch every Friday. We have a weekly technical seminar, where speakers from both inside and outside ITA give talks on nearly anything (for example, recently Tom Knight from MIT told us about his work in synthetic biology). There are informal activities like Movie Night, Movie Camp, Math Lunch, and so on. There’s an entrepreneurial culture and innovative spirit.

I have learned over the years that in a small software company, the CEO has a huge impact on nearly all aspects of the company. Jeremy Wertheimer, ITA’s CEO, is the best I’ve ever worked for. He knows how to run the business, about the industry, and how to hire great people. He’s extremely technical, quite able to participate in engineering discussions, and knows a lot about software engineering and how to run software projects effectively.

And I get to program in Common Lisp again! So I’m having a great time.