Dan Weinreb’s blog

The European Common Lisp Meeting will be in Hamburg, on the weekend of September 12 and 13, 2009.  I greatly enjoyed last year’s ECLM, in Amsterdam. It’s relaxed and gives you a lot of opportunity to meet great Lisp experts from all over the world. Arthur Lemmens and Edi Weitz did a superb job arranging for entertainment and space, and making sure everyone was happy.

I’m also looking forward to seeing Hamburg; I’ve never been there, and it sounds great.

I’m giving a talk entitled A Highly-Available Large-Scale Transaction Processing System in Common Lisp. It’s about the airline reservation system that we’re building at ITA Software, specifically about the issues involved in using Common Lisp, which is not widely thought of as being a language for writing large-scale transaction processing.

The lightning talks at the International Lisp Conference last March went so well that Edi and Arthur are trying out this format at the ECLM.  After the ILC, someone told me that at another sofware-related conference he had been to, the lightning talks fell flat: few people signed up to give talks, and they weren’t very good.  At the ILC, I thought they were nearly all great.  We learned about new tools, stories, and so on.  There was a great one about using Lisp in a Lisp-unfriendly world.  In a nutshell: if they force you to program in PERL, then run a PERL-coded Scheme interpreter and write in Scheme!  I anticipate more fun lightning talks in Hamburg!

So, I encourage you to join the fun!

Here are some upcoming music concerts in the Boston area that I’m interested in.  I hope to go as many of these as possible.  Note that the Van der Graaf Generator concert is only two days from now!

Van der Graaf Generator

Tuesday June 23 8:00pm, at the Regent Theater, Arlington Center.  This is their first American tour in 30 years, and the first time they’ve ever appeared in New England.  There’s also an opening act: The Acoustic Strawbs, which is the current version of The Strawbs.  Van der Graaf Generator is Peter Hammill’s group.  The lineup appearing here are the three surviving members of the original lineup of four who did “Pawn Hearts” and “H to He Who Am The Only One”.  My friend Olin Sibert (a huge fan of this band) and I are going.  If you like progressive rock, this is an opportunity not to be missed!

The Mahavishnu Project

Saturday July 11 7:30pm, at the Regatta Bar in the Charles Hotel, Harvard Square, Cambridge.  Greg Bendian’s band that plays the music of John McLaughlin’s Mahavishnu Orchestra.  (McLaughlin himself is a big fan.)  Greg is a close friend of my friend Tim Blackman, which is how I know about it.  Tim and I are going.

Beatle Juice

Saturday July 18 9:45pm, at Johnny D’s, in Davis Square, Somerville.  They are said to be an excellent Beatles tribute band, although I haven’t heard them.  Here’s their web site. There’s a video of them, performing at Johnny D’s, on their MySpace page.

Yes

Saturday July 25 7:00pm, at the South Shore Music Circus in Cohasset.  Asia will open.  Benoit David will substitute for Jon Anderson, who is still ill, and Oliver Wakeman will substitute for his father, Rick Wakeman, who no longer tours.  I’ve been to see this lineup before: Oliver is only OK, but Benoit is fantastic and sounds amazingly like Jon.  I’ve seen Yes seven or eight times and it never gets old, for me.  Asia is also an excellent progressive rock band. Tickets on Ticketmaster.

Birdsongs of the Mesozoic

Thursday July 30 9:00, at Johnny D’s, in Davis Square, Somerville.  I have a lot of recorded music by Birdsongs and have seen them live several times over the years. “Boston seminal art-rock.”  Check out their web site, and their MySpace page (which has music samples). Ken Field (sax, flutes, keyboard, percussion) used to be a software developer at BBN, but now he’s a full-time musician (and married to well-known local animated filmmaker Karen Aqua).

No Static

Friday July 31 9:45, at Johnny D’s, in Davis Square, Somerville.  They’re a Steely Dan tribute band.  I haven’t heard them yet.  They’ll be doing music from Aja, Royal Scam, and Gaucho, which are definitely my favorites.

Hint for Johnny D’s: come early and have dinner.  The food is fine, and then you get a good seat for the music.  Allow time to find parking in David Square if you’re coming by car, or take the Red Line.

The tag line for the International Lisp Conference 2009 was Lisp: The Next 50 Years.  I am very interested in the future of Lisp, and hope to be one of many participants in creating that future.  A widely-read paper from 1991 introduced the world to the phrase and philosophy called Worse is Better, and says that this philosophy should be used for the design of the next Lisp.  What does that mean, and what parts of the argument still apply and should guide us?

Richard Gabriel and Worse is Better

Richard P. Gabriel is a brilliant computer scientist, probably best known for his company Lucid, Inc., which produced an excellent Common Lisp implementation, and later developed a sophisticated software development environment called Energize.

He has written extensively about the process by which new technological ideas move to the marketplace.  His ideas about this are unique and very much worth learning.  The most well worked-out version of his thoughts are in his book, Patterns of Software, which I recommend highly.

His first essay on this topic is called Lisp: Good News, Bad News, How to Win Big, originally published in 1989.  It’s primarily about why the Lisp language was not succeeding as a vehicle for the delivery of practical applications.  It examines Lisp’s successes and apparent failures, and suggests how to improve things.  I find it very accurate and thoughtful, and holds up well over time.

The part that got the widest attention was Section 2.1, “The Rise of Worse is Better.”  Jamie Zawinsky, then of Lucid, forwarded this section to many people, and soon it was redistributed very widely.  It became, in effect, its own paper, generally known as Worse is Better.  Do a web search on that phrase and you’ll find all kinds of commentary.

It characterizes a school of design which Gabriel attributes to MIT and Stanford and calls “the right thing”.  He contrasts this with what he calls the “worse-is-better” philosophy, which he says “is only slightly different”.  Many commentators have oversimplified and overstressed the dichotomy, and so I strongly recommend that you read the original four points that he associates with each philosophy.  You’ll see that his characterization is careful and nuanced.

The phrase “Worse is Better” is rather over-the-top, and I think some people have misinterpreted the point because of that name.  I sympathize with Gabriel.  If you follow my own blog, you’ll see that I use somewhat provocative names for the articles, in order to attract readers.  Sometimes it backfires.  In my case, I used “Why Did M.I.T. Switch from Scheme to Python?” for an entry whose point was that the switch is not what’s important.  But perhaps since it was the title, people commented mostly on the language issue!  Oops.  Some of the commentary on Worse is Better gets confused and thinks the two philosophies are simple opposites, but it’s much more subtle than that.

Worse is Better contains a story, which starts: “Two famous people, one from MIT and another from Berkeley (but working on Unix) once met to discuss operating system issues.”  He wrote the story based on an oral account from me.  In fact, the “MIT guy” was me, and the “New Jersey guy” (from Berkeley; see the paper for why) was Bill Joy.

His account is basically right.  About the phrase “two famous people”, Bill Joy is far more famous than I am (see the current best-selling book, “Outliers”, for example).  Neither of us said “it takes a tough man to make a tender chicken” (a line from an old TV commercial), as far as I remember.  If you want to know about the issue that he spells “PC-loser-ing”, see the excellent 1989 paper PCLSRing: Keeping Process State Modular, by my friend Alan Bawden.  It has been described as “an unpublished but influential note by Bawden”, and has been widely cited.  (The general concept of PCLSR has to do with forcing a thread of execution to be X-consistent, for some level of abstraction X, even if the thread is operating below the level of X.)

Gabriel’s section ends: “But, one can conclude only that the Lisp community needs to seriously rethink its position on Lisp design. I will say more about this later.”

What does this mean for the future of Lisp?

The paper is about Lisp, but if we look carefully, it doesn’t bring the “worse is better” point to bear on Lisp very much.

Section 3.6, “The Next Lisp”, starts: “I think there will be a next Lisp. This Lisp must be carefully designed, using the principles for success we saw in worse-is-better….  The kernel should emphasize implementational simplicity, but not at the expense of interface simplicity. Where one conflicts with the other, the capability should be left out of the kernel.”

He goes on: “Some aspects of the extreme dynamism of Common Lisp should be reexamined, or at least the tradeoffs reconsidered.”  He gives an example of correct but undesirable Lisp code, in which a function redefines top-level functions.

It’s hard for a compiler to optimize code in the presence of this kind of runtime behavior.  There’s no need to write programs this way.  Lisp has better ways to do what this code fragment is trying to do, and any competent Common Lisp programmer knows that and knows the proper way.  Therefore, the next Lisp should consider omitting this capability.

Reducing extreme dynamism, way out at the edges, sounds promising, and should be considered carefully.  But this specific example is the only one he gives!

The rest of the section is about how to layer the implementation.  All of this is great, but it does not seem to have anything to do with “Worse is Better”!

The next section is “Help Applications Writers Win”, and clearly the right thing philosophy makes things better for application writers than the worse is better philosophy, all other things being equal.  The point of the paper is that all other things aren’t equal because the worse is better philosophy should help get the system done on time and help it spread.  But that’s just the overall thesis of the paper, not specific to Lisp at all.

Why does this paper spend so much time on the Worse is Better philosophy, when it bears so little on Lisp?

I’ll go out on a limb and speculate that this was very much on Gabriel’s mind at the time.  He felt it was relevant to Lisp because MIT/Stanford people were frustrated that Unix seemed to be ignoring lessons and techniques that had been developed, and widely used, over so many years.  He might even have been thinking of the competition between his own Lucid Lisp product and its competitors.  But I ought not put words in his mouth.

Gabriel later wrote much more about the Worse is Better philosophy.  He famously conducted a debate with himself, writing the other side under the pseudonym “Nickieben Bourbaki” (an allusion to Nicolas Bourbaki).  These include Worse is Better is Worse, Is Worse Really Better?, and even more.

What do you think: do the ideas in the Worse is Better series of papers bear on the question of the future of Lisp?  I’d appreciate if you’d take a look at Gabriel’s paper before answering!

P.S. Dept. of Fair Attribution: I borrowed some phrases of text from various Wikipedia articles.  Look here for more general discussions about the future of Lisp.

XSITE 2009 is a one-day conference that will showcase innovative business in New England.  It’s at the Boston University School of Management, 595 Commonwealth Avenue, Boston MA, on Wednesday, June 24, 2009.

High-tech, life-sciences, and energy innovation may well hold the key to the nation’s economic health and long-term competitiveness. And as a hotbed of American innovation, New England is poised to become a key driver of economic recovery in the U.S.

There are many presentations by CEO’s of innovative companies, including many startups.  There’s a panel session on “Investing in Innovation”.  And there’s time for networking: you can chat with and ask questions of the presenters, or just meet the other attendees.  Here’s the preliminary agenda.

I’ve been to meetings like this before, and always find them valuable and interesting.  The Nantucket Conference is like this, but it’s by invitation only. XSITE is open to anyone.

XSITE is run by Xconomy.com. I attended their one-day conferences on cloud computing and on mobile applications, and both were excellent.  So I have high hopes for XSITE as well.  (Full disclosure: I am an investor in Xconomy.)

If you plan to register, the sooner the better, since the price goes up on June 1 and again on June 15.

I’ve been seeing mail and blog postings, particularly from people in the Lisp community, why MIT has switched from using Scheme to Python in the freshman core curriculum for the department of Electrical Engineering and Computer Science.

At the International Lisp Conference, Prof. Gerry Sussman gave a short impromptu talk explaining the new freshman curriculum.  Just to get a second opinion, I later called Prof. Jacob White, one of the designers of the curriculum and lecturers for the core courses.  (Digression: Jacob and I have been close friends since I was six years old!)  He confirmed Gerry’s description.

Asking why they changed languages is, in some sense, the wrong question.

The freshman software engineering course, since 1985, has been based on the book Structure and Interpretation of Computer Programs (known as SICP), which uses Scheme.  The course is now nearly thirty years old.  Engineering has changed quite a lot in thirty years.  Since 1995, Gerry and his co-author Prof. Hal Abelson have advocated changing the freshman curriculum radically, not basing it on SICP.

In 1980, computer engineering was based on starting with clearly-defined things (primitives or small programs) and using them to build larger things that ended up being clearly-defined.  Composition of these fragments was the name of the game.

However, nowadays, a real engineer is given a big software library, with a 300-page manual that’s full of errors.  He’s also given a robot, whose exact behavior is extremely hard to characterize (what happens when a wheel slips?). The engineer must learn to perform scientific experiments to find out how the software and hardware actually work, at least enough to accomplish the job at hand.  Gerry pointed out that we may not like it this way (”because we’re old fogies”), but that’s the way it is, and M.I.T. has to take that into account.

The new approach also has the big advantage that it combines computer science with electrical engineering, whereas the old one taught them as entirely separate disciplines.  This way, students see how they interrelate.  Also, as Jacob points out, some of the same macro-principles apply to both software and hardware, and the students see this illustrated.  There is extensive lab work, making robots and mobile applications.

It just so happens that the robotics substrate software that comes with the system they’re using is programmed in Python.  Similarly, the mobile software environment is based on Python.  (Or, at least, the original plan was to use such a substrate, although it may have changed for various business reasons.)

Changing programming languages was absolutely not a goal of the curriculum change.  It was merely the result of the consequences of various decisions.  We can always discuss how it came to be that the robots and mobile devices are using Python instead of some other language, but that’s not the question being addressed here.  M.I.T. has nothing against Scheme. (And, of course, M.I.T. does teach classic software engineering, later in the curriculum.)

(Here’s another take on this topic.)