The “Worse is Better” idea and the future of Lisp
The tag line for the International Lisp Conference 2009 was Lisp: The Next 50 Years. I am very interested in the future of Lisp, and hope to be one of many participants in creating that future. A widely-read paper from 1991 introduced the world to the phrase and philosophy called Worse is Better, and says that this philosophy should be used for the design of the next Lisp. What does that mean, and what parts of the argument still apply and should guide us?
Richard Gabriel and Worse is Better
Richard P. Gabriel is a brilliant computer scientist, probably best known for his company Lucid, Inc., which produced an excellent Common Lisp implementation, and later developed a sophisticated software development environment called Energize.
He has written extensively about the process by which new technological ideas move to the marketplace. His ideas about this are unique and very much worth learning. The most well worked-out version of his thoughts are in his book, Patterns of Software, which I recommend highly.
His first essay on this topic is called Lisp: Good News, Bad News, How to Win Big, originally published in 1989. It’s primarily about why the Lisp language was not succeeding as a vehicle for the delivery of practical applications. It examines Lisp’s successes and apparent failures, and suggests how to improve things. I find it very accurate and thoughtful, and holds up well over time.
The part that got the widest attention was Section 2.1, “The Rise of Worse is Better.” Jamie Zawinsky, then of Lucid, forwarded this section to many people, and soon it was redistributed very widely. It became, in effect, its own paper, generally known as Worse is Better. Do a web search on that phrase and you’ll find all kinds of commentary.
It characterizes a school of design which Gabriel attributes to MIT and Stanford and calls “the right thing”. He contrasts this with what he calls the “worse-is-better” philosophy, which he says “is only slightly different”. Many commentators have oversimplified and overstressed the dichotomy, and so I strongly recommend that you read the original four points that he associates with each philosophy. You’ll see that his characterization is careful and nuanced.
The phrase “Worse is Better” is rather over-the-top, and I think some people have misinterpreted the point because of that name. I sympathize with Gabriel. If you follow my own blog, you’ll see that I use somewhat provocative names for the articles, in order to attract readers. Sometimes it backfires. In my case, I used “Why Did M.I.T. Switch from Scheme to Python?” for an entry whose point was that the switch is not what’s important. But perhaps since it was the title, people commented mostly on the language issue! Oops. Some of the commentary on Worse is Better gets confused and thinks the two philosophies are simple opposites, but it’s much more subtle than that.
Worse is Better contains a story, which starts: “Two famous people, one from MIT and another from Berkeley (but working on Unix) once met to discuss operating system issues.” He wrote the story based on an oral account from me. In fact, the “MIT guy” was me, and the “New Jersey guy” (from Berkeley; see the paper for why) was Bill Joy.
His account is basically right. About the phrase “two famous people”, Bill Joy is far more famous than I am (see the current best-selling book, “Outliers”, for example). Neither of us said “it takes a tough man to make a tender chicken” (a line from an old TV commercial), as far as I remember. If you want to know about the issue that he spells “PC-loser-ing”, see the excellent 1989 paper PCLSRing: Keeping Process State Modular, by my friend Alan Bawden. It has been described as “an unpublished but influential note by Bawden”, and has been widely cited. (The general concept of PCLSR has to do with forcing a thread of execution to be X-consistent, for some level of abstraction X, even if the thread is operating below the level of X.)
Gabriel’s section ends: “But, one can conclude only that the Lisp community needs to seriously rethink its position on Lisp design. I will say more about this later.”
What does this mean for the future of Lisp?
The paper is about Lisp, but if we look carefully, it doesn’t bring the “worse is better” point to bear on Lisp very much.
Section 3.6, “The Next Lisp”, starts: “I think there will be a next Lisp. This Lisp must be carefully designed, using the principles for success we saw in worse-is-better…. The kernel should emphasize implementational simplicity, but not at the expense of interface simplicity. Where one conflicts with the other, the capability should be left out of the kernel.”
He goes on: “Some aspects of the extreme dynamism of Common Lisp should be reexamined, or at least the tradeoffs reconsidered.” He gives an example of correct but undesirable Lisp code, in which a function redefines top-level functions.
It’s hard for a compiler to optimize code in the presence of this kind of runtime behavior. There’s no need to write programs this way. Lisp has better ways to do what this code fragment is trying to do, and any competent Common Lisp programmer knows that and knows the proper way. Therefore, the next Lisp should consider omitting this capability.
Reducing extreme dynamism, way out at the edges, sounds promising, and should be considered carefully. But this specific example is the only one he gives!
The rest of the section is about how to layer the implementation. All of this is great, but it does not seem to have anything to do with “Worse is Better”!
The next section is “Help Applications Writers Win”, and clearly the right thing philosophy makes things better for application writers than the worse is better philosophy, all other things being equal. The point of the paper is that all other things aren’t equal because the worse is better philosophy should help get the system done on time and help it spread. But that’s just the overall thesis of the paper, not specific to Lisp at all.
Why does this paper spend so much time on the Worse is Better philosophy, when it bears so little on Lisp?
I’ll go out on a limb and speculate that this was very much on Gabriel’s mind at the time. He felt it was relevant to Lisp because MIT/Stanford people were frustrated that Unix seemed to be ignoring lessons and techniques that had been developed, and widely used, over so many years. He might even have been thinking of the competition between his own Lucid Lisp product and its competitors. But I ought not put words in his mouth.
Gabriel later wrote much more about the Worse is Better philosophy. He famously conducted a debate with himself, writing the other side under the pseudonym “Nickieben Bourbaki” (an allusion to Nicolas Bourbaki). These include Worse is Better is Worse, Is Worse Really Better?, and even more.
What do you think: do the ideas in the Worse is Better series of papers bear on the question of the future of Lisp? I’d appreciate if you’d take a look at Gabriel’s paper before answering!
P.S. Dept. of Fair Attribution: I borrowed some phrases of text from various Wikipedia articles. Look here for more general discussions about the future of Lisp.
June 7th, 2009 at 9:30 am
And then Dan Weinreb met Bill Joy: http://www.clojure.org
June 7th, 2009 at 12:17 pm
I have been laboring in the vineyards of WIB (or perhaps, Just Enough Is Good Enough) for a while now, developing a new implementation of ISLisp ISLisp is a WIB Lisp: see http://islisp.info for details on the language. It’s an attempt to take something basically CL-ish, streamline it quite a lot (to about the size of R5RS), clean up many overlapping features (just classes, not structs and classes both, e.g.), and make it as un-dynamic as possible (it still has dynamic typing and dynamic method dispatching, though).
When my implementation is finished, it’ll be a self-hosting compiler that generates dumb, enterprisey Java code, thus leveraging not only the Java libraries but the Java JITs, which are specifically engineered to optimize the hell out of dumb, enterprisey Java code.
(To really do this right, I’d have to do a lot of type inference, but I’m not ambitious enough for that, so the generated code will be full of casts from Object.)
June 8th, 2009 at 1:41 am
I tend to think all of you philosophize too much into success in terms of popularity. It’s all about chance, fame, fortune and bigger fishes eating the smaller ones, a sudden outburst of needs etc. In case of the failure of Lucid it was IBM that crushed it. One could also argue using economical arguments: Lisp machines were far more expensive etc. but this type of argument is also flawed because prizes are not determined by natural laws but by demand and supply and competition etc. There is a vortex of causality but no meaning.
Human beings are predictably irrational and if something positive and worthwhile becomes mainstream one can rationalize this post facto by meritocratic arguments and people start to believe that history can be rationally reconstructed. We are used to seek for constant pattern and rules even if there are just contingent events from which no one can draw reliable conclusions. We attempt to give the meaningless meaning and ask ourselves why we are not as successful as our neighbours and a sudden change in the environmental conditions is going to change our condition and our own religion becomes dominant and persecutes the non-believers.
So worse isn’t better and many things that strive to be cool and pragmatic and user friendly etc. are vanishing unseen while others that break all those rules suddenly keep attention (e.g. Haskell). I’d recommend to just stop suffering from envy and missed opportunities.
June 8th, 2009 at 2:07 am
OT: AFAIK Alan Bawden was a moderator of the Unix-Haters mailing list. Perhaps you know if a fuller archive than this is available (I confess to having read the 1990-07-17 to 1993-10-04 archives several times and deriving perverse pleasure I keep craving for ever since.)
June 8th, 2009 at 3:08 am
I don’t know if I fully understand the implications of worse-is-better. It’s such a complicated argument!
There are so many aspects upon which you might compare any two technologies that it seems like the succesful one is often the worst solution in many of those aspects. Take C++: its dominance had little to do with its qualities as a programming language and more to do with its ability to be acceptable on many levels. This ability to be ‘average’ in very many respects seems to be far more valuable (for programming languages at least) than being good in just a few.
So, without trying to drift off-topic, it seems that worse-is-better is more useful as a tool in hindsight? I’m not sure that it can necessarily tell us where to go next?
June 8th, 2009 at 3:23 am
I’d guess Gabriel focuses on “worse is better” (WIB) because he’s a bit bitter about Lisp’s fortunes, and WIB is a kind of philosophy of defensive consolation.
In other words I think the true implication, for Lisp, of WIB is not just the argument it makes about poor technologies beating out superior technologies. It is also that WIB encapsulates a certain feeling of frustrated superiority, a feeling that seems a little more common in the Lisp community than in most PL communities. (E.g., Google “the bipolar lisp programmer”.)
And those feelings are an obstacle to evolving the language. After all, if CL is already “better”, than why improve it? But everything can be improved. I have my own wishlist of improvements for CL, but I am such a newbie that I doubt my opinion matters much. Ron Garret’s list seems pretty carefully considered.
I have the deepest respect for Lisp gurus, so I hope this doesn’t sound like cheap psychologizing of the issue. It’s quite possible I’m overreacting to my impression from the Lisp newsgroups. But these psychological factors are at least one part of the reality of Lisp today and its prospects.
June 8th, 2009 at 3:24 am
correction: “I am such a newbie I doubt my opinion should matter much”
June 8th, 2009 at 7:53 am
Gabriel was running the Lucid business. Their main product was a CL system and their business strategy was to find other companies selling their CL and adding stuff (the OEM model). These companies mostly were the first workstation companies. CL is a bit over-engineered and large in size. The market was stagnating after some time and then shrinking rapidly (the AI winter). Lucid used the money they earned in the Lisp business to build a C++ environment with similar features (incremental and database-based development): ‘Energize’. This environment did cost a lot of money to develop, it had not enough users and sunk Lucid, the company then (my interpretation). Lucid never invested money in Lucid CL, but used it as a cash cow for developing new (non-Lisp) products.
I would expect that another strategy would have helped Lucid to survive longer, but maybe on a smaller scale. The main competitors from that time (Franz with Allegro CL and Harlequin’s LispWorks) are still maintained have see new versions from time to time.
It is also kind of ironic that Lucid CL was developed out of a Symbolics project, ‘Portable Common Lisp’ – when Symbolics killed that project, the developers did develop ‘Lucid CL’.
My view: it is not so much that Lisp (and ultimatively the Lisp Machine) was better and C/Unix/PC were worse. In my view the C/Unix/PC products were more flexible, cheaper, had lower entry barriers and Lisp did not integrate that well on these new platforms. Lisp was always a bit alien (managed memory with garbage collection, object-oriented, other delivery ideas, …). The Lisp Machine on the other side was a nice single user system written mostly not in Common Lisp. The OS was not easily portable, did not support multiple users at the same time, did not support safe background applications. The whole technology was developed by engineers with government money – other markets demanded more flexible and cost effective solutions and the Lisp community was not able to deliver that (with a few exceptions) and was not able to effectively scale down the technology to make it cheaper and usable by non-MIT people
. Plus the software was not easily available (locked in commercial companies) or could be not be re-created.
In my view the Lisp technology of that time was not ‘better’ – in an evolutionary sense it might be stronger than competing technology (one could solve complex problems given excellent programmers and lots of money), but it did not fit into the changing environments, failed to adapt (also because almost nobody was interested to adapt it during the AI winter) and lost to other more flexible solutions.
June 8th, 2009 at 9:38 am
Kay Schluehr, I think I love you. That was a succinct summary of the problem with language and OS philosophy debates; indeed, many historical debates of all kinds. The fact that Lisp is not where C or Java are today could simply be a matter of the randomness of social networks in the early days of computing. There is nothing wrong with debating the merits of one design philosophy over another, but to attribute popularity of a language to its design is risky business.
June 8th, 2009 at 9:53 am
You point out one thing that, to people who built and used Common Lisp extensively, is obvious the next Lisp should have. I’ve found others by reading Gabriel’s old papers, especially about performance.
The problem is that I find myself disinclined to make The Next Lisp precisely because there are all these guys who know so much more than me. And with Python or Ruby or Perl, the guys who made version 1 made version 2 and are still around. I wouldn’t even know where to find any of the CL designers to ask if I’m making a really dumb design decision. There’s a discontinuity of culture.
(Part of it seems to be that many old-school Lisp programmers think CL is good enough, and that improvements should be at the library or implementation level, which is more true than in other languages, but I don’t accept that as a general solution to all of CL’s problems.)
I know there are newer Lisp dialects (Clojure, newLISP, …), but I’m no Common Lisp expert yet I can see lessons of Common Lisp that they didn’t learn from. And the people who wrote these compilers are probably smart cookies, so what chance do I have?
I’m not complaining to you, and I don’t know what exactly I’m looking for. I just can’t shake the feeling that a lot of computer science knowledge was lost between the 1980′s and 2000′s simply because there was no next-Common Lisp.
cheers,
June 8th, 2009 at 12:08 pm
joswig++
A lot of people get confused by the phrase “worse is better”.
Gabriel was a Lisp programmer speaking to other Lisp programmers; this was the context of his essay. Therefore Gabriel’s use of “worse” was from the point of view of Lisp programming. It’s arguably a complete red herring, but it’s ironic that it’s a red herring generated by and for the Lisp community itself. Only Lispers were worried about being worse or better than C. In a way one could say that “proves” that C really was better – C users and vendors didn’t have to worry about their approach, because it was winning more attention.
From points of view outside of the Lisp community, C and Unix were not worse technologies, they were excellent technologies (I do not say “better” – though I would say better-suited), for all the reasons joswig describes. My opinion is that this is what Gabriel was trying to communicate to his peers. (I think they didn’t really get the message. And Gabriel seems like the type of guy who would accept take some blame for that, in that he used vaguely alarming, fancy phrases like “worse is better” to make his point. I think though he really did think that Lisp was better, in an absolute sense, than C.)
For what it’s worth, IMO, “worse is better” is basically “release early, release often”. On the surface these two attitudes might not seem the same, but when you dig in, I think they do not differ in any important way.
June 8th, 2009 at 9:01 pm
1. Please don’t cite Wikipedia, or any encyclopedia for that matter. Cite the referenced articles. I love Wikipedia, but anyone can write anything in a Wikipedia article at any point in time, and for any particular piece of information in an article, you have no way of knowing for sure if it is accurate or not.
2. The information you present here is very interesting from the historical perspective, but I don’t see how you can possibly talk of the future of Lisp without discussing the goals of Lisp.
June 8th, 2009 at 9:05 pm
Having reread Gabriel’s paper, I am not sure I buy the argument that Common Lisp was “the right thing” and C was “worse is better.” Certainly I would buy the argument that Multics was “the right thing” and Unix was “worse is better.” Or to say almost the same thing in slightly different words, Multics was a large corporate effort and Unix was a small guerilla effort. ITS was like Unix in that respect, but unlike Unix in that ITS people were never interested in spreading it to other users.
But I don’t remember Maclisp, Zetalisp, or Common Lisp being subject to years of careful analysis of requirements to reach a perfect design before implementing anything. The way I remember it, they were just hacked up as we went along. That sounds almost like they were worse-is-better systems. Certainly there was an aesthetic of cleanliness, but it was never allowed to delay typing in some code and trying it out. I think one would have to look elsewhere for the difference between Lisp and C. Their divergent fortunes are perhaps partly due to a different choice of what level of ambition to emphasize in terms of size of programs to support and consequently what size of hardware to assume.
Also don’t forget that Unix and C wandered in the wilderness for many years until they suddenly were adopted as a solution to a new problem. In a parallel universe they might never have replaced some of the proprietary operating systems. This is mostly just an accident of being in the right place at the right time, not something one could replicate by adopting the right (or “wrong”!) philosophy. That’s my opinion anyway.
June 8th, 2009 at 9:49 pm
I am not sure how relevant the Worse is Better paper is to a project to develop a new Lisp starting now. Literally everything has changed in the past 20 years or so. For one thing, least common denominator hardware is 10,000 times more powerful. For another thing, all the dominant application frameworks or operating systems, whatever you want to call them, incorporate garbage collection now. For a third thing, the kinds of applications being written are totally different now. That list could be a lot longer.
If we were serious about developing a new Lisp we would have to start by agreeing on what we mean now by “Lisp.” For example, what do we want from “Lisp” that Java and C# don’t already give us? Or, if we had “Lisp” on the iPhone what could we program the iPhone to do that is too hard to do there today?
Take Java for an example, just to make our thinking more concrete. It already gives us a lot of Lispy things: garbage collected memory, resizable arrays (Vectors), dynamic loading of code including code synthesized at run time, development environments that let you change running code (although the Lisp environments were never so buggy!), lots of libraries, some of which are useful and a few of which actually work as advertised. I am sure the Lispiness of Java is not an accident.
(By the way, Java takes a “the right thing” approach and Lisp takes a relatively “worse is better” approach. So what do you think of that rpg?)
What doesn’t Java give us that Lisp gives us, or should have given us? Surely f(x,y) vs (f x y) doesn’t matter much, nor does assoc vs Map.get(). Well, Java programs tend to be much more verbose than equivalent Lisp programs; there is a lot of bureaucratic paperwork to fill out in a Java program that you just don’t see in a Lisp program. Some of this is the rigid structure of classes and methods, where Lisp programs are more free-form. Some is the need to declare all types, to declare public/private access modes, to declare exceptions, to build and access a Properties object instead of just using keyword arguments, to write a bunch of boilerplate instead of a simple lambda. It’s all a bunch of junk that maybe benefits the compiler but from a Lisp point of view just crufts up the program, getting in the way of understanding what it is really doing.
But there is another way that Java programs are more verbose: Java doesn’t have the same casually useful data structures that Lisp has. You have to write a mountain of code in Java where you would just call mapcar or reduce or assoc in Lisp. Maybe Java gains some runtime efficiency from that, certainly that’s the rationale, but it’s often premature optimization and I’m not even sure the gain is real except in contrived cases.
So maybe we want the next Lisp to be like Java but more concise. Concision would come both from leaving more trivial programming details unstated in source code, and from having a better library of data structures and operations on them. I would also contend that Java’s rigid conflation of classes with method libraries would have to go. No one should have to write a bunch of junk to work around the failure to support multimethods in the language.
But we don’t want to be like Ruby or Python and give up performance to get concision. As Lispers we see no reason why we shouldn’t have both speed and conciseness. Small memory footprint is harder to get, but not very important these days.
Have I mentioned macros? They ought to help with the verbosity problem. A lot.
Java isn’t even a good implementation of Java in some ways. For example, there are too many kinds of collection and half of them don’t even implement the Collection interface! (Why can’t you call contains() on an array?) For another example, there are too many places where you have to use Object instead of the type you really mean. And don’t even get me started on NullPointerException! We certainly wouldn’t want to imitate those mistakes in a new Lisp.
To my mind, no language where + ever returns a mathematically incorrect result is a Lisp. I don’t ever want to add two positive integers and get a negative result! Am I really being overly pedantic?
Also Java doesn’t really give us a standard built-in way to redefine things. There are ways but they are not reliable. The Lisp development methodology where you spend all day evolving your program without having to reload it and start over doesn’t really work in Java. Maybe that’s something we still want?
Also recursion is broken in current Java implementations, where the stack tends to overflow. But sooner or later they will fix that without us having to do anything.
Can the clojure approach of just sticking the right thing on top of Java without changing the guts of Java work? Well it certainly works in some ways.
I am not saying Java is THE platform of the future, just using it as an example of something that is universally available and not totally different from “Lisp.”
So I see two concrete ways to talk about a new Lisp. One is to start from something that exists and is successful (such as Java) and say what we would need to add to it, or subtract from it, to make something we could be happy with as a new Lisp. The other way is to start from Common Lisp (or Scheme) and say what we think is wrong with it. Again, what would we add and what would be subtract? Or there is the more difficult approach of starting from a clean sheet of paper.
But the hardest part of all is agreeing on what we want a new Lisp for. Without that, we can’t agree on how to decide what features are good and what features are bad. Maybe universal agreement is a chimera. Maybe the focus should be narrowed to just do what would help ITA Software be more productive and successful, and then maybe throw that out to the world to see if anybody else wants to use it too.
June 9th, 2009 at 5:57 am
@Ken, if you have criticisms of the new dialects, by all means voice them. Just do it politely and I’m sure that someone will be happy to give you a clear response about why they agree or disagree with your suggestion.
June 9th, 2009 at 6:01 am
@Dave Moon, regarding your first post, I agree 100%. Your memory is entirely consistent with mine: we made it up as we went along. We were quite cavalier about adding new features. One day I thought, hey, Multics’s ioa_ is great, so let’s have one for Lisp, and I wrote and installed “format”. (It was simpler at that point than it is now, of course). There was nobody there to stop us. We did try for an aesthetic of cleanliness, which I learned from you; my memory is that your taste and oversight deserve the primary credit for what we did right along those lines.
June 9th, 2009 at 6:14 am
@Dave Moon, regarding your second comment:
Regarding “What do we want from Lisp that Java and C# don’t already give us”, well, we certainly want a REPL! You mention the lack of Java’s ability to redefine things. I think it’s more profound than that. The whole concept of “redefine things” is out of the scope of Java’s fundamental concept of what it is to be a programming language. It has no REPL, and cannot redefine things, because it is not thought of in terms of what we’d call a “Lisp world”. That’s a huge difference, in my opinion.
I’ve been trying to answer “What do we want from Lisp that Python does not already give us”: I have been writing a detailed comparison of Common Lisp and Python, which I really ought to finish up, for exactly that reason. Doing that exercise for Ruby would be just as valuable, but I am not as ignorant about Python as I am about Ruby.
The comparison is about language semantics and largely ignores performance. However, the two are not as separate as one might at first think, since there are certain kinds of extremely dynamic behavior that are very hard to implement at high speed. Python has some of these, I think; I want to be quite sure before I actually make that claim. In the real use of Common Lisp, we give up some dynamism in order to allow speed. For example, we don’t use “eval” explicitly. (That’s the only example I can think of but there might be others.) And perhaps that’s what will happen to Python, but it’s too speculative for me to comment further.
To me, Clojure takes neither of the approaches you suggest. I certainly don’t see it as “starting from Java”; the language has little in common with Java. Clojure abandons back-compatibility with Common Lisp and starts all over, asking what we’d do if we started again knowing what we know now. As you say, we’d want ‘contains’ to work on arrays; that is, we want much more generic-ness of functions. This is all orthogonal to the issue of running on the JVM as an implementation strategy, and to the issue of making the JVM libraries available.
The number one reason we want a new Lisp for is to get major benefits of good new ideas in languages that cannot reasonably be added to Common Lisp, while keeping the unique and powerful ideas of Lisp. There are several others, but that’s the one to use as the guide star.
June 9th, 2009 at 9:26 am
@sparkane: I disagree about that. One of the main points of this post is that Gabriel actually made his “worse is better” point about Unix. The “worse is better” concept has very little to do with Lisp, despite the fact that it was originally embedded into a paper about Lisp.
Of course C and Unix were not seen as “worse” technologies. As I keep saying, you have to read Gabriel’s paper to understand the way he is using the phrase “worse is better” and the word “worse”. In fact, “worse” is not a good word to describe Gabriel’s actual point! It’s a great attention-getter, of course.
About “worse is better” meaning “release early, release often”: I don’t think the two are identical, but you have a very good point about there being a close relationship between the two concepts. But they do differ; again, read Gabriel’s two four-point descriptions of “the right thing” and “worse is better”. A very key issue is the tradeoff between simplicity of interface, simplicity of implementation, and correctness. The philosophy of “release early, release often” does not address that tradeoff as part of its fundamental definition.
June 9th, 2009 at 1:36 pm
About redefining things and read-eval-print-loop: I didn’t say you can’t redefine things in Java, just Java doesn’t have a standard built-in way to do it. It is a function of the development environment not of the language. The state of the art development environments (Eclipse, Visual Studio) for Java, C#, and C++ let you redefine things, and also have an awkward form of a read eval print loop. The problems I see with these compared to Lisp are:
1) bugs. Too often you try to redefine something and get an error telling you to terminate the program and start over. Maybe some day they will fix these. The bugs are presumably a symptom of the feature being tacked on rather than designed into the language.
2) The asymmetric structure of these languages makes it impossible to do a good read eval print loop. You can eval expressions, but unlike Lisp not everything is an expression. In Eclipse for some reason you have to edit the expression you want to eval into an existing source file if it’s not already there, eval it, then undo the edit! Presumably that’s so it knows the context where you want it interpreted, but it’s awkward.
Anyway I was really trying to suggest questions rather than getting into a critique of Eclipse.
The high order bit is what are your goals? And the overflow bit is who gets to decide?
June 9th, 2009 at 4:40 pm
long post.
quick deployment, initial code base and extensive community(business/academics/hackers) support are the key for the rise of c(unix), perl(unix, awk, shell, web), ruby(perl,rails), c++(unix,c),java(os-integration, ide, jars, web).
“worse is better” directly supports the first two. community support is however vital in bringing in new speakers/new code for the language.
currently common-lisp fails at all three.
scheme seems to have a bit more community support.
clojure is better at the later two.
lisp will never be quickly deployable chiefly becoz of the syntax and “the right way”.
syntax because its way too foreign(recursion, everything’s an expression, PREFIX, new names, no return, … ).
currently infix syntax and obj.method syntax is very popular. 75% natural languages use SVO and SOV. this suggests to me that our brains are simply not tuned for thinking in prefix.
for example a sequence of 10 method calls when read out loud seem more intuitive than a sequence of 10 function calls.
parens becoz they are distracting(indentation syntax … i make them grey in my emacs).
“the right way”
(personal opinions)
calling something “the one and only right way” is utterly irritating for me. i think its inside our heads. (consider your childhood/teenage. when were we ever going in the right way ??). i think the same applies to languages. “worse is better” is always more humble and begs us to extend the bad way to better and best. by calling something the right way or the wrong for that matter, one is trying to preach and when was the last time we liked preaching ??
frankly speaking calling something “the right way” is a mind-block.
i honestly think lisp needs to follow the middle way. it needs to support more wrong and dumb(easy) constructs if it needs to attract the masses. it needs to support goto, iterators along with continuations. immutable and mutable together, its needs to support ugly infix together with prefix, repl along with traditional compiling. in short TIMTOWTDI. and not just from the designer but from the community.
(look all these can be/are implemented in lisp. but that fact it is discouraged must tell you how much of a mind block “the right way” is.)
and frankly the lisp community should not sound upset when some one uses another language. this probably sounds like perl, but it is actually more like dylan. perl however is the only language that gets philosophy right.
June 12th, 2009 at 4:37 am
@Dan: How about starting with Dylan then? What would CL programmers need to change in Dylan to make it the New Lisp? As I read through the article and the comments (particularly Dave Moon’s 2nd) I can’t help thinking it’s mostly already been done.
June 12th, 2009 at 7:00 am
tom, on the topic of “the right way” — actual data doesn’t seem to support your claim: see Python’s motto “there’s only one way to do it” (something in those lines) and Rails “convention over configuration”. People actually strive for “one true way”, but it should be properly given to them (marketed, as some might say). On the contrary Lisp, IMO, always followed the middle way (at least, in most aspects I care).
June 12th, 2009 at 2:35 pm
Just a brief rant about a small example of the philosophical divergence that came to my attention today.
Doesn’t it stand to reason that block comments should nest, so you can comment out a chunk of code without having to tweak the delimiters for block comments within said chunk? It’s clearly TRT. But the WIB design of C said no, we’re not going to bother, and C/C++ users have gotten so used to that that I can’t even persuade my colleagues that it’s an unfortunate choice that should not be copied in new languages.
So, while I understand Moon’s point that a lot of what became Common Lisp was “just hacked up”, I still think it was hacked up from a different design philosophy than that from which C and Unix were hacked up; and I think that block comment nesting is a nice, clear, albeit small, example of that.
June 13th, 2009 at 12:11 pm
> I’ve been trying to answer “What do we want from Lisp that Python
> does not already give us”:I have been writing a detailed comparison
> of Common Lisp and Python, which I really ought to finish up,
Would love to read this, though I’m also curious how Scheme compares as well.
June 14th, 2009 at 8:27 pm
Tom writes “perl however is the only language that gets philosophy right.”
Well, that settles it, worse really is better! Or perhaps Tom is saying that the best philosophy is no philosophy.
I am not bigoted, I have used Perl successfully. I can write it, I just can’t read it.
Let’s not hijack the thread into a Perl discussion. Probably the important lesson for Lisp from Perl is that making it easy and concise to do the things a particular community wants to do a lot is much more important for success with that community than having a beautiful language design. The technical characteristics of Lisp ought to be very consistent with making it easy and concise to do things, and should not require making programs unreadable to accomplish that. I admit keeping concise programs readable is very hard.
Getting back to worse is better, it’s all about worse in what specific dimension and better in what specific dimension, isn’t it? It’s not a paradox if something is worse in one dimension and better in a different dimension. Abstracted of details Gabriel’s argument is that if a system designer only looks at one of the many dimensions of goodness, and optimizes that at the expense of other essential dimensions, then the result will not be successful overall. At a slightly less abstract level he is saying you can’t be successful if you overengineer and undermarket. Can’t argue with that. (Here by “market” I mean everything connected with who will use your thing and how and why they will use it. This isn’t necessarily just the things marketing professionals say they do.)
June 22nd, 2009 at 11:18 am
Overtaken by a wave of nostalgia, I remember meeting Dan on my first day hacking around the lab. I walked into the 7th floor, a 21-year old freshly back from my music-playing 3-year stint in California, and eager to be around some computing devices. Prior to the lab, I had programmed on TRS-80s and Apple ][s. Sitting in front of this ridiculously tall black-and-white screen, covered in piles of parenthesis, I saw this guy type something in that looked like plain english: “(draw-line-high ‘left-foobar)” or something similar. Then, much to my jaw-dropping amazement, the TV set to the right of this crazy monitor suddenly had the circuit diagram on it change colors! I couldn’t believe what I was seeing.
About two weeks later, I asked some bearded guy (Alan Bawden) about this programming language called lisp. He stopped what he was doing, and gave me a 2 hour lesson, at the end of which, I understood more about programming than I had ever understood before.
For those of you that remember me as “the suspicious looking guy hanging around a CADDR on the 4th floor”, I thank you for an incredible experience in innovation and imagination, and I thank you for the attitude that allowed me to learn at a rapid pace in a rich environment.
It worked out well for me and others – I worked with Patrick Sobolvarro and Leigh Klotz at Terrapin, writing the Apple //gs version of Logo, wrote an Emacs clone for the Apple with M-x completion and dynamically loadable libraries, and then worked with rms as his first employee, where I wrote Bash, TeXInfo, and other stuff.
But my favorite experience in the Lab was using Danny Hillis’ AAA60 while he wasn’t around, and asking Hal for help with Emacs. “Emacs is very helpful,” he told me. Just type “M-x apropos”. Unable to find the “meta” key, and unaware of how “apropos” was spelled caused me frustration – but I know those things now!
July 2nd, 2009 at 9:22 am
@Dave Moon:
Saying that having a read-eval-print loop is “a function of the development environment, not of the language”, while almost right in theory, is a lot less relevant in practice.
First, as you say, there really are features in languages that are problematic for the whole REPL concept, such as the distinction between expressions and statements. Lisp also has some features that are mainly there to help with interactive development, such as the print-object method in CLOS. Good introspection features also help support this kind of environment; Java actually does have good introspection but Haskell does not. (Alan Bawden says there is something called “Template Haskell” which is the closest you get to introspection, and is Haskell’s answer to Lisp macros, but it’s not part of the standard.)
Haskell has a more serious problem: the semantics of the program depends on the order in which the top level forms are written, because there is an implicit “if-then-else” structure, and if you change the ordering of the clauses, the semantics of the program can change.
If you are at a Haskell REPL and you type “23″, what happens? It is the job of the Haskell type inferencer to decide whether that’s an Int, an Integer, a Double, or a Float, based on what type the program is expecting. But in a REPL, there isn’t any expectation.
Any one of these things could presumably be addressed, but my point is that there really is a link between the language definition and its suitability for a REPL-Lisp-style development environment.
Furthermore, Alan says that the semantics of a REPL in Haskell is up to the implementation. I guess there’s an interesting question of whether REPL semantics should be considered “part of the language definition” or not. In Lisp, because of the way we heavily use interactive development, it really ought to be, and mostly is (although the spec could be improved). eval-when, for all its faults, at least shows that the language designers were making some attempt to deal those kind of semantic issues.
But Haskell aside, I don’t think it’s just happenstance that nearly all users of the Java language and the C++ langauage do not use development environments that are oriented around REPL’s and Lisp-style interactive development.
In the early Java days, there was a product, whose name I no longer remember, that had a pretty interesting programming environment of this sort. There also was once “Saber C”, a C implementation that provided that kind of programming environment. But you don’t see a lot of use of things like this, and I don’t know of products that you can currently get of this kind for Java nor C.
July 2nd, 2009 at 10:27 am
@Daniel Weinreb: “Haskell has a more serious problem: the semantics of the program depends on the order in which the top level forms are written…”
Of course, Common Lisp has the same problem: macros must precede their use points, or it looks like function calls to nowhere. And functions the macros use at expansion time need to have been loaded. And I remember headaches with symbols interning into the wrong packages, but I’m already blissfully forgetting the details — I think that was load-order dependent too?
July 4th, 2009 at 2:34 pm
@Jeremy, yes, there’s dependence on order, but there’s a qualitative difference here. Once your program is inside the Lisp world, the order in which the forms were originally written doesn’t matter any more. You can recompile/reload any function. In Haskell, you can’t do that.
September 5th, 2009 at 2:02 pm
@Daniel Weinreb: Saber became Centerline Software in 1991 to avoid confusion with the Sabre that you’re rather familiar with today. Through 2004 they sold CodeCenter and ObjectCenter for C and C++, and the latter was a fantastic development environment (at least, it allowed me to develop some software at near Lisp Machine speeds in the early ’90s). They got bought by Integrated Computer Solutions and that company’s web site implies development of both products has ended. Ah well, but this market failure surely implies something.