The Failure of Lisp? A Reply To Brandon Werner
Brandon Werner has written an excellent, thought-provoking post on his blog entitled “The Rise Of Functional Programming: F#/Scala/Haskell and the failing of Lisp”.
He currently works for Microsoft, but I think it’s clear that this posting reflects his own opinions rather than being the corporate voice of Microsoft. He clearly has lot of real experience with Common Lisp and knows whereof he speaks, and I take him very seriously. I started to write a comment for the blog, but it got too big for that. So, here’s my open reply to Brandon:
Hi. I am one of those 50 year old men. (Well, I’ll be 50 in January, and I don’t use IRC.) I was one of the designers of Common Lisp and one of the co-founders of Symbolics. I wrote the second Emacs ever, in Lisp, following along as RMS developed the first Emacs (in TECO), when I was a teenager.
I currently work at ITA Software, a.k.a “the 800-pound gorilla of Common Lisp”. If you use Orbitz and ask “how do I get from Boston to Chicago on 10/4/2000 at 2pm …”, we provide an excellent set of choices of the cheapest routes and fares for which seats are available. This program, known as QPX, is written in Common Lisp.
I am working on our new product, an airline reservation system. It’s an online transaction-processing system that must be up 99.99% of the time, maintaining maximum response time (e.g. on www.aircanada.com). It’s a very, very complicated system. The presentation layer is written in Java using conventional techniques. The business rule layer is written in Common Lisp; about 500,000 lines of code (plus another 100,000 or so of open source libraries). The database layer is Oracle RAC. We operate our own data centers, some here in Massachusetts and a disaster-recovery site in Canada (separate power grid).
There are currently a total of eleven implementations of Common Lisp being actively maintained; see my survey. I see you are clearly much more interested in the seven free ones than the four for-money ones. At ITA Software, QPX uses SBCL, and the airline reservation system uses Clozure Common Lisp (formerly known as OpenMCL). There are a bunch of reasons we use different dialects, including historical and business ones that don’t apply to anyone but us. They’re both great. SBCL takes longer to compile but generates better code, which is more important for QPX (totally compute-bound) than for the airline reservation system (TP). Personally, I’d recommend these two, but reasonable people differ.
I am distressed and sad to hear that the community is judgmental and unfriendly to newcomers and thorny and un-inspiring. I have heard this same criticism from other people than you, and at this point I assume it must really be true. My own point of view is, of course, entirely different from that of newcomers, so it’s probably harder for me to see that this is going on. Indeed, to me it seems that people do get answers on comp.lang.lisp and LispForum, and the tone doesn’t seem so nasty to me, usually. Maybe I’m just not “getting it”.
I would truly appreciate if you could let me know more specifically what kind of incidents you have encountered. (Was this mainly on IRC?) Putting people off like that is, in my opinion, rude, unethical, and obviously very bad for Lisp.
Please send me side mail about ASDF being incompatible in the two implementations. It’s certainly not supposed to be, and I’m not getting this from your installation writeup. Thank you. I agree that installation could be made more friendly and beginner-oriented for all these implementations. It should be much more “Batteries Included”. There has been at least one serious attempt to deal with this: Google for “Lisp in a Box” (and Peter Siebel has plans to improve it even more) . But your general point is still right; we need much more of what “Lisp in a Box” is doing. The fact that too little effort has been put into this is part of a greater issue about the attitude of the community; more below.
You say “.. Common Lisp showed its failure as a community by sitting out this enthusiasm …” and I agree completely. In my opinion, this is part of the same issue about the attitude of the community.
Here’s another one you didn’t mention. Look at www.python.org and www.ruby-lang.com. They immediately tell you what the language is and what’s good about it (and they’re attractive). Now look at www.lisp.org. This will be vastly improved shortly, but the point is that it is yet another reflection of the same attitude issue.
The problems are that there is hardly any organized “Lisp community” at all, and that few people have been putting serious effort into trying to publicize and promote Lisp, and get new people involved.
The only companies with a direct financial interest, as far as I know, are the four for-money Lisp vendors. Franz has made some serious efforts, but what I understand now is that they’ve had so much trouble making money from selling Lisp that they’re spending less effort on that, and more effort on developing new products (many built on Lisp underneath). The other companies are basically too small to be effective.
Open source communities, like those around Python and Ruby, have been very effective. Those of us who’d like to see Lisp promoted need to understand more about how new languages have been so successful at this, and then rouse people to undertake the work of making it all happen. Certainly part of that is to provide a friendly community who can help and encourage new users.
I hope we can address these issues and get some real work going at the next International Lisp Conference. It’s at MIT, Mar 22-25; I’m general chair. (There’s not much info at the web site yet, but it’s coming soon.)
Thank you very much!
– Dan
September 18th, 2008 at 11:16 am
Dan,
Mr. Werner has about zero knowledge of Lisp. He wanted to develop a logic in Lisp and got the website, the archive, the Lisp files – but no Lisp code.
Then he complained that nobody was developing it for him. He has zero Lisp experience. Writing a description logic is not that easy and I know people who did it – but Mr. Werner is obviously not among them. He is a complainer – but not a contributer.
September 18th, 2008 at 12:42 pm
That Lispers are unfriendly are a myth.
It’s new times. At the moment a site stackoverflow.com is the most exiting thing. A bunch of Microsoft programmers is using a open wiki/forum site. The interesting thing is that there are many reddit users there (since reddit started the karma trend) and stackoverflow is building on that concept. So many microsoft programmers are exposed to reddit:ers with a warship of paul graham. I’m browsing through that site and I am thinking. ” – My good, there are 1000 – 2000 that are going to be exposed to lisp now”.
Exiting times.. lisp is coming around. I’ll tell you.
Please write about stackoverflow..
September 18th, 2008 at 12:45 pm
I don’t hang out on #lisp. I go there when I have a question to ask, I ask it, I get an answer — usually a sharp one, in both senses of the term — and I sign off.
I hang out on #scheme. The feeling tone is better.
September 18th, 2008 at 3:29 pm
I think people on comp.lang.lisp find it hard to distinguish between a troll and a genuine newbie, when asked the questions what is so great about lisp, I love x? Typically if one has a specific question then it’s answered. I pretty much learned lisp on my own, but using books and started by editing an existing project and created my own project out of it. So it isn’t that difficult to get into it, and it’s quite rewarding too.
September 18th, 2008 at 3:33 pm
I am your typical beginner. It would help a lot of there were some beginner tutorial for lisp at http://www.lisp.org and handy documentation. For useful examples see http://www.python.org/doc/ or http://www.ruby-lang.org/en/documentation/ or http://java.sun.com/reference/docs/
September 18th, 2008 at 3:34 pm
I’m not sure what Werner is talking about either. As a Lisp newb who frequents #lisp every now and then, I’ve noticed that people in the channel are very open to newcomers.
September 18th, 2008 at 3:44 pm
What about up-to-date packages? All I’ve found is FedoraLisp, which hasn’t been updated in 3 years.
September 18th, 2008 at 3:56 pm
> I am distressed and sad to hear that the community is judgmental and unfriendly to newcomers and thorny and un-inspiring.
My advice to you is to repudiate comp.lang.lisp as an element of the Lisp community. One doesn’t have to be an obnoxious wanker to hang out there, but it certainly helps one fit in. I don’t know where else n00bs are supposed to go for answers, but steering them away from comp.lang.lisp is a step in the right direction.
September 18th, 2008 at 4:00 pm
Lisp needs a dictator. The problems you describe come from not having one. Perl has one, python has one, and Ruby has one. I don’t know how to solve that problem, but it has to be solved. Seven implementations springs from its lack. No central website springs from its lack. No cohesive community springs from its lack.
Imagine if the Lisp community were all under one tent. Do you think the main website would be a lot better? Do you think the library system would be better? I do.
Even the attitude issue can be a leadership problem. Larry Wall specifically instructs his acolytes to be inclusive. It’s right there in the Camel book. If anyone’s giving you guff in a Perl forum, it’s pretty powerful to quote Larry Wall to shut them down. Of course, because of that, it never really comes up.
September 18th, 2008 at 4:59 pm
500,000+ lines of Common Lisp code? Wow!
How do you manage it? I mean, what kind of coding conventions and style rules do you guys use to aid in code creation and maintenance? I’m genuinely curious. Any suggestions or recommendations would be very helpful to me.
I have recently become very interested in Common Lisp, and it is currently the only language I want to program in when I have spare time. Do you recommend any way that I can become involved in “cleaning up” the Lisp image?
Great post!
September 18th, 2008 at 5:04 pm
I’ve been involved in Lisp since college, originally using Lisp/360.
I have helped build several things in Lisp, including a language
for rule-based programming, which IBM brought to market, a
real-time robot control language, and a computer algebra
system (Axiom) which is quite large (about 1M “things of code”).
In my current job I have built a 60k line lisp program, with 6k pages
of documentation in 2 years, which is fast by any measure.
In my 37 years of programming I can only point to a few things
which have consistently made me very productive. The first is
Emacs, the second is Lisp, and the third is Literate Programming.
(My symbolics machine was number 4 but I no longer have one.)
Each one was a steep hill to climb but the long term payback is
enormous.
Curiously, lisp-based projects always seem to be the most
interesting ones. Not only have I found the work interesting
but the people are interesting. Both factors have made every
Lisp job most enjoyable.
Tim Daly
September 18th, 2008 at 5:07 pm
You should take a look at the firestorm of support that is building behind Clojure, a scheme(ish) language for the JVM.
I think LISP is back in way because dynamic languages are back. It is quite funny actually how PHP has ushered in the era of dynamic languages. Huge sites doing great things quickly with PHP, then Rails and of course Paul Graham telling everyone how LISP made him rich has definitely added to the buzz.
I think there a couple issues here. One being that the dialects have a tendency to partition the different groups. Programmers from each camp are linguistically separated.
The other being with a language so powerful and the syntax so amorphous in a way, that it takes awhile to learn ‘the way’ of each codebase. Each project will tend to create its own universe, now on top of its dialect.
Take a look at Clojure, http://clojure.org/
September 18th, 2008 at 5:16 pm
@Dan,
I’ve repeatedly run into this idea that CL is dead. Well I’m here to tell you it isn’t.
There is more than just ‘Lisp-In-A-Box’ There is also an Eclipse plug-in called CUSP. This plug-in for me was just the ticket for getting started with CL.
My previous experience was with OO Programming. You know, abstract out your program then try to pigeon hole it into some pattern or implementation detail. Now, with CL my abstraction becomes my implementation.
As an example, I started using lisp-unit for unit testing. Quickly I ran into a problem in that the tool had no way to create setup code that would get used by tests. So I created another layer over lisp-unit, lisp-fixture.
I no longer need to learn a new language every year.
I write a language for every new problem!
OBTW, I think it is quite funny that the only way they can “extend” these static languages is through the use of dynamic functional languages. Examples would include Boo and F# for C#. RUBY and Scala for Java.
September 18th, 2008 at 5:20 pm
Lisp was born way back when machines were simply not powerful enough to run it effectively for real applications, and it’s been fighting an uphill battle ever since against the inefficiency argument. Languages like Ruby, which got on the bus in the (effectively) post-scarcity era of computing resources were (mostly) able to avoid this whole battle. (Although there are plenty who will point out Ruby’s performance problems.)
Lisp has also always been the most aesthetically pleasing language to mathematically inclined and educated folks. (dons flameproof suit)
As such, it’s drawn many of the brightest programmers, and the most iconoclastic; they also tend not to suffer fools gladly. And because they tend to have large amounts of intellectual horsepower, to them there seem to be a lot of fools… and humility counts for a lot when you are considering building a user community.
It’s a culture of extreme intellectual snobbery and lack of tolerance for the less gifted. This is fine if you are trying to keep up standards in the math department, but not so good if you have any interest in taking over the programming world.
Open source has had a lot to do with it, too; as pointed out, Lisp vendors don’t “get it”. If people are free to use a language unfettered by licensing, communities can build useful stuff rapidly; witness the whole Rails phenomenon. But there are good open source Lisp implementations; perhaps with time it will change.
I think it’s wrong to say Lisp has “failed”; however, it sure hasn’t succeeded yet. With notable exceptions, of course, and for varying definitions of “success”.
September 18th, 2008 at 5:47 pm
Could it be that a Lisp is just not the language people are looking for these days? Other languages with more syntax are close enough that the existence of so much syntax is enough extra in favor of the other languages? Personally I don’t get the resistance to the parens. I like them. But a coworker was just telling me how he only knows of people that run as fast as they can from the parens and thinks I’m the odd one for liking them. Imagine missing out on Lisp due to the parens! It seems very superficial. Python seems to have overcome the significant whitespace issue.
It also seems like other languages are taking more inspiration from Scheme than Common Lisp. Should the effort go behind promoting Scheme instead? (I know I prefer Scheme.)
If courting the web development folks is desirable (and I imagine it is as it is a big group) I think a paper book like “Agile Web Development with Lisp on Rails” would help. A paper book makes a difference as a visible milestone. Want to learn how to develop and deploy Rails applications? Just get a book and you have it all in your hands.
It is reasonably difficult to sort thought all the Lisps/Schemes and decide on a particular one and a good implementation and then figure out if that will be best in production. And after choosing you are using an implementation that has a small fraction of the total community because everyone is using different implementations. That doesn’t sit well with the boss. Learning the basics of Lisp is quite easy with the benefit of many good university texts (e.g. SICP) but learning about all the runtime/deployment options/trade-offs etc is not.
September 18th, 2008 at 6:41 pm
If you were a 20-year-old whippersnapper today, not invested in a language/religion, do you think you’d pick Common Lisp, or even a capital-L Lisp at all?
There’s 30 years of cruft, vestigial adaptations to old problems that were better solved by other means. Smalltalk has the same problem; arguably, so does Perl (for roles other than shell++). When you get old and conservative enough, very few people are going to choose to start using your platform on its merit. The horrors of aging!
As a 22-year old whippersnapper myself, when not constrained by requirements I program in Haskell and Python.
I can see Haskell in 15-20 years maybe suffering from some CL-style problems, though I think some leaders in the community are aware of that and are consciously trying to avoid it. Hell, the Haskell community already has exclusively Lisp-ish problems:
The keyword ‘monad’ and all the psychological framing around it, misleading definitions, misguided tutorials.
The preponderance of young people who are intrigued by the language, dabble a bit, and post on Reddit — but don’t do any real projects in it. There’s an aura of received impracticality, “oh i’ll actually implement it in blub”, that I most associate with young enthusiastic Lisp fans and has come to Haskell in a big way.
These are hard problems, but 15-year-old Lisp probably had even harder problems from being banged on by many more non-academic users, not to mention vendors!
September 18th, 2008 at 9:17 pm
Dan,
My experience with the rudeness of the Common Lisp community comes from their dissing of my favorite programming language: newLISP. Some of the comments were quite vitriolic and unnecessary. Attempting to reason with them only led to further ranting. It got so bad that newLISP’s author, Lutz Mueller, even stopped responding to the attacks.
To be fair, it was only a few Common Lispers who expressed their newLISP hatred, but the loudest and most obnoxious often make the biggest impression on others (think Rush Limbaugh).
All languages have their strengths and weaknesses (and no language is perfect), so why spend our precious time knocking the languages of others? Let’s just code!
m i c h a e l
September 18th, 2008 at 10:37 pm
[...] Hacker News: The Failure of Lisp? A Reply To Brandon Werner [...]
September 19th, 2008 at 12:15 am
Clojure hits the sweet spot on so many points where CL has always (and likely will always) been off-by-one for our purposes. Obviously, lots of people find CL useful and practical within various contexts, but I’d hope that there’s room for many lisps these days (along with a healthy CL community, etc).
FWIW, I’ve found the people on #clojure and the associated google group to be super-helpful, and extraordinarily friendly to newcomers (as well as sometimes-confused regulars like myself!).
September 19th, 2008 at 6:49 am
Having been an Emacs Lisp coder for almost 15 years now, I was excited the day that I chose to move up to Common Lisp. Closures! Lexical scoping! Native code compilation! I discovered a wondrous environment that seemed to good to be true. The more I used it, the more I came to wonder how C and C++ ever became successful at all in the world. So many of Lisp’s ideas are just *right*.
So I dove into learning and using CL full force, successfully porting an accounting system within eight weeks that had taken over a year to developer in C++. Nearly three fourths of the code was deleted, as I discovered feature after feature that was just native in CL — and absurdely difficult to do right in C++. Where had this miraculous language been all my life?
But then, alas, reality struck. Mostly in the form of Lisp people — and mostly on IRC. Here was a group of people that apparently did not want my enthusiasm, or my questions. At several points, when persons on IRC were not being plain rude, it started to feel clear that they didn’t want to be bothered by the problems of modern-day computing.
And the more that I worked with various Lisp environments, and the more I visited Lisp websites, I found this theme to be more and more consistent: Lisp seems to abhor the detail of practice. It loves theory, algorithms, concepts — but it could care less about your need to run the same code on Windows.
I remember the day that I suggested a command-line option to make it easier to dump the Lisp environment’s core, rather than using a hideously over-complicated form at the REPL. It was like talking to drunk people: “Huh? Why do you want to do that?” I realized they just didn’t get it at all.
September 19th, 2008 at 7:55 am
My experience with the Lisp community has been wonderful and for the most part, I’ve seen good treatment of those coming to the community to ask questions. Seems to be the polar opposite experience of those who have something to say on the matter.
Responses like the one you’ve posted here are what make the community look good, in my humble opinion.
September 19th, 2008 at 8:41 am
I have not had the pleasure of meeting Mr. Werner, but I cannot imagine that someone with “zero knowledge of Lisp” could possibly have written http://common-lisp.net/project/cl-semantic/installation.shtml.
If you are saying that he’s not an expert on Lisp semantics or whatever, I wouldn’t know one way or the other, but there are many hints in his post that make me expect that he’s quite knowledgeable about programming languages.
September 19th, 2008 at 9:17 am
We built most of cleartrip with Common Lisp to start with. All the folks we trained, about 6, found it difficult to start off but now will not touch anything else. I would say the lisp community, in my experience, is quite helpful but very difficult to approach. Many times you get scolded off if you say something silly… I remember long back (98 I think) when I first visited c.l.l …
I was a complete noob and got bitten badly. But I was very curious about Naggum and I liked the look of the parens so I stuck around. #lisp on the other hand is a very close esoteric community and very hard to get into for casual stuff. The feeling is that you have to be some sort of a guru – I am dead scared to open my mouth there unless there is no alternative.
September 19th, 2008 at 9:46 am
Lally Singh: I don’t know. In fact I had never heard of FedoraLisp; thanks very much for letting me know about it.
September 19th, 2008 at 10:10 am
I find it very ironic that people have mentioned newLISP, Clojure, Scheme and FedoraLisp (which I guess is sbcl or something similar packaged for Fedora Linux)!
No wonder people are confused when there are so many “lispy” toy languages in every corner of the world. It’s fairly obvious that they will show a fragmented community and interoperability problems.
ANSI Common Lisp, if that were what people meant when they say “lisp”, is not that confusing. Sure, it could be better, but using things like ASDF it’s not hard to install them and a lot of libraries.
I think one main problem is that “lisp” is a family of languages. When people say “C” they do not mean C, C++,C#,Java etc. but they do that when they say “lisp”. Imagine the confusion!
Oh, yeah. ALU has its problems.
September 19th, 2008 at 10:11 am
I’m one of those lisp beginners. I don’t think I found the lisp community unfriendly, if anything I think it is enthusiastic. But I do find lisp very hard for newbies, not when learning the language but when trying to actually use it for something. I spent more then a year on and off trying to get UncommonWeb to run on a port different then 8080, and I remember at some point having to edit/recompile an apache module just to get something (lisplets i think?) to work. And don’t get me started on Weblocks installation and pgp… I still don’t know how you can install it without confirming at least 20 times that “yes, I know everything it’s unsigned and comes from a wiki, and bad things will happen eventually”.
All of these btw are *great* pieces of software. The problem is the unnecessary effort in starting to use them.
On a more optimistic note, since this spring I’m a big Clojure (www.clojure.org) enthusiast. It’s lisp, it’s functional, and it has the whole java platform behind it. The installation of its (still very young) web framework Compojure requires just downloading, unpacking and running “script/run”, and you have a running server and an open console.
September 19th, 2008 at 10:16 am
Dan, I could take a manual of an airplane and bring it to the runway – even though I haven’t flown a single mile.
How do you explain that he has a large website, two mailing lists, etc. and zero Lisp code???
September 19th, 2008 at 11:50 am
> No wonder people are confused when there are so many “lispy” toy languages in every corner of the world. It’s fairly obvious that they will show a fragmented community and interoperability problems.
This is in part due to the fact that the prototypical Lisp app described in one of the most popular introduction book (SICP) is a Lisp compiler built on top of a metacircular interpreter…
It is also the sign that current implementations don’t cut it enough to have people use them rather than implementing new ones in their environment of choice.
September 19th, 2008 at 5:29 pm
The way I see it, there are two ways to make lisp more open to newbies like myself:
(1) A great tutorial and superb documentation with examples. I suppose the “batteries included” motto of Python makes it very attractive.
or
(2) Make one more Lisp to rule them all. Start with the bare bones of Lisp and allow developers to download language packages. I suppose I’m taking Paul Graham’s idea of building the language up literally by suggesting that developers should have one central repository allowing them to build up their own language.
September 19th, 2008 at 6:24 pm
Plons: SICP is an introduction book to Computer Science taught at Universities, not Lisp programming. It also contains some other examples, btw..
Recently lots of people liked Peter Seibel’s book Practical Common Lisp. Have you seen that?
micheal, the NewLisp developer provoked it a bit. You have to read his texts. He was a bit too quick to tell the Common Lisp users that their Lisp is old and less fun. NewLisp has some users who like it – enjoy it and don’t blame others.
Quasi: you should be more active. You guys have been doing good stuff. Think about presenting it at the Lisp conference next year.
Radu: that’s true some stuff is hard to use and has to be improved from a user view. There are not that many users for some libs and some require that you understand a lot of Common Lisp. Common Lisp is more complicated then other languages – I guess it could be made more user friendly. I understand your frustration. Common Lisp is currently in rediscovery mode and not everything is rosy. Contributions to improve this are welcome. Enjoy Clojure. The developer speaks at the Lisp50 meeting at Oopsla this year. If you read that piece from Mr. Werner you saw that he doesn’t want others to use LispWorks. I think that’s bad advice. LispWorks is one of the best Lisp implementations. Those who can afford a commercial Lisp should have a look. Rich Hickey was used LispWorks, too, btw. Mr. Werner mentioned Allegro CL – another excellent choice. It will take some time to improve the general library situation (usability, robustness, portability, …) – but I think there was definitely a lot of progress in the last years. The implementations have improved a lot, new users have taken over, … – but more needs to be done.
I also think that comp.lang.lisp got more friendly in the last years. On IRC there was also a channel for newbies – but that is a bit silent. There are some newbies coming to #lisp but it is not necessarily the best place to ask beginner questions. It also helps to listen to the advice. Some newbies have difficulties unlearning old habits or come with some strange ideas. The typical question is for a Lisp interpreter. Some then have already difficulties understanding that a Lisp compiler is what most people use and that one can use a compiler interactively. Use comp.lang.lisp, the mailing lists for the various implementations and libraries or use http://www.lispforum.com/. Those who want to learn or do serious open source Lisp development should not hesitate to come to #lisp , though.
September 20th, 2008 at 9:01 am
In response to the comment above about Fedora and Lisp, there is a current project to update Lisp support in Fedora (also similar projects for OCaml and Haskell). Lisp guidelines were approved literally this week:
http://fedoraproject.org/wiki/Packaging/Lisp
September 20th, 2008 at 6:51 pm
Regarding getting help: I see that different people have had different
experiences with the community. Evidently the #lisp IRC channel
is not welcoming to beginners (I don’t know; I never use IRC).
Some people have done well with comp.lang.lisp and some have not. I
just looked at the most recent (non-spam) discussions, and they look
great to me. People asked questions and got useful answers in all the
recent threads. Before that, there is one called “Lisp for begginner”
(sic), in which someone just asked how to get started with Lisp., and
was referred to Practical Common Lisp (which I think is the best
answer to that question) and then SICP. Someone asked, politely, what
the original poster expected of Lisp as compared to Python. The next
few posts were good advice about where to start, where to find a good
Lisp style guide, and so on. After that, the regulars got into
conversations about which book is best, whether the Debian Common Lisp
Controller (whatever that is) is good, more advice to the original
poster, and so on. Can someone show me the most recent comp.lang.lisp
post that’s unfriendly or insulting?
That said, I have in the past seen some unhelpful or even nasty posts.
We need to find a way to discourage these. Maybe a few of us should
be more vigilant, look for unpleasant treatment of newcomers, and
politely but firmly chastise such posts. Snobbery and intolerance (as
Nick puts it) are awful and need to be stamped out.
There’s also a relatively new site at http://www.lispforum.com, which looks
very friendly.
September 20th, 2008 at 7:00 pm
Michael Terry: It’s quite impressive that the “benevolent dictator” system has worked well for Perl, Python, and Ruby (not to mention Linux). It must be a lot of work to do that job, and require a great deal of diplomatic ability and excellent judgment. These folks are succeeding at it; my hat’s off to them.
Unfortunately, it’s very, very hard to see how Common Lisp could retroactively acquire such a dictator. The closest we have ever had was Guy Steele, when he put together “Common Lisp: The Language”. Had he decided to devote himself to being the ongoing benevolent dictator, and had the ANSI process been avoided, that could have worked. But Steele had a lot else to do.
Back in those years, it was a given that in order to get people to accept a language, it had to be blessed by an official standards body. The ANSI process is biased way towards fairness at the expense of efficiency: everybody must have plenty of opportunity to contribute, even if they take a long time to participate, and it’s all a huge committee. And the committee goes away when the standard is all done.
Scheme has an official process to come up with new revisions. Most recently, they produced their sixth revision, known as R6RS (The Revised Revised Revised Revised Revised Revised Report on Scheme). It works by a somewhat complicated committee process (which I think has some flaws). There are several implementations of Scheme, and they’re expected to keep up with the new revisions as the specs arrive.
Common Lisp has remained viable for so long because its extensibility lets it change to keep up with the times. For example, when the concept of object-oriented programming became well-known, Common Lisp was able to absorb it and work OOP into Lisp in an extremely nice way. But there are some things that aren’t easy to add if they’re not already there (efficient Unicode support, good thread support, etc.), and they’re being done differently in different implementations. I feel that this is a serious problem. So is the library problem; in a nutshell, we need a CPAN for Common Lisp.
There are many good ideas floating around about how to deal with these problems. I am hoping to provoke helpful discussion at the International Lisp Conference, and get solutions going.
September 20th, 2008 at 7:44 pm
Clojure looks very interesting. Rich Hickey will be giving a talk about it at the Boston Lisp meeting on Monday September 29, which I am very much looking forward to.
I strongly agree that lisp.org should be thoroughly redesigned to promote Lisp and make it easy to learn, including providing a good tutorial. The folks who own lisp.org (the Association of Lisp Users) agree, and it’ll happen pretty soon.
Yes, it’s confusing that “Lisp” is really a family of languages. For one bright shining moment, Interlisp pretty much vanished and everything else merged into Common Lisp. Oh, except for the widely-used Autocad and GNU Emacs Lisps, but at least those were not in the running to be general-purpose languages. And, of course, except for Scheme, but at least it doesn’t have “Lisp” in its name.
But we’ve entered a new period of language proliferation, a sort of “Cambrian explosion”, in which languages are not just being invented but actually being used. It’s not surprising, in this kind of environment, that new dialects of Lisp are arising. I think we just have to be clear about when we’re talking about “Common Lisp” specifically.
September 21st, 2008 at 9:26 am
As a (forever) newb who has asked a question or two on c.l.l, I can say that if you have real questions, properly asked, you will get excellent answers.
If you want snide answers, try comp.lang.perl
September 21st, 2008 at 1:04 pm
skillet-thief,
Agreed. The reason newbies find the communities so … hostile, is because quite often the wrong questions are asked — fixing a symptom, instead of the underlying, actual, problem.
September 22nd, 2008 at 5:35 pm
speaking as someone who’s done a moderate amount of work with the Lisps, primarily Scheme or Scheme-ish ones, and who is now working with Haskell (and will probably work with the Lisps again in the future), here’s where I think the Lisps have more or less fumbled the ball.
1. No strong primary implementation.
you don’t have to have to have a dictator (Haskell does not, SPJ is at best a revered and respected figure), and you don’t have to have a single implementation.
but you do have to have a very strong, preferrably free/open implementation that’s licensed under very liberal terms (read BSD3/MIT). There are a number of devs and researchers who don’t work on OCaml for the license reason because it has a wonky Q License, and if you’re an up and coming language you need all the devs and researchers you can get.
I don’t have to fuck around with deciding if I want to use SBCL, CMUCL, Clozure Common Lisp, or whatever since I’m on a Mac. I don’t have to contemplate purchasing expensive semi-crippled Lisp packages to get good integrated OS support and all the goodies. I don’t have to switch my compiler suite to something else if I’m working on Linux/FBSD, or Windows.
I don’t have to navigate the huge duplication of effort in the Scheme community where there are probably double the number of credible systems compared to CL.
I just use ghc. In fact, more or less everyone uses ghc. There are other implementations of Haskell that try to do things ghc won’t or can’t (strong embedded support, or highly advanced optimizations) but there are generally strong reasons for branching off, and even if they fail, so what? I still have ghc.
Haskell is also standardized in the H98 and future Haskell Prime efforts. And unlike the ANSI process, it is recognized these standards are not immutably fixed in stone because they’re too damn expensive to update with another standard.
unlike the R6RS effort, Haskell standardization has been generally agreeable to most parties and hasn’t alienated significant parts of the community.
Haskell was more or less founded on a recognition that many researchers were experimenting with lazy functional languages and it was going to end with an unnecessary amount of duplication and result in underserving of that niche, and it’s a lesson everyone who wants to roll their own Lisp for use as opposed to learning should learn.
You get good features and good optimization in a compiler by have a good design that can updated as needs change, and by having lots of really smart people bang away at it for a good long time.
there are just as many smart people in the Lisp communities as the Haskell community, but the Lisp implementing talents are being spread thin.
It would be nice if the Scheme/CL debate could be ended by all parties. Maybe we can all agree to agree around Clojure. But deciding on an agreeable reference implementation for both CL and Scheme would be a major step if harmony can’t be achieved. CL seems to be solidifying around SBCL for free/open support, and asdf has real promise. The Scheme situation looks to stay fragmented and even induce more fragmentation around R6RS support and whether a implementation has decided to go along with it or not..
basically, just _fucking pick something_. It’s not going to make everyone happy but it should make most people happy. People will build new implementations to support special cases as necessary, but most people should be working on a common implementation.
2. No batteries included. It’s not a coincidence you’re using that phrase right now – it’s more or less become a battlecry in the Haskell community, and OCaml is now working feverishly to catch up and remain competitive in those terms.
Haskell is building infrastructure that may eventually put CPAN to shame, and at the very least in it’s current state can compete with offerings from Python and Ruby in quality of experience if not in library size.
instead of writing software that only works on one Scheme implementation or hasn’t been ported to your current CL compiler of choice, you end up deploying software that works for all Haskell users and that, most importantly, other Haskell users can build bigger and better things on.
both CL and Scheme need really good library support RIGHT NOW, and using the current methods are not going to get them anywhere near that goal. Haskell has already leapfrogged them and is shooting towards Ruby-like levels, and I’m quite sure they are already making plans on where to go after getting there..
3. Community Friendliness
This is a big win for Haskell and a mixed bag for Lisp. Scheme is generally more friendly than Lisp since it was used as a teaching language for undergrads. I’m not sure there’s much difference between Scheme and Haskell really in terms of language community friendliness.
comp.lang.lisp should just bounce new posters onto gigamonkey’s site and a reference to PCL and #lisp on freenode with a sign “We’re Closed”. I wrote a rant, but I realized doing so put me on their level, so I’ll just say this -
I think it’s pretty self-evident that if you want people to drink your koolaid you better keep the True Believers locked up in the basement before they scare off the less committed first. It might be full of delicious sugar and s-expressions, but it doesn’t go well with poisonous words.
#lisp on freenode is nice enough. It’s not perfect but new people are treated reasonably well and I’ve felt safe to direct new people there. #scheme is also good. If you want to talk about communities that can get more people into the language, start there. The slow death of Usenet as a credible place for discussing technical topics is probably a hidden benefit for the Lisp language community.
also gigamonkey is a fucking saint and c.l.l people should spend less time arguing and more time getting him canonized and/or funding his projects for making Lisp easier to grasp and more appealing to new programmers, at least if they really want a better Lisp.
#haskell is famously friendly, has almost more users than #lisp and #scheme combined on that network, and has gone so far as to ban highly talented but persistently socially disruptive members that many other online communities would and have simply tolerated as a source of knowledge (this isn’t a dig at Lisp communities, more like evidence that people in the #haskell channel are sticking to their guns about staying friendly).
4. A real plan for massive multicore -
Termite Scheme has one. There are effort to implement Erlang-style parallelism in CL along with STM for concurrency. But Haskell offers an embarrassment of riches for solving these problems, has very good support for them, and is managing to build massive threading systems that make Erlang’s processes look heavyweight.
In fact it’s becoming something of a research testbed for massive multicore.
that brings me to another issue.
5. The research fire is gone
there is not a lot of new research action happening in the Lisps. The main trend in theoretical CS languages research is going toward pure functions, more complex and powerful static typing systems, and massive multicore requrements. Guess which languages this favors?
Someone needs to start beating the Lisps with a electrified cattle prod, getting researchers to start extending and evolving the language in new directions and taking it to new heights. I don’t expect the old school guys to do it, they’ve put their time in and deserve a rest, but if there are no Young Turks who take up their place and start revolutionizing the language, the language is going to increasingly fall into academic disfavor and disuse, with negative results on the language community. Sadly, I also know I’m not the one to do it.
Qi is at least interesting.
in short – the Lisps aren’t dead. And macros ensure that, for certain work that needs that kind of flexibility, they won’t die anytime soon. But they are losing real ground to languages with communities that are not subject to many of the dysfunctions the Lisp community has and have their own brand of power, flexibility, and beauty, and have in theory only been in any kind of practically usable state for much less time than even modern Lisps like CL and Scheme have existed.
don’t take this as a hostile criticism or a insult, because it’s not intended to be. Take this as your warcry as things start get fixed….
September 24th, 2008 at 6:09 pm
“I am one of those 50 year old men. (Well, I’ll be 50 in January, and I don’t use IRC.)” and: “to me it seems that people do get answers on comp.lang.lisp and LispForum, and the tone doesn’t seem so nasty to me, usually.”
A-ha! Here’s the disconnect. Perhaps all the old, nice Lisp hackers who are welcoming to newbies hang out on USENET, and all the smug jerks hang out on IRC. (I’ve had bad experiences on #lisp, too.)
Where’s a newbie more likely to go? As much as I love USENET, “we” (speaking as someone younger than 30) grew up with computers and want instant gratification, and that means IRC. If people are rude to us on IRC, we’re not likely to say “well, it’s probably just IRC; I bet USENET has nicer people”.
I wonder if #lisp can be fixed, or if we should spend our energy convincing people to try out comp.lang.lisp first instead. I suspect the more likely fix is that somebody will come up with a killer library for Lisp (a la Rails for Ruby), and people will flock to the IRC channel for that library, and we’ll just route around the meanness while not abandoning IRC altogether.
September 25th, 2008 at 7:14 am
Ken: You might want to check out http://www.lispforum.com.
September 25th, 2008 at 2:56 pm
i’ve asked on #lisp and c.l.l and got good answer for every good question i asked.
regarding the problem of ‘batteries included’ can be solved with a hint from the text below from Erik:
“the reason there _are_ a bunch of “libraries” and packages for all those ****** languages is that people probably didn’t feel bad about not having something nice, so they went ahead and made something ugly but at least functional. now, everyone’s happy, right? why not repeat this for *Common Lisp*, too? ”
qouted from http://groups.google.no/group/comp.lang.lisp/msg/4bda6a98e5cf0bce
September 29th, 2008 at 8:18 am
I think everyone’s missing the point. Modern systems can (and probably should) be developed using multiple languages, according to the “don’t re-invent any wheels” principle (I’m using Lisp, C++, XSLT, Flash/Flex/Actionscript & XHTML/CSS/JavaScript for my current project – to limit myself to a single language would be stupid). If your project/brief/itch matches the 99% of “real world” tasks, just choose the language(s)/framework(s)/etc. that’ll get it finished ASAP.
The magic of Lisp is that you can use it to make software that you just can’t make in other languages (including those other languages). But most programmers never need to (and specifically don’t want to) create software the likes of which the world has never seen, which is fine. If you want libraries, lambdas and LAMP, use Python, for God’s sake. If it’s good enough for Peter Norvig, it’s good enough.
Worse is indeed better (just look at PHP); but popularity is overrated. I’ve been programming for 20 years, and I can afford LispWorks Pro *and* its annual maintenance contract. (And if I couldn’t, and/or if I wasn’t required to use Windows, I’d be using SBCL, end of story.) I’d love to get more Lisp work and work with more Lispers, but I couldn’t give a wet slap whether Lisp is more or less popular than any other language. As they say, “when the student is ready, the master will appear”; it isn’t that hard to learn Lisp if you put your mind to it. I did, and I’m no Bram Cohen.
October 5th, 2008 at 1:37 am
I have bookmarked this informativ site. Thanks Kfz Versicherung
November 16th, 2008 at 8:58 am
[...] user feedback quite seriously.Cleartrip is also one of the handful of companies in India who have based their software stack on Lisp. That in my opinion is worth some brownie points. They are also the [...]
November 18th, 2008 at 4:15 pm
How do you manage it? I mean, what kind of coding conventions and style rules do you guys use to aid in code creation and maintenance? I’m genuinely curious. Any suggestions or recommendations would be very helpful to me. I have recently become very interested in Common Lisp, and it is currently the only language I want to program in when I have spare time. Do you recommend any way that I can become involved in “cleaning up” the Lisp image?
Great post!
November 29th, 2008 at 2:46 pm
Chris; we don’t do anything too special. If you read Practical Common Lisp, we pretty much follow that kind of programming style. We use Subversion for source control. We have unit tests, and before committing any code, it must be reviewed and unit-tested. Then we have a continuously-running “buildbot”, a server that continuously rebuilds the code and runs a larger (slower) set of unit tests on it, and emails whomever might be responsible. We also have a “performance buildbot” that checks for performance regressions. Every night we have automatic tools that snapshot the whole system, including all the components (only the big business-layer component is written in Lisp), package it up, distribute it to the appropriate machines in a cluster, and run system-level tests over the whole thing. All this automatic testing is extremely valuable. We use Bugzilla to track bugs (and sometimes enhancement requests).
July 15th, 2009 at 11:12 am
From last two days I am searching madly for Future of lisp, and by this time I got this post & really want to post something which will boost masters in this field to do something revolutionary for bringing this old aged language in day to day practice.
Before saying some thing more let me tell you something about me. I am not truly professional programmer, but by accident I have became AutoCAD lisp programmer & now I am full time CAD Programmer. I have been successfully doing programing in Visual lisp, vb6 for last 10 years & recently started coding in vb.net & asp.net. Every time when new assignment comes to me I explore that whether I can do it in lisp, I don’t know what but there is some thing addictive in this language that a non-programmer like me has got habitat of it. I think I have said enough of me & I should come to the point.
I terribly think that creator of this language will be filling shame on our lisp fans community, just imagine before 50 years this incredible language was discovered & still it is competing with top class today’s programming languages. Do somebody need need to tell more about this great language. It is very clear that this language by hook or crook was totally ignored & gods wish it is slowly gaining popularity.
I don’t want to discuss here what gains we get why using LISP & with compare to other languages (there are many post already for this).
Lastly I just want to end my discussion my pleading to all Lisp fans, “OK guys, now its enough of proving how great LISP language is what we can do in it”. Now the time has come to do something extraordinarily which will make John soul somewhat peaceful.
I know I am writing something emotional & insensible here, but let me confess I do not have any good educational qualification , no good programming skills. I only know I love LISP language & I can go to any extent for contributing to it. I know some of my sentence will heartened guys who are really working hard on bringing this language to fame. But don’t take something negative feelings from this. I am just nervous b’coz we guys know how this language is still superior then any of today’s language (even after 50 years), and still we are googling just what is future of LISP.
I think we all need to contribute for this, & big guns in this field (Franz) should contribute more on this effort.
I think I have said enough, let me wait for some feedback from you guys to see what I am saying is meaningful or useless.
April 21st, 2010 at 11:40 am
[...] I just re-found an old characterization of (some of) what ITA Software does by — who else? — Dan Weinreb: I am working on our new product, an airline reservation [...]