Dan Weinreb’s blog

So many people have written about the MassTLC Innovation Unconference that I’ll just record my personal impressions.  My own blog comments are in [square brackets].

I was one of the so-called “experts”.  I originally attended in my Common Angels role.  But then Jeremy Wertheimer, CEO of ITA Software, Inc. and my boss’s boss, also attended as an expert.  So I introduced myself in both roles.  James Geschwiler, executive director of Common Angels, was one of the main organizers of the Unconference, and I counted eight members of Common Angels in attendance, although all but James and I came under some other affiliation.

For more info, see the coverage in Scott Kirsner’s Innovation Economy, Mass High Tech, and The Boston Globe.  (There’s a mistake in the Globe article:  Jeremy, did not say “It’s greed on the way up, and it’s greed on the way down”; the second half was “it’s fear on the way down”!

Whenever I have the chance to talk to a college freshman, I advise them to find out which professors are the best lecturers, and take whatever courses they are teaching, rather than browsing the catalog and choosing based on topic.  I followed my own advice and picked sessions based on the speaker.

Bootstrapping, by Sim Simeonov of Polaris Ventures, and Joe Caruso of Bantam Group and Common Angels, was about how to start a business without venture or angel capital, and then if or when to seek out such capital. Sim helps start and work with new companies at Polaris; Joe is on the board of Common Angels.  They both know a tremendous amount about startups.  Sim and Joe mainly let the audience tell their stories and give their opinions, and then they added comments, often of the form “but on the other hand”, to clarify tradeoffs.

Here are some of the most salient points.  First learn what the customer wants, and then think up the product.  Having too much capital can lead to over-hiring and over-spending; if you have the money, you’ll spend it, whether that’s optimal or not.  You should have a very good idea of how you’re going to spend the money!

When you have investors, you’ve taken on business partners; some people feel that they are working for the investors, when they really wanted to be their own bosses.  The reply was that investors do not want to run companies.  Often a CEO is so busy and nose-to-the-grindstone that she can’t see the big picture and think about long-term strategy; a good board of directors can help.

It’s very important to know yourself and understand your personal goals.  Some people don’t want to run a fast-growing business of the kind the attracts venture and angel capital.  Do you want 2% of $1B, or 20% of $1M?  Would you be satisfied with enough money to be financially independent, or do you want more?  Or would you really rather be doing engineering?  There are 10M small businesses in the USA; every year, 3K get VC funding.  For many people, simply getting a good income is just fine.

Everything has an opportunity cost, and you must take that into account as part of your tradeoffs.  Start with your exit plan in mind (how you’re going to cash out, e.g. an acquisition or an IPO); these days, $60-70M is an average exit.  But typical venture capital firms can’t deal with that, because it’s too small for them to devoted attention to, as they have very big pools of money to invest.  If you’re going to deal with venture or angel capitalists, understand their business model.

One way to bootstrap is by sales to customers; in fact that’s probably the best way.  You can’t do that with a high-capital business such as starting a car company.  If you have a track record, you can try raising money from former business associates, as my friend Jules Pieri is doing.  Finally, there’s the traditional way: “friends and family”.

You can start small, conserve money fanatically (except salaries for non-founders), and not accept capital.  That works best if there is no competition and you can take your time.

Sometimes the time to accept capital is when your business is running, but you can see that now your market is growing and there’s an opportunity.  But you need to hire more engineers, marketing and sales people, and so on.  [This is what we call the "go to market" stage, and is typically where Common Angels would invest.]

One attendee recommended reading Paul Graham of Y-Combinator.  [I know Paul through the Lisp world, which Paul is designing a new language called Arc .]  Paul is a great writer, and there’s a lot of good advice in his collection of essays.

If you accept capital, and your company grows enough, your board of directors might want to replace you as CEO with someone who has experience dealing with a company of the new size.  Joe pointed out that while you may not like this idea, it might be a very good move.  Two of us recommended the work of Prof. Norm Wasserman of Harvard Business School, on the subject of “Rich or King?”, the tradeoff between retaining control of your company versus making the most amount of money.  Prof. Wasserman gave a great talk about this at the Business of Software conference, and you can read his paper.  See also his blog, Founder Frustrations.

The most important factor for success is to have customers who love your product.

Mobile Technology was discussed by Rich Miner, who started Android and sold it to Google.  He’s now the head of Google’s Android project, which is creating an open and portable platform for mobile devices.  Vanu Bose of MIT explained about the various forms of wireless technology, such as CDMA, GSM, EVDO, UMTS, WiMax, and many others.  This session was more of a lecture with questions.

I know relatively little about mobile technology, and I learned a lot.  Here are some of the high points, at lesat for me.  This is an exciting, “tipping point” time that has been anticipated for years by those in the mobile industry.  Finally there are mobile platforms with big screens, touch input, simultaneous voice and data, and good network performance.  Carriers are offering unlimited data plans.

This allows a wide variety of useful applications to be written.  Obviously there is now the iPhone and its highly successful App Store, and soon there will be Android phones with the same kind of abilities and store.  Miner pointed out that there were 1.27B (!) mobile phones sold this year, but there are only 10M iPhones, which is a very small fraction.  [But is he counting all mobile phones, including the vast majority that don't have the big screens and cannot run these apps?]

Nokia has bought Symbian, and will turn over their IP to the Symbian Foundation, a non-profit whose purpose is to advance the Symbian platform. Miner stressed that having only one platform, owned by one company, is highly undesirable.  [Gee, I wonder whom he was tacitly referring to?]

It was commented that any country with four or more dominant carriers get many benefits from the competition between the carriers, who are highly motivated to respond to customer demands, and innovate.  The USA has only three dominant carriers now, but two more are coming (one from Craig McCaw ) [I did not get all of this].  There’s something new being tested in Baltimore that’s very promising and working well [was this about Believe Wireless or Sprint's WiMax?  Sorry, Miner was talking quickly and I am not well-versed in all this.]

Vanu Bose explained that the most successful carriers were the ones who used lower frequencies, since they have a higher range and therefore need fewer cell towers.  AT&T and Verizon control the 700MHz and 800MHz [that's considered "low"], forming a duopoloy [which means little competition].  These frequencies are particularly valuable in large cities, because they go through skyscrapers, rather than being blocked by them.  There is always a tradeoff between megabytes [data rate], capacity, and coverage.  [In the context of mobile phones, the word "bandwidth" refers to an amount of space in the electromagnetic spectrum, not to data rates.]

UMTS theoretically can do data rates of 3M downstream and 1M upstream, but it’s relatively new, whereas GSM has been optimized for years, and tests show that UMTS is actually far slower, like 200K.

The reason that wired networks operate according to one standard is that they are carrying so much data so fast that there is no room for improvement; they are very close to the Shannon limit.  Wireless isn’t there yet, so there’s competition to do better by using new proprietary techniques.  We can’t expect better standardization until that process reaches its culmination.  LTE, for all practical purposes, does not exist.  WiMax really is being built out.  [There was a lot more of this kind of thing.]

But application writers need not study and master all this.  What matters to them is the software platform.

For voice communication, research shows that users care more about latency than data rate.  [That was a surprise to me; I rarely hear about latency figures!]

Android lets you download apps from any old web link, not just from their store, although of course you have to first discover the web link.  YouTube is so successful because it’s the only place you go for videos, which makes it easy to discover them, and the Android App Store will be analogous.

I asked how you get your apps into the Apple iPhone App Store, and what you pay for this.  Miner said that first you have to apply to be a developer, but very likely you won’t have any problem.  Then you submit your application, and “the App Store gods cogitate for a few weeks; nobody knows what criteria they use.”  Finally, if you are charging money for the app, they take a 30% cut of all revenue. [Also, they have just announced that they're removing their non-disclosure ageement requirement for developers.]

I know that 30% is the cut they take for music bought through iTunes.  It seemed to me like a large amount, but everybody pigpiled on me saying, no, it was very reasonable.  Distribution costs for software are usually high.  And book authors only get 10%.  I was not convinced by those arguments.  Who cares about any of that?  The point is that it’s very cheap to distribute software to a mobile device, and what value is the store really adding?  If I were able to set up my own, competing store, I could easily charge less than 30% and still make a tidy profit.  The answer to that is: you can’t, because Apple makes it technically impossible (for unhacked iPhones).  Well, doesn’t that mean that Apple is charging a monopoly “rent”?  With Android, would it be possible to set up a competing app store?  By this time the conversation had moved on, so I didn’t get an answer.  If so, that might be a business model for a startup, although I suppose Google would just lower their price to match.  That would be good for everyone, except the startup!

Will you be able to do advertising-supported apps?  Miner thinks it’s possible but points out that it’s hard.  There isn’t much screen space.  The ads really must be ads that the user genuinely wants to see, because they’re relevant and useful.

Does Google make money from Android?  No [by which I take it he means, not directly].  It is strategic for Google, to avoid someone from owning the platform as a monopoly.  It keeps the playing field level.  Google can offer non-free applications, but you can compete with Google on a fair footing.  Chrome (the new Google browser) exists for the same reason.  The Android browser is as good as the iPhone one; they’re both based on the WebKit technology. Local applications will have some important advantages over browser [Software as a Service] apps.  Android will not have Flash support initially, but he thinks he’s heard that Adobe is working on it.  There will be far more web-capable mobile devices than home computers, and we’re already seeing companies maintain two different web sites, one aimed at each.  In the long run, the web (at least some of it) may be more addressed to mobile devices than to home computers.  There will be opportunities for companies to be publishers, a la Broderbund and Electronic Arts.  There will be so many apps that you’ll learn about them through major marketing campaigns.  [It sounds to me analogous to "labels" in the music industry, which have pros and cons.)

Selling was a lecture by Bob Metcalfe, famous co-inventor of the Ethernet, and CEO of 3Com, and now a general partner at Polaris Venture Partners.  He drew a large crowd.

Much of what he said comes from his 1992 Technology Review article, Zen and the Art of Selling, so I won't repeat all of that.  Much of his talk was oriented around direct sales, i.e. when you have real live salespeople going out to specific customers.  [Of course, there are many other models.  My impression is that direct sales is most appropriate when each sale tends to be for a very large amount of money.  Salespeople are very expensive, so direct sales must only be used where it's highly appropriate.]

Engineers must learn to have respect for salespeople; many do not.  The two cultures are very, very different, and being a great salesperson requires a great deal of expertise.  [I first saw these cultural differences at my first company job, at Symbolics.  My friend Bill York went over to the sales side and then came back and explained it to all of us software developers.  The book Corporate Cultures hits the nail right on the head; it's well worth reading.]

When you’re a startup, try approaching a potential customer by saying, very politely: “I’m developing a new XXX.  I’d like to talk to you about what you’re doing, so that you can give me feedback.”  This means that you are not going to make a sales pitch to them, so don’t.  (At least not at this stage.)  Many potential customers are honored by this, and would enjoy talking to you.  The information get is priceless, and it’s also the start of a relationship that might eventually lead to a sale.

When talking to venture capitalists, if you want money, ask for advice.  And if you want advice, ask for money.

Marketing is not sales.  Marketing has many roles, including creating tools that help salespeople (brochures, ads, all kinds of things), generating leads (whom to call on), setting prices, and determining appropriate quotas for each salesperson’s territory.  [I didn't know that quota-setting is (sometimes) done by Marketing; I had assumed that sales management did it.  I suppose these things vary.]

Bringing along an engineer can be good.  The engineer can answer detailed technical questions, and talk to the techies if there are any around.  However, they often feel that they must talk about every feature in the product: “Oh, one other things, I’ve got to tell you about our great …”.  After all, the engineers worked hard on that and are proud of it.  But a good salesperson focuses on the benefits relevant to the customer, and doesn’t waste time on anything else.

Often you must talk to and cultivate more than one person at the account (the potential customer).  You need to learn how the customer’s company is organized, who makes which decisions, and who has which agendas.  [A good example I recently heard: the IT department might not like buying a SaaS product that will make IT less important in the company.]

Eventually, you must ask for the offer.  This can be emotionally difficult, since it’s only natural to be afraid of hearing “no”.  But a good salesperson thinks of “no” as a starting point!  You say, “why not?”, and proceed from there.  One way to ask for the order is “So, would you like this delivered next week or the week after?”

Sales training courses really are good!  If you want to sell better, find a good one and take it.

Credibility matters more than anything else.  The customer is very unlikely to buy without trust.  First you must establish credibility for yourself, then for your company, and then for the product.  A good way to establish personal credibility is to make little promises and keep them.  By little, he means “I’ll meet you at such and such time”: do not be late!  Or “I’ll mail you that tomorrow”: do it!  Even these little things are very effective.

It’s great to get to the customers before your competitor does.  Metcalfe was snookered by an Interlan salesperson who would always get there first, and leave them with five challenging technical questions to ask when Metcalfe showed up!

That was the end of the lecture, and then participants spoke.  People recommended Seth Godin’s book Permission Marketing, and HubSpot.com for their concept of Inbound Marketing.  [I liked Seth Godin's talk at the Business of Software conference and learned that he's highly respected and has written lots of books that people think well of.  HubSpot's CTO, Dharmesh Shah, also gave a great talk at Business of Software.]

James Geschwiler asked Metcalfe about sales force compensation.  Some of the advice: if there are channel conflicts (i.e. it’s not clear which salesperson, under the rules of engagement, is supposed to get the commission), pay everyone fully.  You want to pay the salespeople very promptly; their culture is based on quick turnaround and instant gratification is important.  But what if the customer never ends up paying?  You could delay paying the commission until you actually get the cash, but that takes too long; don’t do that.  Your CFO must make sure that customers are creditworthy, so that if the sale is booked, you aren’t worried about the receivable turning into cash.  Then you can pay the commission when the product is booked, or, sometimes, when customer acceptance happens.

James: What about sales compensation for SaaS (Software as a Service)?  Metcalfe: I don’t know about that.  James: what proportion of the salesperson’s compensation should be a base salary and what from commissions?  Metcalfe: 60/40 is common, but 40/60 can work if you have “swashbuckling egomaniacs”.  Never put a cap (maximum) on commissions; in fact, raise the commission rate after the salesperson reaches the quota.

Metcalfe added that when you hire a salesperson, see whom they have sold to in the past.  They’re bringing all those relationships and contacts with them.  That’s a salesperson’s long-term value: he or she cares more about them than your company, since salespeople often switch companies (they have the highest turnover of any category of employees). [I've seen that; when it looks like the company is heading toward trouble, most of the engineers stay to try to fix it, but the most of the sales force takes off for greener pastures.]

James: What do you think about non-compete agreements?  Metcalfe: For a long time, I was in California, where they are prohibited.  Now I’m a venture capitalists in Massachusetts, and I think they’re great.  [When my brother was at Harvard Law, I asked him about these, since I had been asked to sign one.  He found a law professor who had just done a study about non-competes in Massachusetts, and found that the only cases in which they have successfully been used against an ex-employee was against direct salespeople who were taking away accounts.]

What if your product breaks?  Remember, customers expect this to happen.  What matters is what you do next: you should be all over it!  Do whatever it takes to make it right.  If you rise to the occasion, and fix it fast, they’ll love you.

Hannah Burr had a small group of us discuss “what do you to to keep your life from turning into chaos?”  The group shared hints and techniques.  Now I know some good books to read, website to check out, and tools to use.  I’ll add all of these to my things-to-do list, which is so long that my life is turning into chaos.  Check out Getting Things Done, Unclutter, 43Folders, Lifehacker, remember the milk, OmniFocus if you use a Mac.

There were lots of little ideas.  One interesting one: Have an outsider look over your to-do list and help you consider how to manage it.

During lunch, entrepreneurs got together in small groups with the experts to talk about whatever they wanted.  I told stories and passed on what useful facts I could think of.  I spent the second half of lunch talking to Kimberly Patch of Redstart Systems, a startup using voice-recognition technology to make it very easy and efficient to use computers by voice command.  This is particularly useful for people with disabilities, such as carpal-tunnel syndrome.  I’ve had several friends who were forced to take leaves of absence or even give up their careers entirely because of repetitive stress injuries; this is just what they needed.  With this system, you can even do some things faster than with a keyboard and mouse!  Again, I offered what useful advice I had, and discussed funding opportunities.

Bill Warner, with James Geschwiler, designed the event.  Tom Hopcroft and Heather Johnson of MassTLC did a lot of the work and deserve most of the credit.  Their organizer, Kaliya Hammond, also did a superb job.  Congratulations!

I had a great time at the European Common Lisp Meeting (ECLM) in Amsterdam, April 19-20, 2008. I met many of the important people in today’s Common Lisp world, an almost completely generation set of folks as compared to 15 years ago. The papers were excellent, and demonstrated that Common Lisp is still a vibrant and uniquely powerful language. (I’m writing this on the plane back home, on my OLPC laptop, as I learn to touch-type with my big hands on the child-friendly little keys.) Arthur Lemmens and Edi Weitz did a great job organizing and running the meeting. Everything went entirely smoothly and I felt that everyone enjoyed it very much.

InspireData
Jeremy Jones of Clozure Associates demonstrated InspireData, an educational application that lets you analyze data and draw conclusions. The user interface and interaction design is superb. It performs very well, even on weak, old PC’s, which matters since that’s what many schools actually have. It has gotten excellent reviews and sales, and is widely used in schools.

The user would have no reason to know that it was written in Lisp. No Lisp is exposed to the user. They used LispWorks, since it runs on all of their target platforms (including Windows 95, as well as more modern Windows and MacOS X), it has a good interactive development environment, it provides a portable library for accessing the platform’s native menus and other widgets (called CAPI), and had favorable licensing terms. The rest of the graphics was done by calling the OpenGL graphics library, using a library from LispWorks. They found all of these technologies to work very well.

They wrote it as a contract job, building it to specs provided to them in a 200-page document. It took 8 person-years of developers plus two person-years of Q/A, who were brought on board from the very beginning. It’s 270K lines of Lisp code plus 470 lines of C.

The primary advantage of using Lisp is that they could produce a prototype in only two months, and then do incremental additions and refinements. You might ask, why was this necessary if they had a 200-page requirements document?

• The specs were vague. It would have taken over a 1000-page spec to really be unambiguous. (In my opinion, that’s absolutely normal for software.)
• The specs kept changing. (That always happens. Always!)
• In particular, the designers would change the specs because of what they saw the program doing. In other words, specifying it in advance would have been impossible in any number of pages. Design and implementation must be interleaved.
• Even if the spec were known in advance. the best implementation techniques are not initially appanrent. Sometimes you have to get pretty far along in the implementation to realize that some atrchitectural decision did not work out well.

Lisp is very malleable. Experience over the years has shown that even large Lisp systems are particulary easy to re-factor and even re-architect. (I have seen this over and over again.) In fact, Jeremy feels that they didn’t re-architect enough! (One usually hears the opposite lament.) He emphasized that iterative development — build, test, refine — was the only way to go and the only way they could have succeeded.

LispWorks performance in this application is excellent. As I could see in the demo, it is extremely responsive. Jeremy says he has never perceived any delay from the garbage collector. InspireData is a shining proof that real applications can be done just fine in Common Lisp.

FEMLISP

Nicholas Neuss (IANM, U. Karlsruhe) presented FEMLISP, a system to do finite-element analysis (FEM). FEM is used for solving partial differential equations. It’s used to model things like convection, diffusion, viscous fluid flow, and so on.

I had thought this would just be a numeric library with some API, and wondered why doing it in Lisp would be helpful or make any difference. But it’s not like that at all. First, choosing a good way to run FEM is a hard problem. I only sort-of understand the issues, but I got a sense of it. There’s a big issue of how you “discreetize” and solve the discrete problem. You also must make decisions about how to set the mesh. Second, you want an interactive environment that lets you display results graphically, and make small changes to the input spec and try again. Ideally, you’d like an expert system to make these decisions, but what he’s done so far he described as “rudimentary”.

He created small domain-specific language to represent of how to run a particular FEM problem. This lives in a Slime buffer and can be edited and recompiled quickly and conveniently. It can display the history of the iterations so you can see what’s going on and refine your input. You can insert Lisp code into the input, for computing or debugging.

It runs on CMUCL, SBCL, LispWorks CL, and Allegro CL. He does graphics with OpenDX (Data Explorer), which was written by IBM and open-sourced. (He is considering switching to VTK.)

Why did he use Lisp?

• Dynamic typing worked out well
• Macros and a few reader macros let him make an embedded domain-specific language easily
• Dynamic testing and debugging (read-eval-print loop, etc.)
• High performance compiled code (as compared to Guile and other Scheme implementations that he tried)
• Common Lisp is stable; once you learn it, you know it
• There is no system/user dichotomy
• It only took 30K lines of code
• There were lots of useful libraries

Performance as compared to other available FEM packages is hard to determine for many reasons. For example, who chooses the benchmark? Can you find an informed third party to spec and judge the procedure? How do you know you’re not comparing apples and oranges? (These are all standard pitfalls of benchmarking.) Also, he has not spent too much time on performance improvements anyway.

Nevertheless, he ran some basic comparisons against their company’s in-house FEM system, called M++, not only to measure speed but to make sure he got the same answers (he did). M++ turned out to be faster on small problems, but FEMLISP was faster on larger problems.

One reason for this is extremely interesting. Apparently there is a certain well-known technique for speeding up FEM. He had implemented it, but they had not yet done so. This illustrates the principle that higher-productivity software development can lead to faster performance! When considering the effects on performance of using Lisp, take this into account.

In other tests he found that FEMLISP is about as fast as a leading commercial product (FEMLAB) for comparable accuracy, and much easier to use.

So far he has not tried to encourage other people to use it, mainly for political reasons (his boss wrote M++). He used FFI for certain existing libraries (e.g. LAPACK).

Large Internet Systems

Stefan Richter of freiheit.com talked about “Using Common Lisp for Large Internet Systems”. His company, freiheit.com technologies (it means “freedom”, in the sense not having to use the Microsoft platform any more!) has built many commercial web sites in Java. They have 60 developers, most using Java, but also a 6-person Common Lisp group. In an unusual twist, the manager of the group had to convince the reluctant programmers to use Common Lisp. Also, the clients had to be convinced that accepting a product in Common Lisp was OK. They have delivered one Common Lisp application so far, a social marketing tool.

By “large internet systems” he mainly means scalable web sites. Unfortunately, he has not actually built such a thing in Lisp yet. The talk suggests approaches to the problem, but he did not have actual experiences to report. He primarly prefers Lisp because he feels that Java is too verbose, and Ruby is basically like Lisp.

He explained a lot about how to build scalable and reliable servers (all of which I was very familiar with from my work at BEA and at ITA Software). Clusters, load balancers, stateless app tier, separate DBMS’s transactions and reporting, shared memcached distributed cache, keeping functionally separate data on separate DBMS’s, plus one idea that’s still new or in the short-term future: shared-nothing database clusters using “shards” with replicated data for reliability. All of this is completely right, in my opinion, and I don’t think any of it is controversial.

Java has many good tools for doing such an archtecture: Tomcat providing a framework for servlets/JSP’s, a memcached client, Hibernate for database access, and even Hadoop (a free MapReduce implementation).

How does Common Lisp compare? We have Hunchentoot (a sophisticated HTTP server), cl-memcached (a memcached client), cl-sql (to invoke relational DBMS’s), and two advanced tools for generating HTML: Weblocks (by Slava Akhmechet, I think), and UnCommon Web (by Marco Baringer).

He also suggests using cl-muproc (a library that provides Erlang semantics in Common Lisp, basically) which he feels could be a good basis for a Common Lisp MapReduce. I don’t know exactly what he has in mind here, but apparently he has implemented this.

He doesn’t like existing conventional technology for generating web pages. Servlets clumsily embed HTML in Java code; JSP’s clumsily embed Java code in HTML. Using Common Lisp has many of the advantages of other popular languages that are being used to write HTML generation, such as Ruby, Groovy, and Python. Lisp has major advantages: you don’t have to write out files in order to compile things; CLOS is very useful, including the MOP; we can avoid the need for XML because programs and data use the same format; and of course macros help in all kinds of ways. (And, I was thinking, Common Lisp implementations typically execute code much faster than Ruby and Python!)

He talked about using continuations to save state between HTTP interactions. (Many papers have been written on this topic.) You want to be able to write a program in a normal style, that can say “do this web interaction” in the middle of any procedure; this makes flow of control much easier to understand. A continuation saves stack and execution state across interactions. He talked about Weblocks and how it uses continuations, as well as many of its other virtues (it sounds great, from what he said; I have yet to learn about it).

He feels that what’s needed now is to put it all together, and then write a good book about how to use it. He points out that Ruby on Rails would never have taken off without the excellent book. (I agree completely!) He encourages us to write books, and help develop the framework libraries.

This all led to a lively discussion of continuations, particularly persistent continuations, and how to best implement them for Common Lisp. Weblocks uses the cl-cont library. Marco Baringer said that cl-cont’s continuation states are extremely large, leading to performance problems, although it would not be hard to improve this.

We also talked about just how reliable a system like this needs to be. It often turns out that in exchange for a very small about of unreliability, you can make big improvements in simplicity and performance. On a web site, it’s often quite acceptable to fail now and then, since the clients are human users who are much better at handling failure and retrying or finding alternatives.

PWGL

Kilian Sprotte described PWGL, a tool for music composition and analysis. It is based on an earlier system called Patchwork, by Mikael Laurson in his 1996 doctor’s dissertation, at IRCAM, the famous music lab in Paris. It is ten years old, and has always been in Common Lisp. Originally it was developed in MCL; now it’s based on LispWorks and runs on both Windows and Mac OS X. It’s now being developed at Sibelius Academy in Finland. It’s currently in beta-test, downloadable, and version 1.0 is expected later this year.

According to the description on the web site: PWGL is a free cross-platform visual language based on Common Lisp, CLOS and OpenGL, specialized in computer aided composition and sound synthesis. It integrates several programming paradigms (functional, object-oriented, constraint-based) with high-level visual representation of data and it can be used to solve a wide range of musical problems.

It’s a visual dataflow functional language; in some ways it’s like doing Lisp by drawing boxes and lines.

It uses OpenGL for graphics, the PortAudio library for recording, playing back, and basic sound synthesis, and the libsndfile library for reading and writing files containing sampled sound. (It was interesting to see how many Lisp systems are capable of using non-Lisp libraries easily. This is another important counter-argument to the objection that Lisp has too few libraries.)

Embeddable Common Lisp (ECL)

Juan Jose Garcia-Ripoll described Embeddable Common Lisp. ECL is not just for embedding: it’s a full Common Lisp implementation. It’s a descendant of Kyoto Common Lisp by Taiichi Yuasa and Masami Hagiya at the Research Institute. Juan is the maintainer.

It is designed for portability. Rather than generating machine code for various processors, it generates C, and then allows the target host’s C compiler to produce machine language. This approach lets it take advantage of the target compiler’s optimizations, and specific knowledge of the target architecture. (However, compilation is not very fast.) All platforms these days include a free C compiler (even Microsoft). It makes minimal architectural assumptions: a pointer can be cast to an int, and C functions can be called with many arguments, and a variable number of arguments.

It supports a wide range of operating systems: Linux, NetBSD, FreeBSD, OpenBSD, Windows, Solaris, and Mac OS X.

The core and the Lisp interpreter are written in C; the rest is in Lisp. It borrows the Boehm-Weiser conservative GC, and provides CLOS with the PCL implementation. It uses native threads. It also contains a byte-code compiler and interpreter (instead of direct interpretation of Lisp as s-expressions). The implementation of subtypep uses the efficient method described by Henry Baker, and works with CLOS types.

It can build standalone executables and dynamically-linked libraries, and this is why it’s called “embeddable”. But it can be used as a regular Common Lisp implementation too, so don’t be put off by the name!

For more details, see his paper.

House Developer

Kristoffer Kvello of Selvaag told us about House Developer, which is basically a CAD system for architects. It allows the architect to draw a very high-level drawing, and it takes care of filling in myriad specifics. It decides where to put windows and doors, and which way the doors should swing. It places electrical outlets and switches. It decides on wall types, wall offsets, wall junctions, heaters, fire exit paths, and so on.

There are many details, all of which must conform to regulations, company rules, and best practices. Doing all this by hand is costly, time-consuming, and error-prone. Automating it reduces errors, and lets the architect try lots of ideas and see their consequences promptly.

This is, in many ways, a classic rule-based expert system. They started writing it in 1994, using Knowledge-Based Engineering (KBE) technology of the time, which was primarily in Lisp. However, the rules are not like classic Artificial Intelligence rules; they are more like constraints. An example:

(define-attribute area (window)
    (* ?width ?height))

This defines a constraint that gets recomputed as necessary. These rules can use the full power of Lisp.

The core of the system is written in Allegro CL. There is a Java-based user interface, that sends S-expressions to the core. The core sends XML replies back to the user interface. It uses many available libraries: asdf, zip, cl-sql, cl-utilities, s-xml, aserve, Expresso Toolkit, and Screamer.

The Expresso Toolkit knows the STEP (Standard Exchange of Product data) and EXPRESS (an ISO standard modeling language), which are important standards in the architecture industry.

Screamer supports “non-deterministic programming; it does constraint satisfaction with mixed systems of numeric and symbolic constraints, based on a substrate that supports backtracking and undoable side effects.

The advantages of using Lisp for this system include:

• Interactive development, with fast recompilation, incremental changes, no need to constantly re-create the global state
• Break loops, with the ability to fix things and then restart
• Reader macros, so that we could customize the syntax
• Advice, so that we could customize behavior
• It’s easy to inspect the image to find out what to customize
• Extensibility in general
• handler-bind, for use on our test framework
• Many available relevant libraries, which worked fine

There have not been a lot of users so far, but they are planning to deliver it to a large customer soon.

High-Performance Architecture

Marc Battyani discussed a high-performance computer architecture, using Field-Programmable Gate Arrays (FPGA’s) that are programmed using a high-level special-purpose language, implemented in Lisp. He has a computer based on a Stratix II FPGA with memory and network. The FPGA has modules on it such as adders, multipliers, I/O pins, memory, and so on. Programming one consists of hooking up the modules up to perform a particular special-purpose function. A problem with FPGA technology is that programming them is so hard; the novel feature here is to use a Lisp-based language, called HPCC, that compiles a high-level description into the FPGA’s program.

They have implemented two applications so far. One prices exotic financial instruments using Monte Carlo simulation. Currently, this kind of thing is done with grids of 10K-10K boxes. The other does multicast networking at 1 million messages per second. They plan to get funding, hire more Common Lisp programmers, and do more applications.

Cells

Ken Tilton talked about his Cells library, a dataflow extension to CLOS. The basic idea is that the values slots are determined by a formula, like the cells in a spreadsheet. Cells tracks dependencies between cells and propagating values. He demonstrated widgets that grow and reshape graphics automatically.

Announcements

Randall Pitts is looking for Lisp programmers to work on a speech understanding project, that would help answer email, help call center agents, etc. They’re dealing with language, grammar, and syntax. You must work in Germany.

Nick Levine is looking for work. He has 20 years of Lisp experience and has been consulting for seven years.

Marty Simmons of LispWorks is looking for applications that use concurrency, to help test their new thread support.

One parting thought

One of the most widespread complaints about Common Lisp is the lack of available libraries. However, in several practical applications, we see here that there are many available libraries for Common Lisp that work well and can be built on.

I’ve just returned from the OOPSLA 2007 conference (www.oopsla.org) in Montreal. This was the single best conference I’ve ever been to. As you’ll see, the conference was much less about “object-oriented languages” (a topic which was used up years ago) than about software development and engineering in general.

Actually I attended three events: the Dynamic Languages Symposium, the Montreal Scheme and Lisp User’s Group Meeting, and OOPSLA itself. This meant that I went to evening activities every night, so it was non-stop and exhausting. On top of that, I developed some kind of stomach virus the day I left and was somewhat sick the whole time I was there! But the conference was so energizing that I mostly forgot that I was sick, aside from the fact that I didn’t eat very much. There wasn’t any time to get to real restaurants anyway so all I had was fast food, although some of it was quite good. There were plenty of other parallel activities that I could not get to, such as a day-long symposium about garbage collection, an APL conference, a Wiki symposium, an educator’s symposium, and on and on.

Two other ITA friends came: Francois-Rene (”Fare”) Rideau, and Alexy Radul who is just finishing an internship at ITA. Alexy had previously been an intern at Google with Arthur Gleckler, and the four of us ate and hung out together a lot. Arthur knows so many people who I know that it’s amazing we had never met before. He was one of Gerry Sussman’s grad students, and has worked with my old friends Charles Frankston and Carl Hoffman. We talked about everything, and I found out more about what life is like at Google, where Arthur works on mobile computing.

Many of the conference luminaries and keynote speakers were old friends of mine. Richard Gabriel of IBM Research, who was the conference chair, I had worked with on the Common Lisp spec, and later when his company, Lucid, was using my company’s Objectstore object-oriented database. I had worked with Guy L. Steele Jr., now of Sun Research, for years at the MIT Artificial Intelligence Lab in the late 70’s. I’ve known Gregor Kiczales, now professor at U. of British Columbia, for years. And so on. Dick Gabriel started working on this conference over two years ago and has done nothing for the last year but arrange the conference. I feel that his work was a huge success.

I met and spent time with several other very impressive people:

Mark Miller, formerly of Xerox PARC, and now of Google Research. He’s open source coordinator for the E programming language, a pioneer of agoric (market-based secure distributed) computing, and an architect of the Xanadu hypertext publishing system. I found him to be a remarkably clear thinker and good explainer.

David Ungar, formerly of Stanford and then of Sun Research and now at IBM, was co-designer of the “Self” language, a particularly simple object-oriented langauge in which there are no classes: objects just inherit from objects. He achieved an amazingly performant implementation.

Richard Tibbetts, formerly of MIT, is a co-founder, architect, and team lead at StreamBase, developing a Complex Event Processing Platform. This is one of the startups based on Mike Stonebraker’s recent research work. We talked about databases, startups, and also about back-home things since we live near each other.

Michael Ernst, an MIT professor working on adding abstract types to Java (JSR-308). This will be one of the major language upgrades in Java 7. It’s great to see that there is interest at MIT in Java in specific and programmer productivity in general. He gave a talk later that I wasn’t able to get to, about his Javarifier (inferring annotations to the code) and Javari (type checker) tools.

One thing I heard from many people was a uniformly negative verdict on the new R6 Scheme standard. This is really too bad, since a lot of work went into that standard and the Scheme community really does need a standard. But there are so many problems with the new standard that nobody seems to like it and many Scheme implementors are not intending to support it.

There were so many interesting papers that I can only hit the high points of the most prominent ones. Here goes.

The Dynamic Language Symposium:

Mark Miller of Google Research talked about security: fighting the malware crisis by running everything with least privilege, using capability principles in languages. He discussed various approaches to retrofitting such concepts into existing languages. JOE-E (Google, HP Labs, U.C. Berkeley) is a subset of Java (so all the tools still work and there is no slowdown), checked by a static verifier. W7 (Jonathan Rees’s thesis) and Cava (at Google) work by inserting runtime checks, which allows more flexibility but there’s a slowdown. I am pretty convinced that capability-based systems are the way of the future and if I ever get to work on them, I’ll follow all the references in this talk.

Alexy gave a shortened version of the talk on Probabilistic Scheme that he gave at ITA recently. I felt that it got a good reception.

Morten Kromberg, of APL-vendor Dyalog, talked about how they have added object-oriented programming to APL. APL is still alive and well, being used by engineers, financial analysts, actuaries, and data miners. Apparently many domain experts can learn to program APL themselves, which gives them a lot of personal power. Insurance companies are particularly APL’s sweet spot. The objects use the Microsoft CLR, and APL can treat the contents of an Excel spreadsheet as an APL array!

Shriram Krishnamurthi, a well-known Scheme and languages professor from Brown who I first met at the original Lightweight Languages Workshop at MIT, talked about Relationally-Parametric Polymorphic Contracts. It was about how to do contracts in your language less painfully without giving up power, involving inferencing at runtime using unification (in the sense of things like Prolog).

Jim Hugunin of Microsoft talked about the new Dynamic Language Runtime, essentially an extension of the Common Language Runtime to make it easier to write .NET compilers for dynamic languages. Jim, who had earlier implemented Jython (Python in the Java Virtual Machine), set out to show that the CLR was no good for dynamic languages by showing how bad Python would run. But after he implemented IronPython for the CLR, he was surprised to see that it actually performed very well! Although it has to do some gross things to work dynamically (e.g. each add has to check both argument datatypes manually), it still runs quickly in comparison to the original Python because the original Python is an interpreter, whereas IronPython takes advantage of the JIT compiler in the CLR. His group is also doing a JavaScript, and they intend for the CLR/DLR to be a suitable platform for Ruby, PERL, and so on. He went into a lot of detail and answered many questions.

The Montreal Scheme and Lisp User Group:

Manual Serrano, from INRIA (the French research institute), maintainer of Bigloo Scheme, talked about “HOP”, his system for allowing you to create highly dynamic web sites with much more elegant code than the usual jumble of HTML, CSS, JavaScript, and so on. His language lets you write code and succinctly flag which parts should run on the client and which on the server, at a very fine grain. He showed some very advanced web behavior in only a few lines of code. Very impressive!

Pascal Costanza of the Programming Technology Lab of the Vrije Universiteit Brussel, Belgium, talked about his work in context-oriented programming based on CLOS. (Pascal is also known for his Common Lisp documentation and for work on aspect-oriented programming in CLOS.) Context-oriented programming lets you create “layers”, which can be turned on and off orthogonally and dynamically within a thread. When a layer is established, an associated set of instance variables and methods become visible in CLOS classes. Amazingly, he was able to build all of this on top of the CLOS meta-object protocol!

Finally, the OOPSLA conference itself:

Kathy Sierra talked about how to create passionate users; that is, how to make products that will get users excited and enthusiastic. This was completely fascinating and I took two whole pages of notes. She has given this talk before, and there is commentary on the web about it already; see:

http://conferences.oreillynet.com/cs/os2005/view/e_sess/7077

http://www.oreillynet.com/onlamp/blog/2005/08/tutorial_review_creating_passi.html

And a blog at:

http://headrush.typepad.com/creating_passionate_users/

She’ll also be writing a book about it soon, to be published by O’Reilley. She’s also the co-author of a series of books called “Head First xxx”.

Jim Purbrick and Mark Lentczner of Linden Labs, the creators of Second Life, explained a great deal about what Second Life is and how it works. The most interesting thing is that 15% of the residents actually do coding, in a language that lets you make active objects. There are 30,000,000 running scripts, 2.5 billion lines of code. Generally there are 15,000 scripts actively running on each “region” (processor), updating at 45 frames per second, and there are 4,000 processors. There are 30,000,000 concurrent threads. The language itself they described as “terrible”; they are working on bringing up the Mono implementation of the CLR so they can provide good languages. There are some complicated issues in which threads must be migrated from one CPU to another as an object moves around within regions of the Second Life world. The most impressive thing is how many people with little or no technical training are doing programming. They also talked about how the Linden Labs developers, eight of them at five physical locations, use Second Life itself as a collaborative development environment; they say it works really well, particularly due to having stereo audio that’s good enough that you can tell where sounds are coming from. During the talk they showed Second Life on the big screen and moved around and interacted with people, so that you could see what it’s like. I, myself, hardly have enough time to live my first life, let alone a second life, but it was quite fascinating to see what’s going on in there.

That evening, Guy L. Steele Jr. and Richard Gabriel did a presentation that I can only describe as a cross between a lecture on the history of programming languages, and piece of performance art. They said exactly 50 things, each in exactly 50 words, so it sounded like haiku a lot of the time. They presented many real languages from the 50’s to the present, including some very weird ones, plus several joke languages (such as the famous Intercal which does all its output in Roman numerals). I was particularly thrilled when they used the xkcd.com cartoon called “Lisp”, as well as a hilarious “filk” song about how God created the universe in Lisp. The performance got a huge standing ovation. These guys are so amazingly clever.

Fredrick Brooks Jr., author of the classic book “The Mythical Man-Month”, talked about telecollaboration. Most of the talk was about collaboration itself, and under what circumstances it’s a good thing: not always! His main point is that collaboration is great for determining system requirements and brainstorming about possible approaches, but that you really need a single system architect in order to achive conceptual integrity. The system architect can delegate parts of the architecture to others (e.g. the user interface czar), but he distinguishes sharply between delegating design (OK) and sharing design (not OK). He had insightful things to say about the open source process (good for some kinds of software but not others), pair programming (he likes it), and object-oriented programming (the most powerful technique to come along so far to improve software productivity). He is an extremely clear speaker with a lot of good points to make. He’ll be writing a book of essays soon.

Then there was a panel session entitled “No Silver Bullet Reloaded”, based on the famous 1986 paper “No Silver Bullet” by Fredrick Brooks. See

http://en.wikipedia.org/wiki/No_Silver_Bullet

and follow the link to the paper. This is one of the best and most important papers ever written about software engineering; if you haven’t read it, you should.

The paper starts: “Of all the monsters that fill the nightmares of our folklore, none terrify more than werewolves, because they transform unexpectedly from the familiar into horrors. For these, one seeks bullets of silver that can magically lay them to rest.” The paper surveys all the major techniques that have been proposed for making software engineering far more productive, considering each carefully and explaining why none is a “silver bullet” for solving the “essential” complexity of software. At the panel, well-known author Martin Fowler converted himself into a werewolf (with a funny mask) and made all his points in the negative, which was hilarious. Of course all the panelists agreed that there are no silver bullets. David Parnas was the most specific about what he meant by a “silver bullet”: there is no technique that’s easy to learn (e.g. a three-day seminar) that’ll solve the hard problems of software design and construction. They also all agreed that it’s hard to talk about what techniqes improve productivity by a factor of ten since there really isn’t any good way to measure productivity. Many good and interesting points were made. Brooks highly recommends the book “Peopleware” (I have a copy but haven’t read it yet; clearly I should.)

The next keynote speaker was John McCarthy, one of the creators of the field of Artificial Intelligence, the creator of the Lisp language, the long-time director of the Stanford Artificial Intelligence Lab, winner of the Turning Award in 1971. Unfortunately, this was the one disappointing talk of the conference. He had very little to say, and he just read his slides.

David Parnas of the University of Limerick, one of the first people to define and expound upon modularity in software, was the next keynote speaker. He defines object-orientation as “designing software by desiging and implementing data-holding objects that make the job easier” and insists that you don’t need any special language to do it: object-oriented is about how you design. His talk stressed our inability to write clear specifications for software. (The EU is currently fining Microsoft $3M/day until Microsoft produces a clear and precise specification for their server [I'm not sure exactly what software we mean here], but it’s just too big and hard to do. Microsoft offered the sources, but nobody wants that, since the sources just say what the software happens to do now, not what it’s defined to do over time.) He advocates much more precise and specific documentation, kept up to date, to the point where if someone asks a question about the software, a programmer would go to the documentation rather than the code to answer the question. It finally turns out that he wants extremely precise specs written in a mathematical notation, which is very possibly appropriate for the kinds of software projects that he was discussing (avionics, nuclear power plant safety) but not for anything I’ve worked on. I’m in sympathy with the call for far better code documentation, though.

Next up was my old friend Gregor Kiczales, one of the creators of the Common Lisp Object System and the father of Aspect-Oriented Programming. He talked about Effectiveness Without Formality. This talk was very deep and philosophical. It was based heavily on the work of two Xerox PARC scientists/philosophers, Lucy Suchman (”Human-Machine Reconfigurations”) and Brian Cantwell Smith (”On the Origin of Objects”, known to be a difficult book to read). Unfortunately I had no familiarity or touch-points for their work, and partly for that reason, Gregor’s talk was, frankly, over my head. Among his key points: There is conflict between the power of programming langauges and the flexibility of design patterns. There is conflict between the mathematical foundations of programming and the way people really work. The world is sloppy and imprecise. There is a big distinction between programming languages that are “effective” (actually make things happen), and formal abstractions which are not “effective”. People never understand each other precisely, but we can deal with this. He talked a lot about “intensionality”, which he described as “the ability to be ‘about’ something”, and the intensional relationships between users, code, bug databases, and all the other entities of software development. The world is as far from formal as you can imagine! Abstractions are transient, shifting, and negotiated. That’s about as much as I got from this. If it were not Gregor, I would have been dubious about the whole thing. I asked whether he had any idea how these ideas might turn into something useful, and he said, not yet. On the way out of the room, Mark Miller said that this was not only the best talk at the conference but the best talk he had heard in years. So if the people smarter than me think that, I should probably try harder to understand.

Jonathan Edwards, a researcher at MIT CSAIL, gave a very interesting talk about conditionals as tables. He presented a two-dimensional graphical notation for representing conditionals, which, as I see it, takes something that was imperative and makes it declarative, always a source of power. This makes it much easier to specify multi-armed conditionals, in that you say what you want rather than how to achieve it, much as in a database query language. So far this only works for functional (no side-effect) programs and it’s not clear yet whether the idea scales to larger programs (he showed a very elegant Fibonacci example). A lot of the smart people in the audience were very impressed.

Sebastian Fleisner from Hong Kong talked about Epi-Aspects: Aspect-Oriented Conscientious Software. By “conscientous” he means “self-sustaining”, continuing to work in the face of failure. This is based on a paper last year by Richard Gabriel and Ron Goldman about “Conscientious Software”. Its abstract is: “Software needs to grow up and become responsible for itself and its own future by participating in its own installation and customization, maintaining its own health, and adapting itself to new circumstances, new users, and new uses. To create such software will require us to change some of our underlying assumptions about how we write programs. A promising approach seems to be to separate software that does the work (allopoietic) from software that keeps the system alive (autopoietic).” The presentation talked about the latter part as having operations like monitor, test, repair, and kill. Their proposal has to do with using aspect-oriented programming to achieve this. What they’re talking about sounds a lot like some of what I’m currently doing in my own work on Hot Upgrade, and I intend to read both papers and see if there are any good ideas I can use.

Pattie Maes, from the MIT Media Lab, was the next keynote speaker. (As you can see, they had a lot of keynote speaker and I spent most of the conference at keynotes!) She talked all about interesting projects being done at the Media Lab to try to give people access to information at their fingertips when they’re not sitting at the computer. This was all quite interesting and fun although it didn’t have much to do with the subject matter of the conference.

Finally, the last talk was by Brian Foote of Industrial Logic Inc.: “Big Ball of Mud: An Introduction to Post-Modular Programming”. The key point was that most real-world systems use a “big ball of mud” (totally unmodular) architecture, and why is this architecture so popular and successful? Maybe we should give up trying to teach people to be modular, and embrace the big ball of mud. The talk was very rambling and often funny, and I can’t figure out whether it was three-quarters tongue-in-cheek or entirely tongue-in-cheek. By this time I was so exhausted that I was yawning through the talk, anyway.

There were plenty of other talks that sounded interesting, but I could only be in one place at a time, and I’m not sure that I could have absorbed even more material in one week anyway! The set of speaker was nothing short of stunning, and I also learned a great deal from casual conversations during the breaks. I’m back home now and nearly over my illness, and ready to read all the “to do” and “read this” notes I wrote during the conference and get started.