Software Technologies that I MUST Learn
Helpful friends and acquaintences often let me know about exciting new software technologies that I absolutely must know about. I like learning about exciting new technologies. Unfortunately, the software world is generating them faster than I can learn them. Here’s my list of things I really must learn.
Programming languges
Dylan: A Lisp-family language developed by my friends.
ML: How can I be a “Lisp expert” and not know ML??
Haskell: Functional programming. Alan Bawden says this is fundamentally different and I must know it.
Ruby: Up-and-coming popular language.
Subtext: Programming in trees instead of text. (By Jonathan Edwards, MIT CSAIL.)
Hygenic Macros: A languages feature for doing macros (in the Lisp sense) cleanly.
Groovy: A dynamic language for the Java Virtual Machine.
Ron Garret’s paper on a module system for Lisp.
F#: Microsoft’s new function programming language.
LSharp:
Rlisp: A Lisp embedded in Ruby.
Programming tools and libraries
OProfile: The profiling tool for the Lisp implementation that I use.
Krugle:
LispBuilder: Access to SDL from Lisp for game development.
clbuild: An alternative to Lisp’s asdf-install: helps with download, compilation, and invocation of Lisp apps.
Networking
TRILL: New network protocol designed to solve the problems of the spanning tree.
Databases and Caches
Freebase: “Open, shared database of the world’s knowledge.” From Metaweb, my old friend Danny Hillis.
CouchDB: Highly scalable document-oriented free DBMS written in Erlang.
Chubby: Google’s distributed lock system.
Google Sites: Web page design tool
Kompozer: Web page design tool
GORM: Grails’s object-relatinal mapping tool, using Hibernate 3.
Terracotta: Clustering/caching tool for Java, making many JVM’s look like one.
Drizzle: Stripped-down MySQL, useful for caching too.
Whirlycache: A very fast cache
cl-prevalance:
Mongo: A grid-aware object-oriented DBMS from 10gen
Cloud computing
RightSize
Elastra
10gen
Web tools
Rails: (Ruby on Rails) Very popular, highly recommended by many people.
Grails: Rails for Groovy. Built on GORM and Spring.
Google AppEngine:
Other
VMWare: Can I run Linux on my Windows box?
Software repositories: CPAN, http://planet.plt-scheme.org/, etc.
Jango: A Pandora alternative
I’ll get to them, really, I promise…
October 13th, 2008 at 2:13 pm
Dan, are you aware of VirtualBox from Sun instead of VMWare: http://www.virtualbox.org/. It’s free and after having used both, I prefer VirtualBox.
October 13th, 2008 at 2:18 pm
Good God, why? Surely these can’t all actually be useful.
October 13th, 2008 at 9:03 pm
Don’t forget Eclipse!
October 13th, 2008 at 10:16 pm
JRM’s Syntax-rules Primer for the Merely Eccentric tells you all you need to know about hygienic macros. Indeed, by the time I got to the point where it says “At this point, things are starting to get complicated”, I already knew all I needed to know about them — although the rest was definitely interesting.
October 14th, 2008 at 1:24 am
Dan, what do you think about Clojure? It is, like Groovy, also a dynamic language for the Java Virtual Machine but is a dialect of Lisp.
October 14th, 2008 at 2:27 am
You might also be interested in Elephant (persistent database and MOP).
October 14th, 2008 at 5:08 am
Jeremy: Oh, yes, Eclipse too. Lauri: I recently heard Rich Hickey’s basic talk on Clojure; that’s why it’s not on the list. However, it would be good for me to learn MORE about it. It’s full of exciting ideas.
October 14th, 2008 at 11:16 am
I just looked through the article that John Cowan references about Macros.
I didn’t go through it with a fine-tooth comb, but it looks to me like this
is an article about how to use “Syntax-rules”, not about how “hygiene”
works. One of the reasons “hygenic” macros are so badly understood is that
when you want to learn about how they work, people always hand you articles
that spend a lot of time explaining the syntax rules pattern-and-template
language, and say almost nothing about “hygiene”.
As near as I can tell, all this article says about “hygiene” is: “The
automatic hygiene tracks the context of the code fragments and essentially
allows us to manipulate the intermediate code fragments as elements in an
abstract syntax tree”. That sentence seems to be it. That is not an
explaination of “hygenic macros”.
The pattern-and-template language is completely orthogonal to “hygiene”.
You can have either one without the other.
October 16th, 2008 at 4:35 am
Dan: My fault. I see I did not properly express the sarcasm I was aiming for. What I mean is, my friends and co-workers come up to me and say, oh, Dan, you’ve just GOT to learn about XXX! I sigh and add it to the list; if this guy said it’s cool and important, then I suppose he must be right, since I respect him so much. But when on Earth will I find the time? It’s frustrating, and this blog entry was just me letting off steam, in a sense. That’s what comes of having too many cool friends who know altogether too much technology, in this fast-changing world of software!
October 16th, 2008 at 8:36 am
I suspected sarcasm, but it is really good to put it into the comment instead of the main article.
Maybe people should qualify how much effort you have to invest in order to learn something. I would say that the effort in learning cl-prevalence is on a whole different scale than to learn ML or Haskell.
October 17th, 2008 at 3:55 pm
Running Windows on Linux or vice versa is definitely possible with vmware. I’ve done both.
In my experience, running Windows in a vm is more efficient than the reverse.
Btw, good list!
October 20th, 2008 at 9:27 am
OK, so I know this is really a list of things you don’t have time to learn. Still, I’d like to speak up about ML–not necessarily *for* it. I love ML, but here’s a few things you may not like about it (basically, places where it’s decidedly unLispy):
* No macros (surprise).
* There’s a REPL, but you can’t redefine functions. You can provide new definitions, but existing functions will still call the versions they were compiled with.
* Datatypes. To build, say, trees, you have to define a tree type; you can’t just start slamming cons cells together. (Personally, I love doing it this way, because the pattern matching can make sure I handle all types of tree nodes. Matter of taste.)
* The function calling syntax is pretty much the anti-Lisp: concatenation is function calling. So, if you take the usual way of defining “foo” to take two arguments, then you’d write “foo 1 2″, which really means “(foo 1) 2″; “foo 1″ is a curried function, defined under the hood. The catch comes when you want to replace “1″ with another function call; say, “bar 7″. “foo bar 7 2″ won’t work, because “foo” can’t take the function “bar” as an argument; you have to write “foo (bar 7) 2″. You can get used to this, but it still trips me up from time to time.
On the other hand, if you want to compare it to something like Java, it kicks cdr. You get the benefits of compile-time typing without having to write type declarations.
October 21st, 2008 at 3:19 am
F# is almost equal to OCaml, with some extensions.
Erlang is good language, that should be taken into account
October 24th, 2008 at 8:34 am
John, I just returned from the OOPSLA 2008 conference. On the way there, on the plane, I started reading Ullman’s textbook on ML. I was struck by the number of warnings, in the early chapters, about all the ways that you could get fooled by the infix syntax, e.g., be careful you don’t do xyz because it looks like it does this, but really it does that because abc is right-associative, or whatever. It’s been so long since I learned one of these languages! It makes me wonder why people complain about Lisp, which does not have any of those problems! (Just kidding, I know all about why people complain about Lisp.)
Anyway, there was a talk at the conference about Haskell’s type system. I asked the speaker whether he thought it was worth learning ML now that there’s Haskell. He said that it might be worth it because it’s easier to learn Haskell if you already know ML, but other than that, in his opinion Haskell pretty much makes ML obsolete, unless you are trying to study the history of programming languages, or comparing type systems for academic purposes, or something. (This was Mark Jason Dominus; the talk was very funny as well as real.) I’ll have to ask people whether they concur. I do get the impression that learning Haskell is more important than learning ML, at this time, for me, since I am getting very interested in functional (in the sense of “no side effects”) languages.
October 31st, 2008 at 11:20 pm
I just found your blog after catching up on Mark Jason Dominus’ blog, which I recently began reading after I finished his book “Higher Order Perl”
Anyhow, I just wanted to add my $.02 about new technologies. Erlang is quite interesting because of it’s real-world success in being used to create highly parallel systems with near-linear (or better!) scalability. It has a relatively simple syntax, though I sometimes feel the standard libraries could use some polish.
Erlang is a language that sometimes feels very restrictive. However, I have learned that those restrictions have a direct benefit – that of forcing the programmer to create solutions that allow the Erlang runtime to efficiently parallelize programs, with almost none of the traditional issues with multi-threaded and multi-process applications.
While my exploration with Erlang has only been cursory toying around, inspection of the code of applications such as YAWS has shown me that there is much I will learn from using it further. I’ve already found that the core concepts of Erlang’s ‘concurrency model’ are already improving my work with code in other languages.
November 1st, 2008 at 12:01 pm
What about Clojure?
November 1st, 2008 at 2:12 pm
Oh, Clojure is way up on the list! I’ve now heard Rich Hickey’s talk twice, but I have not had a chance to read the documentation or try writing code in it. Rich was on a panel that I moderated at the OOPSLA 2008 conference on The Future of Lisp, and will be at the International Lisp Conference 2009 which I’m chairing (see ilc09.org). Clojure is extremely impressive.
Erlang isn’t on the list because I have already basically looked into it. (It’s not that new, chronologically, although its ideas are still cutting edge.)
More things I wish I had time to learn more about:
Implementation techniques for what functional programming people call “Persistent” data structures (used in Haskell and Clojure, and surely others).
Factor, a new language by Slava Pestov.
How Eiffel deals with abstraction boundaries between superclasses and subclasses. That particular modularity is not supported well by most OO sytems (C++’s public/protected/private is too simple but is at least a start!).
The E programming language (http://www.skyhunter.com/marcs/ewalnut.html).
November 4th, 2008 at 9:13 am
For persistent data structures, I liked Chris Okasaki’s book “Purely functional data structures” from 1998 (uses SML with lazy extensions; Haskell code is provided in an appendix).
I read it when I was an undergrad, and for me, it was worth it for its explanation of amortization alone.
November 18th, 2008 at 7:30 pm
I learned language the “old fashioned” way, as I needed them for projects or classes.
So my path was BASIC, Pascal, LOGO, Lisp (Scheme), CLU , and finally C/C++.
CLU made it easier to ultimately pick up Java applications, but I’m curious where Java
or VM technologies (like Squeak) fall on the list.
November 23rd, 2008 at 12:34 am
Okasaki’s book is fantastic. Dan should definitely add reading it to his growing list of things to do!
December 21st, 2008 at 1:51 pm
My sympathies: I’m basically in the same boat. So many “must-learn” computer languages, so little time.
F# is OCaML + .NET Framework; OCaML is ML + objects. The documentation for OCaML is pretty good.
Microsoft Virtual PC is free, and will let you run almost any Linux you want from your Windows XP or Vista machine. There are some tricks to setting up Linux in a VPC: the key trick is to make sure you select a supported video mode during installation. Some Linux distros try to run the virtualized video in 24-bit mode by default, which doesn’t work.
If you figure out Haskell monads, please explain them to me. Others have tried, and I thought I understood the explanation, but 10 minutes later it didn’t make sense.
December 21st, 2008 at 1:58 pm
One thing to add to the list (
) is Erlang, I was looking at it the other day because someone recommended doing so, and it had this really cool concept of parallel execution, which seems to be simply enough that you queue up messages, and the process does its computation with no shared state, and returns a value. So there’s no hair from threads and synchronization and locks and semaphores and deadlocks and all that crud.
I’ve been programming a lot in Javascript lately, in a non-threaded environment, and it is really a luxury to not have to use threads and concurrency. I could get used to that…
December 23rd, 2008 at 8:32 pm
It’s nothing to do with fancy functional programming, but you should also look at CUDA. This is NVidia’s framework for “GPGPU” — general-purpose GPU computing. While CUDA itself is merely nifty — C-like syntax compiled to shader language, and NVidia-proprietary — the general concept of moving computation to the GPU is incredibly compelling, and has produced 1-200X speedups vs state-of-the-art CPUs for certain problems.
What I find even more exciting about this is that, in general, *any* dynamic programming algorithm can be re-implemented on the GPU to take advantage of the 64-256 parallel threads of execution the GPU offers, as long as it requires less than the 1GB of RAM that these cards now offer.
There are a few books in the graphics gems series called “GPU Gems” that are worth getting.
December 23rd, 2008 at 8:36 pm
Regarding Henry’s comment about JavaScript: the things I dislike about JavaScript are the looseness of the type-checking and the aggressiveness of the type coercion. While dynamic typing is nice for scripting, JavaScript really takes it a bit too far. For example, I was recently writing some code to return a Java byte array to JavaScript (using an obscure Netscape technology called LiveConnect — don’t ask), and JavaScript interpreted each byte as a sequence of digits, composed a huge bignum, and the converted the whole thing to float, without me asking it to do any of that, and without any warning. The result: “Inf”.
December 24th, 2008 at 8:49 am
Dave, regarding CUDA: Have you been looking at OpenCL? Some recent info is in this article. It mentions that any card that supports CUDA will support OpenCL, although it also sounds like it’s lower-level than CUDA. Part of the claim here is that by using this technology, you’ll be able to retroactively change whether your code runs on the GPU or on your multi-core CPU, and so you need not pre-commit to which of those strategies is going to be looking better, years down the road. It looks like the language makes C look rich and powerful in comparison, but I guess for the kinds of things people want to do at this level, that might be OK. Are people using this kind of technology in real products yet, or is that still to come?
February 6th, 2009 at 2:25 pm
[...] in order to observe any low-frequency mummers disparaging Rich Hickey. In any case, Clojure has garnered a lot of excitement from some highly credible LISP hackers. If Mr. Kenny Tilton would come around, then the coup would [...]
May 20th, 2009 at 2:50 am
JavaScript + JQuery