Monday, January 5, 2009

What Is All the Fuss About How You Can Write DSLs in Lisp?

I saw Chris’s recent post, and felt compelled to respond, despite the fact that I’m pretty sure I’m going to regret it. :)

Here’s what he says, abridged:

(when (and (< time 20:00)
      (trade (make-shares 100 x)))

IMO, that's not a DSL -- that's just a set of function calls in an existing language.

The short version of my response to this is: “Yes! Exactly! That’s a feature, not a bug!” On to the longer version.

This argument, as a question of semantics, is fundamentally doomed to remain unresolved, unless someone can magically provide an exact definition of DSL that a) has no imprecision, and b) everyone agrees with. I’ll just pass on waiting for that to happen, and make my own observation:

A set of function calls in an existing language is generally preferable to a completely separate language. Particularly if it’s the language that you’re using to write the rest of your system. Sure, there are cases where using a specialized language is the thing to do, but over the last few years I’ve come to have a lot more - what’s the word? respect? fear? caution? – anyway hesitance around introducing additional languages into the development process. It’s not rare to see C#, SQL, XSLT, XPath, NAnt XML, regular expressions, and four or five other grammars floating around in the development lifecycle of a single process, and it’s got a real price. If nothing else, it means that not everyone can work on every part of the system. Throw in the fact that some of those are hard to debug, or have conflicting metaphors, or different performance characteristics, and it adds up.

So yes, one of the reasons that I’m excited about Clojure is that it’s a Lisp, and as a Lisp it is flexible enough to adapt to the point where I often don’t need a DSL. Or, if you like, where my DSL is also a Lisp. That doesn’t mean “I never want to use a DSL” or “I always want to use Lisp” – all things in moderation. It certainly doesn’t mean “You shouldn’t use M” – how the hell would I know what you need? But for me, I think it’s nice to have a tool that gives me the choice.


  1. A few things here.... You are blinded by your imperative mindset.

    The truth is Lisp makes it harder to tell if you are using code or data, because the two ideas are seamless.

    In this particular case, Lisp provides several ways of interpreting this DSL, only one of which is regular function calls:

    Second, This could also be quoted data.

    Third, The DSL can also be macro-expanded into code or other data.

    Not only are there multiple ways of interpreting this data, but you can combine ways, too.

    The quoted data, for instance, has the option of later being "EVAL"uated as code or it could be processed by code. If it is evaluated, you can also include other LISP operators in the DSL.

  2. I responded on my blog (link attached).

  3. @Wesner: I assume your comments are directed at Chris, not me.

    @Shawn: Looks like you forgot the link. I assume it's this one:

  4. @Shawn: re your post:

    So what's the bright line between "internal" and "external" DSLs? Presumably it's the syntax. What are the units of syntax? :)

    Not that *you're* saying this, but this seems to be to be the "Lisp looks weird" argument. That's an old one, and I'm not going to try to change anyone's mind on it, despite my personal opinion that it is completely wrong. I will say that I haven't heard this one from anyone who's actually *tried* Lisp, and that it's pretty funny to hear it from anyone who doesn't also complain about XML. Again, that comment is not aimed at you.

    BTW, if you don't like that particular "DSL", there's nothing stopping you from using Lisp to express it as this:

    (when (< time 20:00) and timing-is-right then
    (trade (make-shares 100 x)))

    Or even

    (when time < 20:00 and timing-is-right then trade 100 shares)

    or whatever. (See the loop macro in Common Lisp for a really interesting demonstration of this sort of thing.) As you progress towards the latter, you move closer and closer towards writing a parser/lexer, and farther and farther from the beauty of using a homoiconic language.

  5. I think all that Chris was trying to say is that using an external DSL, it is easier to make a language that is readable to a business person. LISP just happened to be an easy dog to kick.
    The other advantage of external DSLs, if you are going to allow "users" to input logic, is you can easily avoid the (sql | html | script) injection attacks.

  6. @Darrel: then why the comment about functions?

    I think the idea that we're somehow going to come up with a language that is easy for a person with no programming experience to write is laughable - businesses can't even define their requirements in English, and whatever the DSL looks like, it's not going to be English. It's going to be a programming language, with variables, flow control, etc. etc. You can reduce the amount of repetition and provide convenient libraries to make it more tractable than, say, Perl, but at the end of the day you cannot abstract away the fact that you're specifying a protocol.

    Real example: SQL is a DSL. Do we generally let business users write it directly?

    Also, I don't see that external DSLs are inherently better at avoiding injection attacks. That depends entirely on the implementation.