An Interview with Lutz Mueller, Creator of newLISP

I recently had lunch with Lutz Mueller, creator and author of the programming language newLISP.

I’ve never used Lisp, but I’ve always had a fascination with the “AI” languages. There’s something magical about a language designed to solve ridiculously difficult problems.

In the same vein, there’s something magical about an individual who writes a programming language on his own. So someone who writes an AI language on his own ranks pretty high on the list of intelligent people I’ve met.

Like a renaissance man, Lutz’s experience runs far and wide: he has a PhD in Psychology, a background in statistics, and spent many nights programming punch cards (and many nights working for an internet startup). He’s now the one-man show behind the open source scripting version of Lisp called newLISP.

Rob: Give us some background on newLISP. How long has it been around, how has it evolved, etc…

Lutz: newLISP started on SunBSD in the early 90’s, then moved to Windows 3.0/1 as a 16-bit application, then to Windows 95/98 as a 32-bit program. During all that time newLISP was more like a hobby for me. Around 1999/2000 I made it Open-Source under the GPL license on Linux. At the end of 2002 I started working on newLISP full time and a community of users has been growing quickly since then. In 2005 it was adopted for the first time by an internet startup as the only development language for a large scale (~100 CPU’s) distributed application running a specialized (sentence) search engine.

Rob: How is newLISP different from LISP?

Lutz: newLISP is a scripting language very much in the culture of Perl or Python; quick to learn and to get the job done. newLISP takes its core elements from LISP, such as lists and lambda expressions, and adds name-spaces and implicit indexing.

The style of programming has changed dramatically during the 90s and the internet age. Programming is more casual, interactive, and performed by different people than before the internet and personal computer, and is now used for different sorts of applications. Scripting languages have made access to programming easier for people with no previous programming experience and has greatly increased speed of development for professionals.

Rob: What inspired you to create newLISP?

Lutz: I was always intrigued by LISP but found that the current standards of Common LISP and Scheme were behind modern times in their approach to programming, and too overloaded with academic concepts to be useful for solving real world problems. I wanted to create a LISP with a scripting feel, like Perl or Python, but still a LISP.

Rob: Creating your own language seems like a massive undertaking. How is writing a language different from other software development you’ve done?

Lutz: What you are creating has the potential to be used by many different people with different backgrounds, motivations, and on different applications. Making a language has as much to do with Psychology as it has with Software Engineering. It is very important to listen to users and to know many other programming languages: from machine assembly language to high level languages like SQL.

The core of a well designed programming language should be small. The bulk of the newLISP code consists of the newLISP API functions that are, internally, relatively independent from each other. But looking from the outside, from an API-level, those functions have to work together and should paint a cohesive picture of the language.

Designing and writing the software is only about 25% of the entire project. The rest is QA (Quality Assurance), documentation, and contact with the community of users via support and exchanging newLISP code and ideas. The way a programming language is used differs very much from person to person and from application to application.

Rob: Are you the sole developer on the project?

Lutz: For the core of the language I am the only developer, but many bits and pieces in the project come from other projects and programmers, as is typical for many Open-Source projects. I especially depend on help on the documentation side since I am not a native English speaker.

Rob: Most of my readers are enterprise developers using .NET, Java, or PHP. Why would an enterprise developer want to learn newLISP?

Lutz: Many people like the functional style of programming, but they feel that Common LISP and Scheme do not offer the API they have come to expect from .NET, Java, or PHP. Things like regular expressions, network functions and support for popular protocols like XML or HTTP are all built in to newLISP without the need of external libraries.

Many types of problems, i.e. complex statistical analytics, modeling and simulation are much easier to design and implement in a functional programming style using readily available functions in newLISP. This also includes financial functions and Bayesian modeling implemented on a high level.

newLISP has a documentation facility similar to JavaDoc, which makes it easier to use in an enterprise setting. It also allows splitting large programs into independent modules. But using newLISP is not an all-or-nothing decision, it can live and cooperate well with other languages in a project.

Rob: How does newLISP compare to Python in terms of AI programming?

Lutz: newLISP has better functional abstraction and symbol handling and is more flexible in modeling. The same list syntax for program and data makes it easier to write programs which generate or modify other programs. newLISP has built-in support for distributed applications.

For example, you can use the same function to read/write from/to the local disk or over the network, or the same functions to load program code locally or remotely. A built in function called net-eval exists to distribute processing to many remote nodes in one function call.

Rob: I’ve never used LISP or newLISP. If I want to learn newLISP, where should I begin?

Lutz: On http://newlisp.org you will find introductions created by professional writers and a few small tutorial movies. The newLISP manual and reference is full of examples and there is a ‘Code Patterns’ document describing common solutions for common problems. Traditional books on Common LISP or Scheme are not recommended for learning newLISP, because they emphasize a style of programming not suited for newLISP.

Here are some pointers to pages on the newlisp.org site:

For introductory and reference documentation:
http://newlisp.org/index.cgi?Documentation

Short code snippets:
http://newlisp.org/index.cgi?Tips_and_Tricks

User contributed short examples:
http://newlisp.org/index.cgi?Code_Contributions

Support and community:
http://www.alh.net/newlisp/phpbb/

A blog about newLISP:
http://newlisper.blogspot.com/

The last link also contains links to other newLISP related sites.

Rob: Have you seen the Netflix contest (www.netflixprize.com)? Is newLISP up to the challenge?

Lutz: I believe this is a typical problem that can be solved with newLISP, but I don’t have the time and do not quite agree with their terms. Perhaps someone else will take on the challenge using newLISP, it would be the best tool for this contest ;-)

Rob: Thanks for your time, Lutz.

Lutz: Thanks for talking with me.

Start Small, Get Big
Growth Secrets for Self-Funded Startups. It'll Change Your Life.
What you get for signing up:
  • A 170-page ebook collecting my best startup articles from the past 5 years
  • Previously unpublished startup-related screencasts
  • Exclusive revenue-growing techniques I don't publish on this blog
"The ideas and information Rob provides should be required reading for anyone that wants to create a successful business on the web." ~ Jeff Lewis
Startups for the Rest of Us...
If you're trying to grow your startup you've come to the right place. I'm a serial web entrepreneur here to share what I've learned in my 11 years as a self-funded startup founder. Luckily several thousand people have decided to stick around and join the conversation.

For more on why you should read this blog, go here.

9 comments ↓

#1 http:// on 04.20.07 at 8:49 pm

I’ve been working in Lisp for 30 years now and after reading the manual for newLISP my feelings are that Lutz Mueller never understood what it is in Lisp that makes it still the most powerful programming language around. What he created is far far closer to scheme than to a modern Lisp. I wish he had called it newSCHEME so that people don’t get a bad impression of Lisp from newLISP.

#2 http:// on 04.22.07 at 2:45 am

> what it is in Lisp that makes it still the most powerful programming language around. Could you please explain to us what “it” is, John? > I wish he had called it newSCHEME All this fuss over a name. Maybe Lutz should have called it “notYourLanguage,” so programmers of other languages wouldn’t feel so needlessly threatened by it. m i c h a e l

#3 http:// on 04.23.07 at 3:30 pm

GPL is an unfortunate choice since I cannot embed nLisp into an application without making the application GPL as well.

#4 http:// on 04.23.07 at 9:52 pm

The key feature of an program language that claims to be a Lisp is that: nil is the same as () and it is used to represent false. Also (car nil) and (cdr nil) return nil. This is crucial to making the code easy to program and to make the code handle a wide variety of arguments. It’a similar in power to the feature of the C programming language that 0 is false. Note that Scheme doesn’t have this feature and thus this is a good way to tell a Scheme variant from a Lisp variant. In this respect newLISP is not a Lisp and in fact the way that newLISP’s cons function works is so bad that even Scheme people would be horrified (and they are used to very terrible stuff). A second feature of Lisp is that it has separate namespaces for functions and variables. This is not nearly as distinguishing a feature as the first one but still the fact that most Lisps have two separate namespaces for functions and variables makes them better suited to serious practical programming.

#5 http:// on 04.24.07 at 9:17 am

Thank you for you quick response to my question, John. > The key feature of an program language that claims to be a Lisp is > that: nil is the same as () and it is used to represent false. > A second feature of Lisp is that it has separate namespaces for > functions and variables. At the bottom of page one in “History of Lisp” (McCarthy, 79) begins a list of ideas that characterize a LISP, and nowhere in those fifteen ideas do I see mention of the two features you cite here. > It’a similar in power to the feature of the C programming language > that 0 is false. Then I wonder why C99 added a boolean type? > Note that Scheme doesn’t have this feature and thus this is a good > way to tell a Scheme variant from a Lisp variant. I remember reading a person’s post somewhere that passionately and with excruciating detail explained why you should never, ever, ever say that Scheme is a Lisp. I found it strange that someone should care so much. Clearly, Scheme, like Common Lisp and Emacs Lisp, is a dialect of Lisp. To argue that newLISP should be viewed more as a Scheme than a Lisp is like saying you shouldn’t say you’re from your paternal grandfather’s side of the family, only your mother’s. > and in fact the way that newLISP’s cons function works is so bad > that even Scheme people would be horrified (and they are used to > very terrible stuff). This and the earlier post’s insults (here you manage two in one sentence!) are what I find strange. newLISP and even Lutz himself (who does not profit monetarily from selling the language or anything relating to it) are being disparaged by someone who has been doing just that for some time now. m i c h a e l P.S. For anyone wondering about those fifteen ideas characterizing LISP, they are: 1. Computing with symbolic expressions rather than numbers. 2. Representation of symbolic expressions and other information by list structure in computer memory. 3. Representation of information on paper, from keyboards and in other external media mostly by multi-level lists and sometimes by S-expressions. It has been important that any kind of data can be represented by a single general type. 4. A small set of selector and constructor operations expressed as func- tions, i.e. car, cdr and cons. 5. Composition of functions as a tool for forming more complex functions. 6. The use of conditional expressions for getting branching into function definitions. 7. The recursive use of conditional expressions as a sufficient tool for building computable functions. 8. The use of ?-expressions for naming functions. 9. The storage of information on the property lists of atoms. 10. The representation of LISP programs as LISP data that can be manip- ulated by object programs. This has prevented the separation between system programmers and application programmers. Everyone can “im- prove” his LISP, and many of these “improvements” have developed into improvements to the language. 11. The conditional expression interpretation of Boolean connectives. 12. The LISP function eval that serves both as a formal definition of the language and as an interpreter. 13. Garbage collection as the means of erasure. 14. Minimal requirements for declarations so that LISP statements can be executed in an on-line environment without preliminaries. 15. LISP statements as a command language in an on-line environment.

#6 http:// on 04.24.07 at 5:13 pm

> (apply new-tricks old-dog) nil

#7 http:// on 04.24.07 at 5:23 pm

I’m coming at this from a practical viewpoint, not an academic one. I get paid for programs that work and not for writing papers about programming languages. Thus it’s critical that the tools I use are the best they can be. When I talk about what Lisp is I refer to what Lisp had evolved into in the late 70’s (roughly 30 years ago). This was the time of the Lisp/Scheme split as well. What if you had a great screwdriver that you used at your job every day and someone came along and changed the bit, put big plastic safety shields all over it and changed the handle to something that required two hands to hold. You complained “this isn’t a screwdriver”. The reply was “it certainly is, I just changed a few things on it and added a few things”. The point is that to the layman Lisp and Scheme look identical. To the expert they are vastly different, more different even than C and Pascal. I heard too often someone disparaging Lisp and saying that they used it once in a course in college and that it was impratical for doing serious work. Upon questioning them I find that they had used Scheme instead. There’s no one out there protecting the name Lisp. You can bet that if someone created something unlike Java and called it newJAVA they would get a call from Sun’s lawyers. I don’t know Lutz Mueller but I have no doubt that he’s a gentleman and a scholar. But even the best artists create some bad works. I’m criticizing his work and not Mueller. Yes, I’m trying to be a detective as well. I’m trying to figure out why something he calls a Lisp is so much more like Scheme than Lisp. I though initially that he just misunderstood what made Lisp so good. Now I’m thinking that he was using a Scheme system and mistakenly believed that Scheme and Lisp were two names for the same thing.

#8 http:// on 04.29.07 at 8:27 am

Hi John, Although I don’t intend to continue in what I suspect would be a pointless exercise, I do want to complement you on the civil tone of your last post, John. Thanks. m i c h a e l

#9 http:// on 06.08.07 at 2:46 am

Great article Rob.