Planet Lisp

Rob Warnock shares some Lisp-helpful vi settings and Mikael Jansson gives you Lisp argument lists in vim.

The most significant changes from the previous version include:

• All objects are memory managed (garbage collected), which means they can have indefinite extent like any other Lisp object. There is no need to put the object creation in a letm; in fact letm doesn't exist anymore. Use let, or defparameter, etc.
  
• All array (vector and matrix) element types that are supported by the platform, CL implementation, CFFI, and GSL, are supported by GSLL.
  
• On SBCL both Lisp and C use the same representation for array contents; they are not copied between the two sides. Thanks to Tamas Papp's foreign-friendly arrays for the inspiration.
  

There have been some function name and argument changes, so users of the previous version will need to update their code. Please use the git repository and discontinue using the old svn repository.

Feedback welcome, here or on the mailing list.

The universe just told me again that I can make dumb mistakes with the best of them (note to universe: “you don’t have to tell me this, I already know!”). For the sake of this post, imagine that you need to optimize some code similar to

where doit is something like

I.e., I need to call some function using a closure that makes reference to an outer lexical variable (other in this specific case). A problem with the code above is that the closure is going to get re-allocated with every call: if I call (doit #'test-0 2) a 100,000 times, I’ll need make space for the closure every time and I’ll cons more than necessary and take extra time at that.

I decided that I wanted to pull the lambda form out so that I could tell the compiler more about it. Thus I had:

(You gentle readers probably already see the morass towards which I head… keep quiet.)

The intent of separating out the lambda this way is that I can tell Lisp that it needed allocate the closure on the heap; i.e., that it can go away once I leave the body of test-1-a; i.e., that it has dynamic-extent. This would look like:

If your Lisp doesn’t notice your mistake and tell you about it, then you might be like me and wonder why this optimization doesn’t help. Here is table showing the functions and the amount of time and space is takes to call them 100,000 times on misterx, my oldish OS X laptop.

As you can see, the declaration doesn’t do squat (I’m not sure yet why the space needs actually double for the test-1 variants but that’s not part of the sad story I’m telling…).

A co-worker more clever than I pointed out that fun isn’t a function, it’s a variable and that I’d hopped to the wrong level of abstraction. The variation I really wanted was

I find these variants more transparent and only wish that I hadn’t fallen into my initial lambda trap in the beginning. As is often the case, once I’d headed down door #1, it became hard to see my mistake until someone else’s eyes pointed out the obvious. Does it matter? Oh yes! Here’s the completion of the table above:

Not only do we get a speed boost, we also get a big space win from the dynamic-extent declaration.

Two addenda:

1. A positive outcome of this muck-about is that I found and fixed a bug in bind’s handling of some declarations. It’ll get pushed out eventually.

2. I also noticed that SBCL (version 1.0.20) catches the errant declaration and prints a warning. As of version 8.1, Allegro Common Lisp does not (if you want to send reports from others Lisps, I’ll update this post).

Happy New Year; let’s make it great one!

• When you update to HEAD make sure you enable the slime-repl contrib, otherwise you'll be stuck to an old-school inferior-lisp REPL, only.
  Recommended: Use the slime-fancy meta contrib which enables various useful features you'll probably enjoy.

• C-u C-c C-c will compile the defun at point with maximum debug optimization setting. (SBCL only so far.)

• Bleeding edge feature: M-- C-c C-c will compile the defun at point with maximum speed optimization setting (still SBCL only.) This is very useful to benefit from SBCL's type inference and elicit compiler notes.
  

While trying to update my WordPress installation to 2.7 I managed to overwrite my old MySQL database. Whoopts. Or, more colloquially, oh sh*t.

Alas, I was a bit over-eager with the cleanup this time and I am considering putting up a static page with "commercial content archiving policy" stating that, yes, you can post spam and have it stored, at a cost of UKP 10 per day, per message. This is, of course, a more modern version of my spam archiving policy from the mid-90s (no, I received neither apologies nor money, but I did have great fun sending out a couple of emails asking for it, referencing said web page; I also know that it was adapted by some other people).

Since everybody else is implementing mimeparse in their favorite programming language, I thought it would be appropriate to have a Common Lisp version of mimeparse. I'm not completely happy with it and it's certainly not anywhere near ready for production use, but maybe other people can have some fun with it. Only tested in SBCL, but should be portable enough.

I only knew Thiemo indirectly, as a SBCL contributor, but I was saddened to learn of his death. He gave much and he will be missed by many communities.

• The CAR of a form must usually be a symbol, naming the function, macro, or special operator that is supposed to be invoked.
  

  As a special exception to that rule (historically for ISLISP compatibility), the CAR of a form can also be (LAMBDA (...) ...).

  ((lambda (x) (* x x)) 2) ==> 4See CLHS section 3.1.2.1.2.

• The :interactive, :report, and :test arguments of a clause in RESTART-CASE take (LAMBDA (...) ...), but not #'(LAMBDA (...) ...) forms.

• Likewise for the :report option of DEFINE-CONDITION.

• Likewise for the :print-function, and :print-object options of DEFSTRUCT.

I'm glad to announce the first public release of XCVB, the eXtensible Component Verifier and Builder for Common-Lisp, a shift from ASDF. While the software is still very primitive and under-documented, it comes with a simple example of how I automatically converted Exscribe to be compiled with it. XCVB currently depends on cl-launch and asdf-dependency-grovel.

The documentation I wrote already has everything there is to say about the little that XCVB is and what it isn't yet, so I won't repeat it here. Instead, I will discuss the deeper rationale for XCVB.

It is common wisdom that a (programming) language can't be designed perfectly in one go, but instead has to be grown (and trimmed) over a period of time to fit the needs and capabilities of the actors involved in communicating (programs), whether humans or machines. Well, Lisp in general, and Common Lisp in particular, has got the growing bit just right, where most other languages just don't get it: with macros and reader-macros, you can effectively turn Common-Lisp into any language you like. To quote Guy L. Steele Jr., "If you give someone Fortran, he has Fortran. If you give someone Lisp, he has any language he pleases."

However, since in Common Lisp these work by irreversibly side-effecting the read-time, compile-time, run-time and coffee-time environments, and since there is (currently) no standard way to cleanly restrict the scope of such changes or specify which set of changes you're using, any extension you use makes such side-effects exponentially more likely to confuse other people and programs, to the point that if you want your code to be reused by other people, you must refrain from using such extensions altogether, and stick to the base language (which incidentally isn't all that portable, with plenty of unspecified, mostly uncheckable corner cases).

And so, while Common Lisp enables you to grow your own language, it makes it difficult for that language to be the same as anyone else's. You're either using Common Lisp without much in terms of extension, in which case it's a rather bad programming language as compared to all there is out there; or you use it with plenty of home-grown extensions, in which case you're growing your own autistic language that only you can use -- plus you have to do all by yourself the heavy lifting of making that language not suck.

That's where XCVB comes into the fray. XCVB is a mechanism to specify a set of language extensions to use, and to restrict the scope of the use of such extensions. In other words, it's the bottom half of a module system. Unlike a full-fledged module system, XCVB won't handle syntax, namespaces, hygiene in macroexpansion, or any such fancy topic. However, XCVB will provide you with enough sanity that you will hopefully be able to handle these on top of XCVB. The promise of XCVB is that each module will be separately compiled, that the compile-time side-effects required to compile one module won't escape to break other modules.

Of course XCVB will also handle the proper staging of compiling the required compile-time dependencies (e.g. macros and declarations) before to attempt compiling the modules that use those dependencies. It will also play well with your larger-than-Lisp build (Makefile), and will even allow you to parallelize your Lisp build in the marvelous new world of multiprocessors. In the future, it may even handle distributed compilation farms, etc.

All these features will I hope make XCVB an attractive proposition, at least when XCVB itself will have grown beyond the current prototype. But the deep purpose of XCVB is not incremental or distributed compilation, it is to make Common Lisp a proper platform to grow languages spoken by more than one person.

• Getting started in Common Lisp on Ubuntu is a decent guide to getting CLISP and SLIME running on Ubuntu, with Ubuntu packages
• Daniel Herring wanted me to mention read-macros. It's not asdf-installable, is barely documented, doesn't include any version information, and uses a weird (for Lisp) license. Enjoy!
• Alexander Lehmann created some highly regarded screencast tutorials about writing a ray tracer in Common Lisp. I love to see people doing stuff like this, but personally rarely find the time to actually sit down and watch them, which is frustrating. Alexander has created over five hours of Lisp tutorial material, so even if I don't find the time to watch it, you should.
• Mikael Jansson wrote up the retro Lisp meeting in Stockholm.
• Have you been looking for a really long list of websites powered by Lisp? Here's one such list, powered by Applied Stacks.
• The Common Lisp Weekly News site has started, with two weeks summarized so far. I like it.

A note from Franz, Inc:

Franz Inc. is the provider of Allegro Common Lisp and associated
products.  We are looking for a qualified applicant to handle the
following responsibilities:

 * Special Lisp-based projects–some of it is work on internal
  programs and processes, but some is on IT-related external
  products.

 * IT administration and maintenance of internal computer systems.
  Possible tasks could include:

    - hardware and software upgrades of Linux or Windows servers
    - hardware and (system) software troubleshooting
    - virtualization of existing servers
    - network management

Experience in the following areas would therefore be valuable:

 * Common Lisp, C, Bourne shell, Perl, and Javascript.

 * Linux installation and administration - shell scripting, perl, NFS,
  autofs, DNS, DHCP, SMTP, Samba, iptables, RAID and LVM.

 * Windows (2000 and later).

 * Mac OS X experience is a plus.

To start out with, this is a part-time position.  If things work out
well, it might turn into a full-time position.

If you feel you are a good match for the above, please send a letter
of interest and your resume to hrbox at franz dot com.  EOE.

Nice! I went away on a business trip the early part of this week and came back to find that we're over 450 members on LispForum. 451 to be exact. I have been really happy with the discussions happening there. Lots of good info being exchanged and none of the attitude of comp.lang.lisp.

Plenty of Hackers: I heard over the grapveine that I am "the last man standing" when it comes to SBCL development. While it is true that I am the most active committer at the moment -- which can give the casual eye a wrong impression -- my disproportionate number of commits does not mean others are idle: over the last ~9 months 11 different committers have been active. This does not include non-committers who have submitted patches, so the real number is somewhat higher.

Interesting Stuff in CVS: I have just merged two noteworthy patches. One fixes stack allocation by nested structure constructors (bug 430), and is very much worth checking out if you have structures that could be stack allocated. The other addresses some long-standing thread-safety issues in CLOS: most notably now DEFCLASS, ENSURE-CLASS-USING-CLASS, and obsolete instance updating is expected to be thread safe. Bugreports and possible performance regression reports most welcome on sbcl-devel, as ever.

• On one hand, I want to offer a ressource for people that already have experience with Common Lisp to gain even more thorough understanding of the language---so they can ascent to the next level so to speak.
  

• On the other hand, I want to preserve information for myself. The human mind is generally not capable of keeping too many details for a long period of time around. Hence, I'd like to use this blog as a swapping device for stuff I once spent time on.