Hacker News Reader

Comment by flavio81 | Hacker News Reader

What is even more amazing is to start a web server from the Lisp command line (the "REPL"), and then change the definition of a function. The app will use the new function without stopping and/or restarting. And this includes the intermediate step of compiling the whole function!!

You can also update the definition of a class and then force all instances of this class to use the updated definiton... all of this while the program keeps running.

I always tell my friends to try some Common Lisp. The runtime environment is rather amazing. The exception handling system ("condition" system) is also another thing that is impressive on CL. It is so far beyond what Java (or C++) brings to the table, that it isn't even funny to compare them.

View on HN

I've sometimes used a snippet like this:

    (defun rebuild (&optional (sha1 (sha1-of-current-commit)))
      (asdf:operate 'asdf:compile-op :my-package)
      (asdf:operate 'asdf:load-op :my-package)
      (push sha1 *sha1s-of-loaded-commits*))
    
    (defun rebuild-if-necessary ()
      (let ((sha1 (getf (current-commit) :sha1))
            (loaded (car *sha1s-of-loaded-commits*)))
        (unless (equal sha1 loaded)
          (format t "Rebuilding since ~S differs from ~S.~%"
                  sha1 loaded)
          (rebuild sha1)
          t)))

And then in for example a web service project, I would just call that function in the basic request dispatch handler.

When a web request comes in to the Lisp system, it will check the working directory's Git repository. If the HEAD points to a new commit, it will compile the new code (to machine code, with optimizations!) and reload the whole web app package. When the REBUILD-IF-NECESSARY function call returns, the new code is already loaded, and the web request proceeds as normal, but with the new code loaded.

And yeah, the conditions system is really radically better than exceptions. Raising a condition doesn't unwind the stack, but just calls the closest condition handler, more like a function call than an exception. The handler is offered a choice of restart opportunities, so for example a FILE-NOT-FOUND condition might offer to restart with a different file path, to just retry, or to error. Having restarts available for almost all exceptional conditions makes the system much more flexible.

What I can imagine freaking people out looking at your code is not knowing what, if anything, is atomic or otherwise in a transaction. Presumably that's well-defined by the ASDF operations and whatever multi-threaded solution you're using in Lisp along with the Lisp runtime guarantees, like presumably (I haven't looked) if you compile and load a package through ASDF it will do it all at once (but for each thread?) and you won't get in a state where one thread has a reference to both an old and a new function in the package. You might still have an issue where one thread expects a database schema that no longer applies because you've updated the schema and the mapping code was updated by the package reload but this thread hasn't had that happen yet...

For your web system, are you guaranteeing that if any subsequent requests come in, they wait for the current request to completely finish before being processed? If so, at least there's no obvious bug and the recompile only happens once, but that will turn people away when we're in a many-CPU age. So it's really not that different from the CGI model that PHP has used so successfully. FTP a new copy of the PHP file onto the server, presto, all new requests use that code and you didn't have to restart anything.

Oops, forgot to mention that this is a total hack that I would never put in production. I'm actually very curious about the atomicity of ASDF loads, and I don't know how it works. I wouldn't expect it to be for example atomic with respect to building several dependencies, but maybe it is.

The difference from the CGI model is that the Common Lisp runtime is stateful. In this particular experiment, I was working on a phone menu system, and it was really convenient to have the state of that system as Lisp variables.

You can also do that on Lisp's cousin, Smalltalk. :)

Have you ever delved into Xerox PARC documentation?

Lots of goodies regarding Interlisp-D, Smalltalk and Mesa/Cedar.

Java, .NET and Swift are probably the only modern environments that expose that experience, even if not at the full extent of those environments.

Of course, given the budget, I guess a better option would be to buy a license to one of the surviving Common Lisp commercial environments.

Here's a link for you to give people on Smalltalk hotloading:

https://www.quora.com/What-was-it-like-to-be-at-Xerox-PARC-w...

The impressive part is how Jobs would complain about something in the desktop with Ingalls just changing it on the spot with Smalltalk's rapid prototyping and live update abilities. Key example:

"One of Steve’s ways to feel in control was to object to things that were actually OK, and he did this a few times — but in each case Dan and Larry were able to make the changes to meet the objections on the fly because Smalltalk was not only the most advanced programming language of its time, it was also live at every level, and no change required more than 1/4 second to take effect."

> You can also update the definition of a class and then force all instances of this class to use the updated definiton... all of this while the program keeps running.

Right. In fact, you can define a method to be run on instances that were created before the class was redefined, that updates them for the new definition. So if you add a slot to the class, and you need it to be initialized in some way, you can define this method to do what you need.

Also check out https://www.snellman.net/blog/archive/2015-07-27-use-cases-f... for some really interesting, but niche, things you can do to classes.

I have the same discussion every few months, using FORTH as an example. I guess there are a lot of environments where you can use this style of dynamic updates - TCL, FORTH, Lisp, & etc.

And Erlang! Don't forget Erlang :)

Yes, with Tcl starting a server from an interactive interpreter and changing function and class definitions on the fly is trivially simple.

I'm writing a Tcl web sever that uses Tcl's namespace features to change the execution environment the client sees on a per-connection basis. Trivial. Anybody else got that?

I think people have done this I'm c/c++ with loading shared libraries at runtime. Its just a bit ugly BC the tooling isn't there. But I don't think there is anything inherently magical about it. To oversimplify it, you just have your function table point at a new location with the same signature, no?

Here is an example in C http://nullprogram.com/blog/2014/12/23/ , you can also check the Handmade Hero Youtube series for a more elaborate example.

Please note that the above approach is not as flexible as what you can do with Common Lisp. Also, you will need to use a subset of the language in the case of C++.

You can kind-of do it in C/C++. Unless the function is inlined. The sad reality is that the optimizer screws you over here - either you run your code unoptimized, with all the metadata present (no inlined functions, every function and every method of every class has a symbol, etc.), or you run your program fast. There is no notion of de-optimizing the binary, replacing the definition of a function and then re-optimizing.

You could write that, but it would have to be some kind of hypervisor that combines make+gcc+ld in one package that can dynamically recompile, relink and replace code that changes.

Is CL doing just that?

Reply on news.ycombinator.com