Restarts in Common Lisp
sulami.github.io
sulami.github.io
> While Common Lisp calls them “conditions”, I will stick with “error” in this post, because I don’t think “condition” is very self-explanatory if you’re not familiar with the language already. Please forgive my inaccuracy.
This is misleading. It is possible to signal conditions that are not errors, and therefore do not require stack unwinding, like errors do. The condition facility allows one to execute arbitrary code with arbitrary arguments at the SIGNAL site.
`SIGNAL` walks the list of condition handlers and executes the type-matching ones in order of their binding. If a handler returns normally, then next ones are executed; if all matching handlers are exhausted, `SIGNAL` returns, and code execution continues.
CL-USER> (handler-bind ((condition (lambda (c) (format t ";; Foo: ~A~%" c)))
(condition (lambda (c) (format t ";; Bar: ~A~%" c)))
(condition (lambda (c) (format t ";; Baz: ~A~%" c))))
(signal 'condition)
42)
;; Foo: Condition CONDITION was signalled.
;; Bar: Condition CONDITION was signalled.
;; Baz: Condition CONDITION was signalled.
42
While I understand that this post is trying to appeal to language newcomers, simplifying the whole condition system to an "error try-catch" is a drastic misrepresentation of the condition system that will leave the newbies with the impression that that's all that is possible with that feature. Since it seems that the author is a Lisp newcomer as well (which I wholeheartedly admire! :), I'd advise him to use the original notation of "condition", instead for the one of "error".
You can implement dynamically scoped variables with a stack of environments, onto which you push entries atop when entering new dynamic scope and from which you pop entries when leaving a dynamic scope. (That is how the original dynamically scoped systems, including Emacs Lisp, were implemented.)
Example: https://github.com/svetlyak40wt/python-cl-conditions
Having the call-stack available at the time the handler runs can be very useful; many modern dynamic languages will save some of that information in the exception object. It's still surprising to me that so very few languages have copied this feature (or even independently discovered it).
(I agree it's a nitpick, but since your comment is likely directed to people who do not know the condition system, it's better to be precise.)
It's vastly simpler in languages where you have resources that must be freed in the correct order (C++ destructors, Java monitors being released, etc.) to ensure correctness. It's difficult to correctly reason about mutating sate to recover from an error while other pieces of code might actually be holding the mutexes/monitors keeping that data safe.
It's an optimization, because you already have to do much of the work of stack unwinding (in the absence of destructors, monitor releasing, and the like) in order to find the correct exception handler.
(This is particularly visible when debugging, because in the debugger, you still see the original stack at which the exception occurred when it's reported as unhandled - but it had to walk the stack and observe that there's no matching handler to determine that it's unhandled!)
This has an interesting side effect - since .NET catch-blocks can have filters, which are just predicates. Those predicates have to be executed while searching for the handler. Thus, they can observe the original stack before unwinding.
Win32 SEH is similar, wrt filter expressions in __except blocks.
I'm pretty sure that most C++ implementations are similar, except that they don't have filters, and so the two-phase process is not observable from within the application itself (but is observable under debugger).
Python, on the other hand, really does unwind the frames to find the handler - every function only looks at its own handlers, and if it can't find one, then it re-raises the exception to the caller. Thus, you can't readily tell if an exception is handled or unhandled in Python at the point where it occurs.
Not sure why that was considered useful in Visual Basic, of all languages :).
Old basic had an error handling model that is exceptionally elegant in some cases: "on error goto ...", with an an indication (errline, I think?) of where the exception happened, and then you could "resume next" to continue (perhaps after fixing the error condition, if the error handler is somewhere else) or directly restart at an earlier point if that makes sense.
It's simple, a little goto-ish, and has abuse potential - but if actually used, tended to end with robust code where the "usual path" was the simplified "that's what we usually do", and all the "in case something was unexpected" are listed independently. C code using goto for error handling (e.g. in the linux kernel) has similar properties.
try/catch, in all languages it exists, does not let you restart in the middle unless all the error handling code is in-line (which is comparatively disgusting IMO); lisp conditions do, and more - but they only exist in Lisp.
Windows's SEH is essentially simplified CL condition system.
However, there is no Restart system, allowing the code that raises an exception to also declare how it can be handled. And there are no alternative exception handling strategies than searching the stack and unwinding in case a suitable handler is not found. Unwinding is not a property of Conditions in general CL, it is just one of the strategies that can be used when signalling a condition.
I have often wondered why this is. It seems like Lisps did so many things right and here we are in These Modern Times waiting for people to rediscover "the secret is to bang the rocks together, guys".
Are these features just hard to implement in some platforms/languages? Why? Have they been de-prioritized generally until the zeitgeist decides otherwise?
Look at Rust removing CL-style conditions for a good example and Rust is by no means a popular language or a language that makes technical sacrifices in the interest of popular appeal (unlike Python and Javascript).
Making stack undwinding the decision of the exception/condition handler is strictly more powerful, but it greatly increases the number of corner cases that need to be considered when writing correct handlers. Just as a small subset of the issues, consider some data structure for which a mutex must be held in order to correctly recover. Assuming the mutex isn't held at the time the handler is installed, you need to consider 3 cases: (1) the mutex isn't currently held (so the handler must acquire it and remember to release the mutex before resuming normal execution), (2) the mutex is held by some active call frame sitting between the thrower and the handler (in most cases, it would then be safe to modify the data structure, but the handler absolutely must not release the mutex), and (3) the mutex is held by another thread (it's probably not possible to recover in this case).
Standard exception handling is non-local control flow, and that can make some situations difficult to reason about. Exception/condition handling where stack unwinding is optional is non-local control flow on steroids, engaged in 3D Chess Boxing.
That's a fundamental difference between Common Lisp and other more popular languages. CL tries to give you all the tools and trusts you to use them responsibly. Other, more opinionated languages, limit the problem solving approaches they offer in the interest of (popularity|performance|implementation simplicity|personal philosophy).
The CL philosophy makes a lot of sense if you examine its background and the culture that birthed it: a language designed to solve hard problems that did not have well-defined solutions.
1. It's a non-obvious solution (Lisp had (and in some cases, still has) other methods of doing what other languages do with exceptions before the current system
2. It was hard to implement in popular languages; any language without closures is unlikely to have this.
3. More dynamic languages that could implement these fairly easily inherited their exception system from less dynamic languages, with some other features tacked-on, or got the dynamic features after the exception system was already created.
4. Situations in which you aren't going to restart are already (mostly?) solved by the "shove the call-stack into the exception object" which significantly reduces pain compared to e.g. C++ exceptions.
Fun fact: the restarts based CSV parser was based on some real world work me and a colleague had done for an online travel portal. I wrote about it here: https://www.reddit.com/r/lisp/comments/7k85sf/a_tutorial_on_...
My goto recommendation for newcomers is Norvig's PAIP which is full of insightful moments and unbelievably good code and for people with more experience "Let over Lambda" by Doug Hoyte, which is a mind-blowing book mainly because of the time-transcendent nature of the examples Doug chose.
I know next to nothing about AI, beyond the basics of RNNs, so I'm hoping to learn a lot.
(Not to discourage you from reading the book, of course!)
Treat it as a review of some of the historical classics of computer science and you can't go wrong.
Knowing classical AI IMHO should be considered as part of programmers toolbox. What was once bleeding edge AI is now high level heuristic problem solving.
I agree that the MP3 streaming server example in PCL looks outdated, but so are many topics in PAIP as Norvig himself points out at the end of his retrospective [1]. (And I still think that the MP3 parser in PCL is a really good example for showing Lisp I/O, binrary processing abilities, and subtleties of CLOS, despite not being something that anyone would code as a side project nowadays.)
Some of the examples in PAIP may be outdated in a historical but not practical sense. I've joked in the past that the code is so well-written than it can be excised and framed on a wall. With code this good, the lessons one can learn are numerous and the code stands the test of time.
Unfortunately, the same can not be said about the PCL examples since - to name two - there are far better ways to do binary parsing and pathname handling today than binary-types and cl-fad. The PCL examples are a product of the time that they were written but they are tied to a particular verbose, subjective style and were not "optimal" or even particularly good, back then.
Ultimately, PCL introduces Common Lisp to the audience as does PAIP. The difference is that reading PCL in 2020 will have minimum, let's say thought-provoking, impact. Reading PAIP (or Let over Lambda, SICP, On Lisp, Lisp in Small Pieces) remains -and will remain- a paradigm-shifting experience.
Binary parsing depends on the application constraints.
You can't go wrong by studying Frode's binary-types (not the same as PCL binary-types) though [2].
It does come together however, with some additional effort and extra passes.
Guys, this is software not Milan Fashion Week. It's okay: those are just examples. Software still has to parse things, even in 2020!
P.S: I found the examples/practice part quite boring in both cases but for sure PAIP is more interesting as there is a chance to learn something new/unusual.
The documentation has dialect notes about this with a comparative example:
I wouldn't recommend using it, but it is an interesting error/control model, and I always wondered why it hasn't seen wider implementation - I presume because it is closer to the "dangerous" end of the power spectrum.
Raw css is here: https://github.com/edwardtufte/tufte-css
My additions on top (and all other source) here: https://github.com/sulami/sulami.github.io/tree/develop/css
Scheme has them (continuations) explicitly. CL chose not to, I think for performance reasons if I recall correctly.
Handlers are the actual hooks; conditions are the object that may trigger some of the hooks when they are signaled. Restarts are an addition on top of the hooks that allows for easier non-local transfer of control; they provide little that handlers do not provide unless you count the Lisp debugger - by design, the only way to leave the debugger is by using a restart.
In other words: no continuations here.
https://z0ltan.wordpress.com/2016/08/06/conditions-and-resta...
The big plus of the Common Lisp condition system is that it gives you a programmable substrate that you can use to implement a wide variety of signaling and control flow systems. It does give you error and exception handling -on top of the condition system- but that shouldn't shoehorn your thinking into just these problem domains.
http://jacek.zlydach.pl/blog/2019-07-24-algebraic-effects-yo...
Progress can be signaled, and then dealt with if needed. In headless mode it won't be handled. In a console it could be a progress bar or statement. In a GUI, update a variable that changes the status display.
And it ends up as little more than a no-op if there are no handlers for it. Nothing exceptional about it, but creates a great deal of flexibility in managing a system.
According to a Honeywell PL/I reference manual, "a condition is a state of the executing program".
The progress of some activity being 50% through is a state of the executing program.
(All the built-in PL/I conditions are exceptional situations though. This is really the same use of the word "condition" that we find in "condition status register", a machine word that tells us via flags that exceptional situations have happened like numeric overflow.)
If every time somebody writes something about Lisp and it doesn’t tell 100% of the story, and somebody else chimes in “well ackchewelly you must not be very experienced”, nobody is going to write Lisp.
I think the author did a phenomenal job referencing the book they’re reading, and discussing what they learned.
Besides, when’s the last time you or anybody else in this thread actually defined and used in a serious manner a non-error condition? It happens, but beyond WARN, it’s so rare and you probably want to use other dynamic constructs anyway.
The author could improve simply by saying “Conditions are most often used for error handling” and change little else.