Andrey Listopadov

This is yet another follow-up post in the Emacs configuration series, which is also about Tabs. The previous post was about how tabs behave when you close them, and how I think the algorithm can be improved. This post is more about visuals and horizontal space management. By default tabs in Emacs are named after their respective buffers. This is how tabs look with my configurations and default naming: Looks fine, but there’s a catch. Tabs with really long names take a lot of horizontal space, which can be a problem when dealing with buffers that were created by some package, like CIDER: With tabs of this size, no wonder we get complaints that tabs are not usable. And on this screenshot, you can see that all tabs have different widths, which doesn’t really look good in my opinion. Both problems can be fixed with some code, but first, let’s check how the Atom editor handles tab sizes. ## Tabs in Atom This is the default look of tabs in Atom editor with One Dark theme: By default tabs in Atom occupy more space than is actually needed for a tab, and I think that this is beautiful. Someone may be concerned that it wastes horizontal space, but it’s actually not - if we open more tabs, their width will shrink to make those fit in the window: But this not going to happen all the time, and eventually, once some minimum width is reached, tabs will no longer shrink, and we’ll have to scroll those. We still can get the idea of what file is opened in the tab, and there are a lot of them in such a tiny window. And if some tab has really long name that doesn’t fit into a tab, it gets truncated: This is the behavior we’re going to replicate in Emacs. ## Making Emacs tabs resize automatically First, we will need two variables - one for minimum width, and one for maximum width. I like to define variables that extend some built-in packages via defcustom because if someone else will use my config they will be able to customize such things with the custom interface or via code. Also Custom makes persistent configurations a bit easier as well. emacs-lisp (defcustom tab-line-tab-min-width 10 "Minimum width of a tab in characters." :type 'integer :group 'tab-line) (defcustom tab-line-tab-max-width 30 "Maximum width of a tab in characters." :type 'integer :group 'tab-line) Good. This will later be used in a function that creates a name for the tab. Now we need to think about the general algorithm. Note that we’re defining size in terms of monospace characters. This algorithm will not work for variable pitch fonts. In Atom, tabs use fixed width up until there is no more horizontal space in the window to fit full-sized tabs. In Emacs, we can access the width of a window with the window-width function. So our first step will be to obtain such width, and check how many tabs will fit into it. However, instead of checking the number of tabs, we will calculate the width of the tab that we will need for all tabs that we currently have: ![\text{window-tab-width}=\frac{\text{window-width}}{\text{amount-of-tabs}}] So, if our window is 300 chars wide, and we have one tab, it will occupy the whole width. If we have two tabs, each should be 150 chars long, and so on. This is great, but if we have 70 tabs, each would be only 4 chars. Given that we need at least one char to display the left padding, one char to display the close button, and one char for the right padding, we’re left with 1 char for the name which is not good at all. However we don’t want to use such wide or such narrow tabs, so we need to adjust their length based on these conditions: ![ \text{tab-width}=\left(\begin{cases} \text{tab-line-tab-max-width}&\text{if window-tab-width}\geq\text{tab-line-tab-max-width}\ \text{tab-line-tab-min-width}&\text{if window-tab-width}<\text{tab-line-tab-min-width}\ \text{window-tab-width}&\text{otherwise} \end{cases}\right)-\text{close-button-width} ] This way if we have a tab width larger than our maximum width, we will use maximum width. If we have a width that is less than our minimum width, we will use minimal width, and this will enable scrolling. If we’re in between, we’re using the width that was calculated in the previous formula. Note that we’re subtracting the size of the close button from our width which is represented as × with spaces at the front and the end. The second thing we need is to produce a valid name for the tab, but right now we will simply string-trim buffer name, and calculate its width. This should leave us with these computations: | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 | emacs-lisp (defun aorst/tab-line-name-buffer (buffer &rest _buffers) (with-current-buffer buffer (let* ((window-width (window-width (get-buffer-window))) (close-button-size (if tab-line-close-button-show (length (substring-no-properties tab-line-close-button)) 0)) (tab-amount (length (tab-line-tabs-window-buffers))) (window-max-tab-width (/ window-width tab-amount)) (tab-width (- (cond ((>= window-max-tab-width tab-line-tab-max-width) tab-line-tab-max-width) ((< window-max-tab-width tab-line-tab-min-width) tab-line-tab-min-width) (t window-max-tab-width)) close-button-size)) (buffer-name (string-trim (buffer-name))) (name-width (length buffer-name))) | | ------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | Now, when we have all this information, we can compute the name for the tab and its paddings. First, we need to check if the trimmed buffer name exceeds the tab width. If it is, we truncate the name to the width of the tab minus 3, because we need to add single space padding before a name and add an ellipsis symbol followed by a space at the end. | 17 18 | emacs-lisp (if (>= name-width (- tab-width 3)) (concat " " (truncate-string-to-width buffer-name (- tab-width 3)) "… ") | | ------------- | ----------------------------------------------------------------------------------------------------------------------------------- | If name-width is less than tab-width minus 3, we can produce left padding, by computing difference between tab-width and name-width, and dividing it by 2. Then we concatenate this padding and calculate the right padding needed for the name. This extra calculation is needed when a name is odd or even, so we produce equally sized tabs for any buffer name: | 19 20 21 22 | emacs-lisp (let* ((padding (make-string (/ (- tab-width name-width) 2) ?\s)) (buffer-name (concat padding buffer-name)) (name-width (length buffer-name))) (concat buffer-name (make-string (- tab-width name-width) ?\s))))))) | | ------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | This gives us such results. With the same window width as in Atom examples, two tabs will occupy this amount of space: If we will open four tabs, the size of each tab will shrink accordingly: Though this is not as nice as in atom, due to the small unused space on the right side of the window. Unfortunately, we can compute width only in terms of characters, which can’t really have variable width, unless we use variable pitch font, but as I’ve mentioned earlier, this algorithm will not work with variable-sized fonts, and perhaps such computation will be quite expensive. Computing variable width in terms of pixels may be possible, but I couldn’t find any information on this topic. But the character-based solution should also work fine in terminal Emacs. When we open too many tabs, we can see scroll buttons, and tabs no longer shrink in size beyond tab-line-tab-min-width value. We also can disable the close button if we want: ## Rationale Before: After: In the beginning, I’ve said that there are a lot of complaints that tabs unnecessarily waste horizontal space. Even though we’ve fixed this, now you may think, that this is still impractical, because you can’t see the whole name of a file in the tab, and tabs are not usable as a concept overall. And I can agree to a certain point. An interactive buffer list, that we can fuzzy match through, has no issues with space because we pop it up when needed, and can show you full buffer names any time. But I find myself using tabs very often simply as a visual indicator of what buffers are next and previous in the list, and switch between those via C-x right and C-x left. You can’t really do this with a buffer list unless you display it somewhere all the time, but then it takes much more space than tabs. Tabs also help a lot when someone, who isn’t an Emacs user and is unfamiliar with the concept of buffer list, sits near me, since the concept of tabs is well known and obvious to most other people, because of tabs in browsers or other editors. One thing that I have not yet succeeded to achieve is to automatically recalculate tab width on frame or window resize events. It seems that force-mode-line-update does not initiate the process of name building, but for some reason, it happens on focus events, so I have yet to investigate this part. Turns out there’s a way to update tab width on resize events by clearing the cache of tab-line in each window. As a quite dirty hack, we can use this hook: emacs-lisp (add-hook 'window-configuration-change-hook #'(lambda () (dolist (window (window-list)) (set-window-parameter window 'tab-line-cache nil)))) This makes tabs resize whenever you change window or frame width with your mouse, or when you split windows via C-x 3 and other windowing commands that change window layout. Though this post only mentions the naming function, I’ve set up a bit more things to make tabs look as represented on screenshots. This configuration can be found in full form in my .dotfiles. Comment via email

While working on my previous post I was mainly using Emacs, because it has the best support for Lisp languages. It has great integration with the REPL, can run a server for my application in the background, and so on. And actually, I use this a lot while working on this blog - I run hugo process in the background to see how my page is looking. This got me thinking about handling all of this in Kakoune. Kakoune has facilities to run processes in the background, and it does this for many tools like grep, make, ctags, and other common shell tools that we use often. Kakoune also has built-in REPL support with repl and send-text commands, which can be used when I work with Lisps. However, there’s a problem. When working with Clojure, I need to run two JVM instances in the background. This is due to the architecture of the project, and I can’t really do anything about it. So when I’m programming in Emacs, an amazing package, named CIDER(Clojure IDE that Rocks) does this for me. But in order to do this in Kakoune, I have to start another terminal or create a new window in Tmux, and I don’t really like this. So I’ve thought, why not start this from Kakoune itself? The problem though is that there’s no built-in way that I can think of. There are ! and | commands to run a synchronous process and pipe its result into a buffer, which is not for me. I’d also like to kill a process if I don’t need it, and see its log as if it ran in a terminal. Turns out it isn’t a really hard task. I’ve created pmanage.kak plugin to test this idea. This plugin is quite small and has these three commands: - pstart <command>: Starts command as a background process, and creates FIFO buffer for that process named after process PID. - pstop <PID>: delete process buffer and send SIGTERM signal to process. - pkill <PID>: delete process buffer and send SIGKILL signal to process. So now, I can do this: - open Kakoune in project root; - call :pstart 'npx shadow-cljs watch app' to start background watching process. * A new FIFO buffer is created, and process PID is stored into a special option; - Then I can do :repl 'npx shadow-cljs cljs-repl app' and open REPL window that will be connected to that watcher process; - Develop! I am also now able to run :pstart 'hugo server -D', and edit any Markdown files that my blog is made of. And when I don’t need that process I simply can call :pstop PID or :delete-buffer PID. I know about kakoune:1414, but I think the solution I came up with is fine for most use cases. Comment via email

For a long time, I was a fan of Hack font. It has really nice language support, great readability at a size of 9pt, and zero with a dot. I love it when zero comes with a dot. Many fonts use zero with a line, to differentiate it from capital O, but on small sizes, it is not great, however, a dot looks fine when both small and big. But one thing was lacking from Hack, and it is ligature support. I always wanted to try out ligatures, but there were no other fonts that were visually close to Hack, appealing to my taste, and had good programming ligatures support. I’ve tried Fira Code, but it was too curly for my liking, Iosevka was a bit too narrow, and there’s also Hæck, that is based on Hack, and has ligatures, so looked like a perfect candidate. But my eye was caught by JetBrains Mono. While it’s not the same as Hack, it is visually similar, has great ligature support, and looks a bit nicer in Emacs! The Hack font works for me in a terminal, where I don’t need ligatures, while in Emacs something a little thicker is preferable. So I’ve chosen it, but the ligatures are not working out of the box. ## Enabling support for ligatures There are several options for enabling ligatures in Emacs, listed at Fira Code Wiki page. There are four options: - composition mode in Emacs Mac port - prettify-symbols - composition char table - font-lock keywords We’re interested in the composition char table. The basic idea is to provide starting character, and a regular expression, that matches all ligatures starting with that character, and put it to the composition table. Here’s the code for JetBrains Mono: emacs-lisp (dolist (char/ligature-re `((?- . ,(rx (or (or "-->" "-<<" "->>" "-|" "-~" "-<" "->") (+ "-")))) (?/ . ,(rx (or (or "/==" "/=" "/>" "/**" "/*") (+ "/")))) (?* . ,(rx (or (or "*>" "*/") (+ "*")))) (?< . ,(rx (or (or "<<=" "<<-" "<|||" "<==>" "<!--" "<=>" "<||" "<|>" "<-<" "<==" "<=<" "<-|" "<~>" "<=|" "<~~" "<$>" "<+>" "</>" "<*>" "<->" "<=" "<|" "<:" "<>" "<$" "<-" "<~" "<+" "</" "<*") (+ "<")))) (?: . ,(rx (or (or ":?>" "::=" ":>" ":<" ":?" ":=") (+ ":")))) (?= . ,(rx (or (or "=>>" "==>" "=/=" "=!=" "=>" "=:=") (+ "=")))) (?! . ,(rx (or (or "!==" "!=") (+ "!")))) (?> . ,(rx (or (or ">>-" ">>=" ">=>" ">]" ">:" ">-" ">=") (+ ">")))) (?& . ,(rx (+ "&"))) (?| . ,(rx (or (or "|->" "|||>" "||>" "|=>" "||-" "||=" "|-" "|>" "|]" "|}" "|=") (+ "|")))) (?. . ,(rx (or (or ".?" ".=" ".-" "..<") (+ ".")))) (?+ . ,(rx (or "+>" (+ "+")))) (?\[ . ,(rx (or "[<" "[|"))) (?\{ . ,(rx "{|")) (?\? . ,(rx (or (or "?." "?=" "?:") (+ "?")))) (?# . ,(rx (or (or "#_(" "#[" "#{" "#=" "#!" "#:" "#_" "#?" "#(") (+ "#")))) (?\; . ,(rx (+ ";"))) (?_ . ,(rx (or "_|_" "__"))) (?~ . ,(rx (or "~~>" "~~" "~>" "~-" "~@"))) (?$ . ,(rx "$>")) (?^ . ,(rx "^=")) (?\] . ,(rx "]#")))) (let ((char (car char/ligature-re)) (ligature-re (cdr char/ligature-re))) (set-char-table-range composition-function-table char `([,ligature-re 0 font-shape-gstring])))) By enabling auto-composition-mode we can see the ligatures in action: Figure 1: Ligatures off Figure 2: Ligatures on > Hint: open images in separate tabs and toggle between those to spot the difference. Note, in order for ligatures to work, Emacs needs to have support for HarfBuzz which was added in Emacs 28. But I don’t want these ligatures to be enabled everywhere, only in programming-related modes, so here’s a config for composite package: emacs-lisp (use-package composite :hook (prog-mode . auto-composition-mode) :init (global-auto-composition-mode -1)) This takes care of most of the errors, listed under composition char table section in Fira Code Wiki. Also by using rx macro we avoid writing complex regular expressions with backslash escape hell, and handle this task to Emacs itself. It’s good when Emacs does boring stuff for us, isn’t it? Comment via email

Today’s topic will again be about text editing software (one of my favorite topics actually). However, instead of discussing text editors themselves, I’ll share my opinion on such things as editor-hopping. What is editor-hopping? It’s, well, when you change text editors every day/week/month/year. Not because you have to, as it is required by task or something doesn’t work as intended, but because you’re bored. To discuss this topic I want to list these questions I’ll ask, and (try to) answer: - Why this happens? - How this happens? - What should I do? - Is it okay to hop around? Without further ado, the first question: ## Why this happens? I don’t know. Seriously, why would I know? Ask scientists or psychologists. Why are you looking at me like that? Next question. ### How this happens? Why this might happen? On a more serious note, I think that most of the time this happens out of boredom. But I’ll get to this a bit later. Sometimes though, you have to hop around when you search for a specific feature, that is not well known. And it is hard to immediately tell which editor has it or has the best implementation. For example, Parinfer - it’s a niche plugin for Lisp editing. There are a lot of editors that are supported, mainly Atom, Visual Studio Code, Emacs, Vim, Cursive, Sublime Text, and others. But Parinfer is a tricky piece of software, and there are some quirks in implementation, therefore not all versions are equal. For example, parinfer-mode for Emacs, listed among other implementations, has no Smart mode1, as of this moment. And what exactly is a smart mode? It’s not presented on Parinfer’s web page, and it’s not clear what it does compare to indent or paren modes. To find out you either have to download another editor, which has it, but which one does? Therefore, the search begins, and hopping is kinda required. But again, this is not exactly what I mean by editor-hopping. Because there’s a difference when you change editors because you need to, or out of curiosity/boredom. Let’s take a look at how someone may become an editor-hopper. ## How this happens? Well, imagine this. You’re a happy Notepad++ user, and someone in a fabulous suit just passed by with the newest Macbook in one hand and a cup of elite coffee in another. But most notable, they were actively writing code in some dark-colored, stylish-looking, colorful text editor. Curious, you jump in front of this person, making they spill their coffee on you, and have this dialogue: – Excusez-moi, monsieur, what is this bizarre, cool-looking software you’re using for writing another software, that I have never seen? – Sublime Te.. And immediately you jump back at your laptop, open your favorite web search engine, and type in this newly acquired knowledge. You hit Enter, and you’re presented with a modern, slick website dedicated to demonstrating you key features of this editor. That’s it, you no longer can stand the look and feel of Notepad++, and you’re installing this new shiny toy on your PC. And it’s great, it’s fast, it’s everything you’ve wanted, and, uh oh… What’s this? Atom? VSCode? Brackets? Nah, Sublime is just fine! Only, there’s a plugin for Atom, that looks interesting… /Nonono, I’ve just switched from Notepad++, why would I change editor again? On the other hand… I’ve not that used to Sublime yet, so it’s not a problem to try another thing or two… Original comic by Extra Fabulous Comics Of course, this is more like a made-up example, although it is still a real scenario. Another possible way is by watching someone doing some weird things with their text editor, and wanting this kind of power. But since you have no skill yet, when you hop in, you quickly find that you can’t really do the same stuff, so you jump back to where you were. But now there’s a chance that you already shifted a bit more towards using another editor, and it will be buzzing in the back of your head until you give up. This can happen several times, thus making you shift more and more in small steps, eventually making you switch. I’ve tried to get into Emacs 5 times :)2. ## What should I do with editor-hopping-thingy? Is it a bad habit? This is an interesting question. Well, trying new things is good, it is like a breath of fresh air - you can find facilities you didn’t know you wanted, you can see different approaches to the same tasks you do, and so on. A friend of mine shoot a video about a similar problem: Is Distrohopping a Dangerous Addiction? This is a common problem of many Linux enthusiasts, and I’m no exception - I was distro-hopping for ~6 years. But if distro-hopping usually means changing preset environments in search of a perfect setup - most users hop between user-friendly distributions, like Linux Mint, Ubuntu, Fedora, ZorinOS, Pop_OS!. These distributions are more or less the same, and there’s nothing wrong with this. But some hoppers remain unsatisfied and go deeper into more advanced distributions, such as Void, ArchLinux, and Gentoo, which allow more configuration options. And there are two3 possible endings - either you return to one user-friendly distribution, like Fedora, and embrace it, or you shape advanced distribution into something you really like. Editor-hopping4 can start the same - you try out some preconfigured editors, like Atom, Sublime Text, or Visual Studio Code, and go down the rabbit hole to Emacs, Kakoune, Vim, Vis, Ed, Micro, Acme, and all those other weird editors available on GNU/Linux systems for everyone to use. I’ve done a pretty big review of text editors that I’ve used or tried among the years of my Programming career, so I was editor-hopping for some time already. But there’s a difference between distro-hopping and editor-hopping, that can easily be seen. In his video5, Wolfgang mentions that > Tinkering with your computer, and trying out different Linux distros is actually a very cool way to spend your time, and might teach you a lot of useful skills that you can use later for a job if you want to get into a Linux job, for example, DevOps, or SysOps, or Sysadmin. This can’t really be applied to text editors. Or can it? Maybe if we’re talking about configuring such editors, like VSCode, or Atom, then I don’t think that it can be really good for you or for your possible job. No offense, but I doubt that anytime soon we’ll see something like TextEditorOps job offers. However, being able to set up complex text editing environments still is a good skill. For example, knowledge of how to configure an IDE can help with the onboarding of a new employee. But I doubt that you’ll get paid for that kind of thing. However, if we’re talking about more complex editors, like Vim or Emacs, knowing how to extend those means knowing how to write programs. Because both editors are extended via programming in the respective language - Vimscript, or Emacs Lisp. Emacs Lisp can make you better, because it is quite a different language, and Vimscript also can teach you a thing or two. And yes, you can write plugins for VSCode and Atom in some dialect of JavaScript, and this can boost your skills in the same way as writing for Vim or Emacs. There’s also Kakoune. It is a bit different, as it has no programming language embedded into it, and uses shell scripting as a way of interacting with other tools and user scripts. When I switched from Vim to Kakoune, I didn’t knew awk or Perl, and thanks to Kakoune, I now know awk and Perl to some degree. Kakoune also expanded my knowledge of other command line tools and their parameters. But is it good? Yes. Do I need Perl or awk at work? No. Did it improve my skills? Yes. So, in my opinion, editor-hopping can be useful for your job, but only when you learn new things, and the things you learn to make you better at doing your job. ## To hop or not to hop? You decide, but my word is - go for it. You’ll never know where it will make you end up. I was pretty much a happy C programmer for a long time, but at some point, I discovered Emacs, and now my focus has shifted towards Lisp a lot. And I’ve tried a lot of other editors before that. Now C is no longer my main language, and I write more and more Lisp code. But even if not talking about Lisp, writing editor plugins also ignited my interest in programming a lot, and introduced me to a lot of new and fun tech, such as Kakoune. Of course, editor-hopping should not interfere with other hobbies and cause a loss of time. Too much tinkering with the text editor (or operating system) can be a problem if you notice that this is the only thing you do, and it’s not really making you better, or more productive. It’s good to know when to stop and observe what tools you have, and which one suits your needs best. My complete editor-hopping history is: Embarcadero Delphi → Code::Blocks → Gedit → Kate → Mousepad → VSCode → Vim → Kakoune → Atom → Emacs. It was somewhere in 2010, and Embarcadero Delphi wasn’t an editor, but an IDE, though it was the first thing I’ve started with as something I can code in. We used it at the university for doing our computer science classes. After some time we moved from Delphi to C, and the next thing I used was Code::Blocks. Again, it was fine for doing C, but I quickly understood that it is not really versatile, and can’t be comfortable used to take notes, or write prose. So I moved to Gedit, as at the time I was using Ubuntu, and it was shipped with Gedit by default. It was a nice experience, but there was a little number of plugins that I could utilize. Kate has proven that it is a bit better option, as it had many plugins, integrated terminal, snippets, and so on. But I’ve got a new job and Kate there was from the KDE4 era, thus outdated, and a bit buggy. Afraid of losing data, I’ve switched to Mousepad, while at the same moment tinkering with VSCode and remote file access. Unable to set up remote access without constant failures, I’ve embraced remote editing in Vim and used vim for about 4 years since. Until some of my friends, who were mentioning Kakoune every day, actually made me try it out, and I was hooked. I wrote a lot of plugins for Kakoune, and somewhere at this point I started tinkering with Lisps, and thus Emacs, and Atom. Initially, I’ve chosen Atom, as it had this very interesting REPL, but quickly understood that it is not as good as Emacs. And now I’m on Emacs, which I’ve modified to look as close as possible to Atom because I like how it looks. This was going on for the last ten years, and I don’t regret any moment in this journey. The first two switches were not because of hopping, but still gave me a taste of what different environments can offer. And yes, there was a situation when I saw someone using Sublime Text 2 on their Macbook, and I’ve actually tried it out, but it wasn’t free. Later on, Vim and Kakoune introduced me to some nice people, and we still chat with each other, even though I’m no longer using Vim or Kakoune. Don’t be scared of change and new things, try, explore, and experiment! --- 1. It has, but it is locked behind a branch and is not available for most Emacs users, who install packages through a package manager. ↩︎ 2. And now I can’t escape! ↩︎ 3. Three, if you count “using another operating system” as an option. ↩︎ 4. Not sure if this a real term as “Distrohopping”, but it represents the problem quite well. ↩︎ 5. https://youtu.be/ojC7NXE69z4?t ↩︎ Comment via email

Not long ago I posted a small article on a Condition System in Clojure language. In that article, I was mostly trying to understand what a condition system is, and how it can enhance error handling in the code I write. Since that time, I’ve understood this system a lot better, by actually trying it in the Common Lisp language, the place where it came from, as far as I know. And you know what it means (because you’ve read the title) - it’s time to implement such a system for Fennel! I like Fennel a lot, it’s a really cool language to tinker with - it’s small, slick, Lua runtime is actually really versatile, and can be bent to your needs without a lot of extra work. But the reason I’ve decided to implement this library for Fennel, and not for Lua itself, is because of a feature that is not available in Lua but is available in Fennel - lisp macros. The library is already available for everyone to use: fennel-conditions, and in this article, I’ll do a deep dive into the details behind the implementation. ## Condition system? First, let’s actually explain what the condition system is. Condition system is a feature of the Common Lisp language, that can be very roughly described as resumable exceptions. Though, while this isn’t the only way you can use the condition system, it is the main reason I’ve decided to port this system into Fennel, as Lua error handling is not the best, and we can do better. You still can use a condition system for basic control flow to some extent, though not as versatile as in Common Lisp, I don’t think that Fennel really needs these features. In Common Lisp, a condition is an object, which represents a certain event. You can bind handlers for particular events, and trigger these handlers by raising a condition. This may sound similar to the classic exception model found in languages like Java, but it has one major difference. Exceptions are a two-part system, while the condition system is three-part. It’s really hard to come up with a small example to show what condition the system has to offer that exceptions don’t, and I don’t really know Common Lisp well enough to write it. So I encourage you to read Chapter 19 from the Practical Common Lisp book which covers this topic in detail before continuing reading. There’s also a good article on the wiki.c2.com. If you’ve read that before or just now, and didn’t understand what the condition system is, hopefully, you’ll get the idea in this post, but I can’t guarantee that. If I were to describe the main difference between this system from classic exceptions, I would say that when you write a code that throws an exception, you have to deal with this exception somewhere else. And the place where you handle that exception is the place where the program would continue running after the handling is done. In the condition system, the place where you’ve thrown such an exception can be the place where the program continues running if you use some special constructs, that allow you to return to that place. ### An example In the spirit of the Practical Common Lisp book example mentioned above, here’s a rather naive Clojure code that parses a log file, and uses exceptions to indicate that parsing failed: clojure (ns log-parser (:require [clojure.java.io :as io]) (:import [clojure.lang ExceptionInfo])) (defn well-formed? "Checks if the line contains basic date-time, a service name, and a message separated by colons." [line] (let [date-re #"([12]\d\d\d-[01]\d-[0-3]\d)" time-re #"([01]\d:[0-5]\d:[0-5]\d\.\d+)" service-re #"([^:]+)" message-re #"(.*)"] (re-matches (re-pattern (str date-re "T" time-re ":" service-re ":" message-re)) line))) (defn parse-line "Extract `date`, `time`, `service` name, and a `message` from a `line`." [line] (if-let [[_ date time service message] (well-formed? line)] {:date date :time time :service service :message message} (throw (ex-info "malformed log entry" {:line line})))) (defn parse "Return a vector of parsed lines, each element being a map with the following keys: `:date`, `:time`, `:service`, `:message`." [lines] (try (mapv parse-line lines) (catch ExceptionInfo e (binding [*out* *err*] (prn (format "%s:%s" (ex-message e) (:line (ex-data e)))))))) (defn parse-log [file] (with-open [f (io/reader file)] (parse (line-seq f)))) > Note: Yes, there are more idiomatic ways of doing this task in Clojure, but for the purposes of demonstrating the difference between exceptions and conditions this will do. So this code checks if the line has a valid format, and if not it will throw an exception. And here’s a problem - if a single line is invalid, we will get no results at all. We can avoid that by moving try into our iteration: clojure (defn parse "Return a vector of parsed lines, each element being a map with the following keys: `:date`, `:time`, `:service`, `:message`." [lines] (mapv #(try (parse-line %) (catch ExceptionInfo e (binding [*out* *err*] (prn (format "%s: %s" (ex-message e) (:line (ex-data e))))))) lines)) This will correctly parse our log, but we will have nil in place of each malformed line. This can once again be fixed by using keep instead of mapv, but what if we can’t afford to lose any lines? Maybe we can fix the line and parse it one more time? This again can be done by modifying our anonymous function, so it would catch the exception, try to fix the line, and then call parse-line again. And if we can’t fix the line, we will log it as before, and don’t add it to the resulting vector: clojure (defn fix-line "We don't care when this happened, just want to know what happened." [line] (when-let [[_ service message] (re-matches #".*:([^:]+):([^:]+)$" line)] (str "1970-01-01T00:00:00.000:" service ":" message))) (defn parse "Return a vector of parsed lines, each element being a map with the following keys: `:date`, `:time`, `:service`, `:message`." [lines] (into [] (keep #(try (parse-line %) (catch ExceptionInfo e (if-let [fixed-line (fix-line (:line (ex-data e)))] (parse-line fixed-line) (binding [*out* *err*] (prn (format "%s: %s" (ex-message e) (:line (ex-data e))))))))) lines)) But then again, when we will use parse-line in some other part of the program we will need to do all this stuff over and over. The main problem here is that we’re building concrete ways to recover from error as a part of the calling function that is meaningful only in the context of the parse-line and the responsibility for recovering should really be on it. Condition system can help us write more generic code, that knows how it can recover from the error, but let the recovery happen on the calling side. Consider the following Fennel code that uses fennel-conditions library: clojure (local {: error : make-condition : invoke-restart} (require :fennel-conditions)) (import-macros {: handler-bind : restart-case : define-condition} :fennel-conditions) (define-condition MalformedLogEntry) (fn well-formed? [line] "Checks if the line starts with basic date-time followed by a colon and a word." (let [date-pat "([12]%d%d%d%-[01]%d%-[0-3]%d)" time-pat "([01]%d:[0-5]%d:[0-5]%d%.%d%d%d)" service-pat "([^:]+)" message-pat "(.*)"] (string.match line (.. date-pat "T" time-pat ":" service-pat ":" message-pat)))) (fn parse-line [line] "Extract `date`, `time`, `service` name, and a `message` from a `line`." (restart-case (match (well-formed? line) (date time service message) {:date date :time time :service service :message message} _ (do (print line) (error (make-condition MalformedLogEntry {:line line})))) (:use-value [x] x) (:reparse-line [fixed-line] (parse-line fixed-line)) (:ignore [ignore-val] ignore-val))) (fn fix-line [line] "We don't care when this happens, just want to know what happened." (match (string.match line ":([^:]+):([^:]+)$") (service message) (.. "1970-01-01T00:00:00.000:" service ":" message))) (fn parse [lines] "Return a vector of parsed lines, each element being a map with the following keys: `:date`, `:time`, `:service`, `:message`." (handler-bind [MalformedLogEntry (fn [e {: line}] (match (fix-line line) fixed-line (invoke-restart :reparse-line fixed-line))) MalformedLogEntry (fn [e {: line}] (io.stderr:write (.. "malformed log entry: " line "\n")) (invoke-restart :ignore nil))] (icollect [_ line (ipairs lines)] (parse-line line)))) (fn parse-log [file] (with-open [f (io.open file :r)] (when f (parse (icollect [line (f:lines)] line))))) For those unfamiliar with condition system special forms or Fennel itself, handler-bind is something similar to try with the difference that the only way to recover from the error is by calling invoke-restart 1. with-open is the same thing as in Clojure or Common Lisp, and icollect is something similar to mapv in Clojure, but it’s a macro, that accepts body form instead of function. So what’s different about this code from the code in Clojure’s example? Let’s look at parse-line functions in Clojure and in Fennel: clojure ;; Clojure (defn parse-line "Extract `date`, `time`, `service` name, and a `message` from a `line`." [line] (if-let [[_ date time service message] (well-formed? line)] {:date date :time time :service service :message message} (throw (ex-info "malformed log entry" {:line line})))) ;; Fennel (fn parse-line [line] "Extract `date`, `time`, `service` name, and a `message` from a `line`." (restart-case (match (well-formed? line) (date time service message) {:date date :time time :service service :message message} _ (error (make-condition MalformedLogEntry {:line line}))) (:use-value [x] x) (:reparse-line [fixed-line] (parse-line fixed-line)) (:ignore [ignore-val] ignore-val))) Apart from some syntactic differences, you can see that both functions are throwing an error - by using throw in Clojure, and error in Fennel. However, while the Clojure version only throws the error, the Fennel version does some additional things. What we’re interested in here is the restart-case macro, which is a special macro from the condition system, that allows us to return to its location when we want to handle the error. So when parse-line throws an error in Clojure, JWM unwinds the stack, and we basically leave this function completely. We then catch the exception in the parse-log function, which can’t do anything about it but handle it right there or let it fall through. However, when parse-line throws an error in Fennel, we don’t actually unwind the stack at all. Instead, we travel up the dynamic scope, in search of a handler bound to this error by the handler-bind macro in the parse function: clojure ;; ---- 8< ---- (handler-bind [MalformedLogEntry (fn [e {: line}] (match (fix-line line) fixed-line (invoke-restart :reparse-line fixed-line)))] ;; ---- 8< ---- ) This handler tries to fix the line, and calls (invoke-restart :reparse-line fixed-line). invoke-restart is another special function in condition system - it once again travels through the second dynamic scope, and finds that there’s a restart named :reparse-line bound by restart-case in the parse-line function. This restart is executed right there, despite the fact that we’ve “left” the lexical scope of that function to find a handler, and its value is used in the original parse-line call. So the order in which all functions are called looks something like that: In other words, when we wrote parse-line in Clojure, we only defined how to detect malformed lines and how to signal an error. When we wrote parse-line in Fennel, we not only specified what to do if the line is malformed but also provided concrete ways to recover, that are meaningful in the context of this function. When we caught the exception in Clojure, we had no choice but to handle it right there. In Fennel, when we caught the exception we could decide whether we can handle the error, and how we can handle it without leaving the original call. I hope this illustration is clear enough to demonstrate the main difference between exception and condition. Again, yes, there are better ways of dealing with this particular error in Clojure, but what I wanted to illustrate is that there’s no way of achieving the same flexibility of the condition system with just exceptions. I’ve covered possible ways of implementing a rather basic condition system in pure Clojure in my previous post Condition System in Clojure language, so it’s not entirely impossible, but has its own shortcomings. With that in mind let’s talk about the implementation of this system. ## The implementation As I’ve mentioned previously, the dynamic scope is what allows us to travel through our code without transferring the control. However, Lua doesn’t have a dynamic scope builtin into the language. So how is this possible? Well, as you might know, Lua has tables, and these tables are dynamic and mutable. So theoretically, we could create a table, that is available globally, properly encapsulate it, and use it as our dynamic scope. And that’s exactly what the fennel-conditions library does. However, implementing dynamic scope via tables is not enough. First of all, we need to manage the dynamic scope in a sensible way. Second, we need some way of actually traveling through the lexical scope to the point where the dynamic scope was created, once we know where we need to move after all handling has been done. And finally, the scope should be thread-local, and even though Lua doesn’t have threads, it has coroutines, which can act as threads, so we should maintain dynamic scope with respect to this aspect of the language. Let’s begin by listing things that will be implemented in the condition system: - condition objects - conditions are objects, that can have inheritance, and can embed additional information inside to convey more info about a conditional situation when needed. Condition objects are defined with define-condition macro, and a particular instance is created from a base condition object with make-condition function. - handler-case, handler-bind, restart-case - macros for creating new frame of the dynamic scope, in which the handlers, and restarts are bound to condition objects. As well as: - cerror - a macro for raising a condition that can be ignored by invoking special continue restart. - ignore-errors - a macro for suppressing all errors, similarly to try with a catchall clause that does nothing. - unwind-protect - a macro for catching all errors, similar to try with just a finally clause. All these macros share the very common part about setting up and cleaning the dynamic scope, as well as some pcall trickery to make it all work, which makes it a good entry point to understand what’s happening under the hood. ### The dynamic scope Beware, this is going to get really messy. Dynamic scoping is a foreign concept for Lua, so we have to deal with it ourselves, and ensure that it is correct. But before we begin, I should illustrate the structure of the library: text fennel-conditions/ ├─ impl/ │ ├─ condition-system.fnl │ ├─ debugger.fnl │ └─ utils.fnl ├─ init.fnl └─ init-macros.fnl The init.fnl, and init-macros.fnl at the top level are the main entry points for runtime functions, and macros. These modules depend on the impl/condition-system.fnl module - the library’s private API. Which in turn uses impl/debugger.fnl that holds implementation of the interactive debugger, and impl/utils.fnl which stores common functions, used by condition system and debugger. The main things we’re interested in are init.fnl, init-macros.fnl, and impl/condition-system.fnl - the meat and bones of the library. The dynamic scope is stored in the impl.condition-system module and this module is required every time we need to access the dynamic scope. Lua has a system for caching required modules, so this is a rather cheap operation and this way we don’t need to expose dynamic scope in the global scope. This is how the dynamic scope table is defined in impl.utils: clojure (local {: metadata : view} (require :fennel)) (local dynamic-scope (metadata:set {} :fnl/docstring "Dynamic scope for the condition system. Dynamic scope is a maintained table where handlers and restarts are stored thread-locally. Thread name is obtained with `coroutine.running` call and each thread holds a table with the following keys `:handlers`, `:restarts`, and `:current-context`. Handlers and restarts itselves are tables.")) It’s nothing special really, just an ordinary table, but the documentation string mentions several interesting points. First, that this is a maintained table, meaning that it should not be modified manually. Second, that it stores scopes in thread-local contexts, meaning that each thread will have its own dynamic scope table. Each such table in turn has a table for handlers, a separate table for restarts, and some current-context field, which we will cover a bit later. So, to illustrate how the dynamic scope may look at some point of the program running: clojure {"thread: 0x564cab9a12a8" {:handlers {:handler {#<condition MalformedLogEntry: 0x564caba2c6e0> #<function: 0x564cabb01500>} :handler-type "handler-bind" :parent {:handler {#<condition MalformedLogEntry: 0x564caba2c6e0> #<function: 0x564caba5ccf0>} :handler-type "handler-bind" :parent {} :target {}} :target {}} :restarts {:parent {:parent {:parent {} :restart {:args ["ignore-val"] :name "ignore" :restart #<function: 0x564cabb01580>} :target {}} :restart {:args ["fixed-line"] :name "reparse-line" :restart #<function: 0x564cabaa7a10>} :target {}} :restart {:args ["x"] :name "use-value" :restart #<function: 0x564caba5cd30>} :target {}}}} This is the state of the dynamic scope when we enter the restart-case macro and do all setups, in the fennel code from the example section. There’s quite a lot happening here - lots of nested tables with similar internal structures, and even some empty tables. First thing you can spot is that the outermost table has a key "thread: 0x564cab9a12a8" which holds the table with handlers and restarts. This is the thread-local dynamic scope, representing the main thread. Inside of it in the table under the handlers key, we see a table with a handler key holding a table with a condition, (which is actually just another table2), and a function: clojure {:handler {#<condition MalformedLogEntry: 0x564caba2c6e0> #<function: 0x564cabb01500>} :handler-type "handler-bind" :parent {:handler {#<condition MalformedLogEntry: 0x564caba2c6e0> #<function: 0x564caba5ccf0>} :handler-type "handler-bind" :parent {} :target {}} :target {}} This table corresponds to all dynamic scopes set up with handler-bind macros. But as you can see here, it has a parent key, that holds another very similar table, but as you might remember, there was only one handler-bind instance in the example code - why there are two tables? Here’s a quick refresher of how handler-bind was used: clojure (handler-bind [MalformedLogEntry (fn [e {: line}] (match (fix-line line) fixed-line (invoke-restart :reparse-line fixed-line))) MalformedLogEntry (fn [e {: line}] (io.stderr:write (.. "malformed log entry: " line "\n")) (invoke-restart :ignore nil))] ;; ---- 8< ---- ) Note that there are two separate entries for MalformedLogEntry condition. Each such entry creates a separate frame in the dynamic scope, which results in two nested tables as shown above. This is done this way because the handlers are bound sequentially, and their order in the handler bind determines the order of how they will be called. We directly use this property, by writing the first handler in such a way, that if the line can’t be fixed with fix-line function, we are not calling invoke-restart and the condition falls through to the next handler. This handler in turn calls :ignore restart, and successfully recovers from the error. Now let’s look at the dynamic scope stored under :restarts: clojure {:parent {:parent {:parent {} :restart {:args ["ignore-val"] :name "ignore" :restart #<function: 0x564cabb01580>} :target {}} :restart {:args ["fixed-line"] :name "reparse-line" :restart #<function: 0x564cabaa7a10>} :target {}} :restart {:args ["x"] :name "use-value" :restart #<function: 0x564caba5cd30>} :target {}} Very similarly, each level stores a :restart table, with an argument list, name, and the function itself. Under the :parent key, the very similar table is stored, holding the next restart. Since our code defined three restarts, we get a table of the depth of three. One might wonder, why this is necessary, restarts are selected and not tried in order. While this is true, nothing prevents you from creating two restarts with the same name: clojure (restart-case (/ 1 nil) (:use-value [x] x) (:use-value [x] x)) This will create the following table: clojure {:parent {:parent {} :restart {:args ["x"] :name "use-value" :restart #<function: 0x564cabb01380>} :target {}} :restart {:args ["x"] :name "use-value" :restart #<function: 0x564cabaa5b40>} :target {}} You then will be able to choose which restart to call via the interactive debugger interface. But more about that later. Now that we have a rough understanding of how dynamic scopes are composed inside, let’s look at how those are really constructed. ### Constructing dynamic scope for handlers Let’s look at the handler-bind macro invocation once again: clojure (handler-bind [MalformedLogEntry (fn [e {: line}] (match (fix-line line) fixed-line (invoke-restart :reparse-line fixed-line))) MalformedLogEntry (fn [e {: line}] (io.stderr:write (.. "malformed log entry: " line "\n")) (invoke-restart :ignore nil))] ;; ---- 8< ---- ) Much like let, it accepts a vector of bindings - a condition object, and a function that will be called. Similarly to let it binds sequentially, but doesn’t override bindings, but creates a set of nested scope frames to store each binding. Then it executes the body in the context of this dynamic scope. Let’s look at the actual implementation and break it down: clojure (fn handler-bind [binding-vec ...] (let [target (gensym :target) scope (gensym :scope) binding-len (length binding-vec) setup '(do)] (assert-compile (= (% binding-len 2) 0) "expected even number of signal/handler bindings" binding-vec) (for [i binding-len 1 -2] (let [condition-object (. binding-vec (- i 1)) handler (. binding-vec i)] (assert-compile (or (sym? handler) (function-form? handler)) "handler must be a function" handler) (table.insert setup `(assert (not= nil ,condition-object) "condition object must not be nil")) (table.insert setup `(tset ,scope :handlers {:parent (. ,scope :handlers) :target ,target :handler-type :handler-bind :handler {,condition-object ,handler}})))) `(let [,target {} {:pack pack#} (require ,utils) {:pcall-handler-bind pcall-handler-bind#} (require ,condition-system) ,scope ,(current-scope) orig-handlers# (. ,scope :handlers)] ,setup (pcall-handler-bind# #(pack# (do ,...)) ,scope ,target orig-handlers#)))) This is the entirety of handler-bind macro, minus the docstring and some comments, and while the actual body that gets spliced into the program’s source code is not very big, the setup looks somewhat complicated. So let’s break it down. clojure (fn handler-bind [binding-vec ...] (let [target (gensym :target) scope (gensym :scope) binding-len (length binding-vec) setup '(do)] (assert-compile (= (% binding-len 2) 0) "expected even number of signal/handler bindings" binding-vec) ;; ---- 8< ---- )) First, we create some unique names that we will use both at compile-time, and then at runtime. You might remember the :target key in tables, that represented the dynamic scope. This key is holding this target symbol, which we define at run time, but we need a way of putting it into the dynamic scope at compile-time, that’s why it’s defined here. I’ll explain why it’s an important key soon. Next up is the scope, this is an instance of dynamic scope we’re going to construct at compile time. binding-len is just to avoid small pieces of code duplication, and the last one - setup is the code that will fill the missing bits of our dynamic scope at runtime. After let, we’re doing a check, that the binding-vec, a vector that holds condition objects and their handlers, has an even amount of values. We can’t properly construct the dynamic scope if a handler is missing, so it’s a compile-time error. Now for the main part - the for loop, which creates a set of instructions that will construct the dynamic scope: clojure ;; ---- 8< ---- (for [i binding-len 1 -2] (let [condition-object (. binding-vec (- i 1)) handler (. binding-vec i)] (assert-compile (or (sym? handler) (function-form? handler)) "handler must be a function" handler) (table.insert setup `(assert (not= nil ,condition-object) "condition object must not be nil")) (table.insert setup `(tset ,scope :handlers {:parent (. ,scope :handlers) :target ,target :handler-type :handler-bind :handler {,condition-object ,handler}})))) ;; ---- 8< ---- It does several things - first, it checks that the handler is either a symbol or a function form, i.e. a list that will evaluate a function definition. Next it inserts the runtime assertion, that verifies that the condition-object is not nil. And finally, the tset call is inserted, that will populate current scope’s :handlers table with a new table, that refers to the original table via the :parent key. You’ve might notice that the loop actually goes backward. This is done to preserve the correct order when constructing nested tables. As the dynamic scope is constructed as a set of nested tables, the order of construction has to be reversed, so the travel order gets preserved. But this code doesn’t actually construct the scope, it just describes how to construct it. The last portion of the macro is what gets spliced into the user’s code, so let’s look at it: clojure ;; ---- 8< ---- `(let [,target {} {:pack pack#} (require ,utils) {:pcall-handler-bind pcall-handler-bind#} (require ,condition-system) ,scope ,(current-scope) orig-handlers# (. ,scope :handlers)] ,setup (pcall-handler-bind# #(pack# (do ,...)) ,scope ,target orig-handlers#)) We’re finally setting some of those symbols, we’ve defined before! The ,target {} thing, is a way to make sure we get a unique object address that we can later compare, to determine if we’ve reached the correct destination when “unwinding”. Next, we import some helper functions, like pack# and pcall-handler-bind# - the first one is a portable table.pack implementation, and the last one is a runtime part of handler-bind. After that, we’re calling something like current-scope. It is a commonly used part of all macros, so I’ve put it into another macro. In short, it just returns the current thread’s dynamic scope or creates it if it doesn’t exist yet. And right after that, you can see that we’re caching original handlers right before ,setup. This ,setup will splice in all that setup we’ve been constructing at compile, that will populate the current scope with new tables. And then we’re calling pcall-handler-bind# function with all necessary info. So this code from the example: clojure (handler-bind [MalformedLogEntry (fn [e {: line}] (match (fix-line line) fixed-line (invoke-restart :reparse-line fixed-line))) MalformedLogEntry (fn [e {: line}] (io.stderr:write (.. "malformed log entry: " line "\n")) (invoke-restart :ignore nil))] (icollect [line (f:lines)] (parse-line line))) uses handler-bind macro we’ve just examined. Let’s look at what this code expands to: clojure (let [target_1_ {} {:pack pack_10_auto} (require "fennel-conditions.impl.utils") {:pcall-handler-bind pcall-handler-bind_11_auto} (require "fennel-conditions.impl.condition-system") scope_2_ (let [cs_3_auto (require "fennel-conditions.impl.condition-system") thread_4_auto (or (and coroutine coroutine.running (tostring (coroutine.running))) "main")] (match (. cs_3_auto "dynamic-scope" thread_4_auto) scope_5_auto scope_5_auto _ (let [scope_5_auto {:handlers {} :restarts {}}] (tset cs_3_auto "dynamic-scope" thread_4_auto scope_5_auto) scope_5_auto))) orig-handlers_12_auto (. scope_2_ "handlers")] (do (assert (not= nil MalformedLogEntry) "condition object must not be nil") (tset scope_2_ "handlers" {:handler {MalformedLogEntry (fn [e {:line line}] (io.stderr:write (.. "malformed log entry: " line "\n")) (invoke-restart "ignore" nil))} :handler-type "handler-bind" :parent (. scope_2_ "handlers") :target target_1_}) (assert (not= nil MalformedLogEntry) "condition object must not be nil") (tset scope_2_ "handlers" {:handler {MalformedLogEntry (fn [e {:line line}] (match (fix-line line) fixed-line (invoke-restart "reparse-line" fixed-line)))} :handler-type "handler-bind" :parent (. scope_2_ "handlers") :target target_1_})) (pcall-handler-bind_11_auto (hashfn (pack_10_auto (do (icollect [line (f:lines)] (parse-line line))))) scope_2_ target_1_ orig-handlers_12_auto)) Quite a lot of code, and there are even more things going on when we’re calling the pcall-handler-bind function! But you can see that it does exactly what I’ve described above - requires some API functions, creates a thread-local scope, sets it up, and wraps our code with the pack function, for later execution. Let’s now look at what pcall-handler-bind does. ### Implementing control transfer via protected calls The pcall-handler-bind function is defined in the library’s private API: clojure (fn pcall-handler-bind [f scope target orig-handlers] "Call `f` in given `scope` and pass result up to `target`. Restore dynamic scope handlers to `orig-handlers`." (let [(ok res) (pcall f)] (doto scope (tset :current-context nil) (tset :handlers orig-handlers)) (if ok (unpack res) (match res {:state :handled : target} (raise res.type res.condition-object) {:state :error :message msg} (if (or scope.handlers.parent scope.restarts.parent) (error res) (error msg 2)) _ (error res))))) Looks pretty simple, but this is because we delegate most of the work to other functions. It starts with calling f in a protected call and immediately removes current-context from the scope, and sets handlers to the original handlers, which we’ve stored back at the macro side. Then, if the function exited successfully, we just return the results by unpacking those (for keeping nils in multiple value returns). But when the function didn’t succeed, we expect a set of certain things as a return. For those unfamiliar with match - it’s a pattern-matching macro from Fennel standard library. It accepts an expression, and a pair of patterns and their respective bodies to run, if the result of an expression matches this pattern. The first pattern here is {:state :handled :target target}, meaning that the function f returned a table with a key :state holding a :handled value, and a non-nil :target key that is equal to the value of target, passed as function argument. The table returned in this case usually looks like this: clojure {:state :handled :target {} :type :error :condition-object #<condition MalformedLogEntry: 0x564caba2c6e0>} In this case, we raise the condition object stored in the result from calling f, with a correct type, meaning that the type is :error, we raise it as an error, or if it’s a :warning, as a warning and so on. We’ll look at raise in a moment, but for now, all you need to know is that it will throw the condition object further up. If the pattern contained :state :error we check if we have more handlers up the dynamic scope, and if we do, we throw the result as is, so it could be handled. If not, that means that we’re already at a top level, so we convert this to Lua error with a message. In all other cases this is an error that we want to throw up, as we can’t do anything about it because it is either not a conditional situation or target doesn’t match the expected one. So the basic idea here is that we’re ensuring that the function will either succeed or fail as we expect, and if it fails in an unexpected way, we just rethrow the error. Lua doesn’t have a way to rethrow the stack traces, but it’s a small price to pay here. Let’s look at raise now: clojure (fn raise [condition-type condition-object] "Raises `condition-object' as a condition of `condition-type'. `condition-object' must not be `nil'." (assert (not= nil condition-object) "condition must not be nil") (let [thread (current-thread) thread-scope (do (when (not (. dynamic-scope thread)) (tset dynamic-scope thread {:handlers {} :restarts {}})) (. dynamic-scope thread))] (when (= thread-scope.handlers.handler-type :handler-bind) (match thread-scope.current-context scope (let [target scope.target] (var scope scope.parent) (while (and scope (= target scope.target)) (set scope scope.parent)) (tset thread-scope :handlers scope)))) (match condition-type :condition (raise-condition condition-object) :warning (raise-warning condition-object) :error (raise-error condition-object)))) This function does something interesting with current-context. As you saw in pcall-handler-bind, we’ve set the current-context to nil, but raise actually check it. Why? This is because the condition can be raided inside the handler. In such a case, we need to unwind the stack to the point where we were in the handler. So each invocation of the handle function sets the current-context field, and this field is cleared when we exit the handler. This stack unwinding needs to be done in order to avoid infinite recursion, which will result in a stack overflow error. After that is done, we branch on the condition type and raise the condition accordingly. For the purposes of this article let’s look at conditions of error type only, by looking at the raise-error function: clojure (fn raise-error [condition-object] (match (raise-condition condition-object :error) nil (if _G.condition-system-use-debugger? (invoke-debugger condition-object) (error (compose-error-message condition-object) 2)))) In case the condition is not handled it’s raised as a Lua error unless the debugger feature was enabled. The condition itself is raised with the raise-condition function: clojure (fn raise-condition [condition-object type*] (match (handle condition-object (or type* :condition)) (where (or {:state :handled &as res} {:state :restarted &as res})) (error res 2) _ nil)) Here’s the handle innovation, mentioned above. The result can be either a table with :handled or :restarted states, each of which tells the condition system what to do next. Even though the condition is handled this result gets thrown up as an error because we actually need to tell the condition system that the condition was handled, and differentiate it from a successful return from the function. So let’s look at how conditions are handled. ### Handling conditions The handle function looks like this: clojure (fn handle [condition-object type* ?scope] "Handle the `condition-object' of `type*' and optional `?scope`. Finds the `condition-object' handler in the dynamic scope. If found, calls the handler, and returns a table with `:state` set to `:handled`, and `:handler` bound to an anonymous function that calls the restart." (let [thread (current-thread) thread-scope (. dynamic-scope thread)] (match (find-handler condition-object type* (or ?scope thread-scope.handlers)) {: handler : scope} (do (tset thread-scope :current-context scope) (match scope.handler-type :handler-case {:state :handled :handler #(handler condition-object (unpack (get-data condition-object))) :target scope.target :condition-object condition-object :type type*} :handler-bind (do (handler condition-object (unpack (get-data condition-object))) (handle condition-object type* scope.parent)) _ {:state :error :message (.. "wrong handler-type: " (view _)) :condition condition-object})) _ {:state :error :message (.. "no handler bound for condition: " (get-name condition-object)) :condition condition-object}))) Let’s dive into it. After getting the current thread and the scope, we call find-handler. For now, it’s enough to know, that it takes a condition object, and walks the dynamic scope comparing this object to handlers. Next up, if the handler were found, we set the current-context to the handler’s scope. This will ensure that we know where we last were before actually executing a handler, so we could unwind the stack to this point in case an error happens inside the handler. This was a very painful thing to figure out when I worked on the implementation. After that, we branch on the handler type. The handler-case macro allows handling without calling invoke-restart, so we just return a table with :state set to :handled, and a handler that actually does the handling. The handler-bind is different - it requires handling with invoke-restart, which, as you might remember throws an error to transfer control flow. So if the handler function exited normally, we go to the next handle invocation, which will propagate the condition object to another handler up the stack. This ensures that we can decline to handle a condition. If no handler is found the error is returned. A similar pattern occurs for other condition types, with the difference being that some may not return an actual error when unhandled. So how handlers are found? The find-handler function, which accepts a condition object and a handler type, will walk the scope comparing the given condition object to bound handlers. However, condition objects actually support inheritance, so this gets a bit more complicated. We need to look for a handler in the current scope, and if no handler is found, we need to get the parent from the condition object and search its handler in the same scope. If no handler and no parent handler are to be found, we go to the next scope frame. Additionally, conditions may not be of the condition-object type, but be an arbitrary Lua value: clojure (handler-case (signal (math.random 5)) (0 [] :zero) (1 [] :one) (2 [] :two) (Condition [] :else)) ;; an unnecessarily complicated equivalent of (match (math.random 5) 0 :zero 1 :one 2 :two _ :else) Notice, that while I’m handling conditions as numbers, the catch-all condition object is specified as Condition. Let’s talk more about condition inheritance. ### Condition inheritance As was already mentioned, condition objects can have parent condition objects. A parent can be explicitly set by using the :parent key when defining a condition: clojure (define-condition MathError) (define-condition DivideByZero :parent MathError) (define-condition NegSquareRoot :parent MathError) (fn sqrt [x] (if (< x 0) (error (make-condition NegSquareRoot x)) (math.sqrt x))) (fn div [a b] (if (= b 0) (error (make-condition DivideByZero)) (/ a b))) (fn main [] (handler-case (sqrt -1) (MathError [_ x] (io.stderr:write "Can't take square root of " x "\n"))) (handler-case (div 2 0) (MathError [_ x] (io.stderr:write "Can't divide by zero\n")))) Here’s an example of handling different errors through their common parent. You can create hierarchical errors, which then can be handled with more granularity, or as a whole class of errors. But there are a bit more rules regarding inheritance. All conditions raised with error function will by default inherit from the Error condition. All conditions raised with warn will inherit from a Warning condition. Both Warning and Error are inherited from the Condition condition. This also applies to any arbitrary data you can throw with error, warn, or signal: clojure (fn raise-random-condition [] (match (math.random 2) 0 (error "an error") 1 (signal "a signal") 2 (warn "a warning"))) (handler-case (raise-random-condition) (Error [msg] (print "caught an error" msg)) (Warning [msg] (print "caught a warning" msg)) (Condition [msg] (print "caught a condition" msg))) Here all conditions are just Lua strings, but we still have some hierarchy. Note that the order of handlers here is very important - placing the (Condition [msg] (print "caught a condition" msg)) as the first handler will prevent Error and Warning handlers from ever running. It is very similar to match in that regard - placing _ as the first pattern will make all other patterns unreachable. Another thing to point out is that Lua errors also inherit from Error condition: clojure (handler-case (/ 1 nil) (Error [] 0)) This is done for seamless integration with libraries that don’t use the condition system, or if your code did something wrong by mistake. However, the integration is not as seamless as it could be. The point where it get’s not as good as it could be is when interfacing with the interactive debugger, provided by the library. ### The debugger Speaking of which, the debugger is not as featureful as in Common Lisp, but still provides a way to recover from an error, when the current scope allows for it. Consider this: fennel (restart-case (cs.error 10) (:use-value [x] x)) Executing this code will raise a condition, which is not handled, so it gets propagated to a Lua error: text runtime error: condition 10 was raised However, if we set _G.condition-system-use-debugger? to true, we’ll get this instead: text Debugger was invoked on unhandled condition: 10 restarts (invokable by number or by name): 1: [use-value] use-value 2: [throw ] Throw condition as a Lua error debugger>> Now every unhandled condition will invoke the debugger, which will provide you with available recovery options. In the case of this example, there are two restarts bound - use-value, bound by the restart-case macro, and the other one is an automatically bound throw restart, that will exit the debugger. By typing 1 or use-value in the prompt, we’ll enter the next prompt, that asks for a value to use for a selected restart: text debugger>> 1 Provide inputs for use-value (args: [x]) (^D to cancel) debugger:use-value>> 5 5 Thus making the result of the expression to be 5, accordingly to the restart’s body. This debugger has no runtime penalty unless the condition was not handled. So only When an unhandled condition pops to the top of the dynamic scope stack, the debugger begins its work. Debugger again traverses the scope, gathering all available restarts, and forms this interactive prompt, with each restart in order of their definition: clojure (restart-case (restart-case (restart-case (/ 1 nil) (:use-value [x] x) (:use-value [x] "use-value (impostor)" (- x))) (:do-something [] (+ 1 2 3))) (:do-something-else [] (- 3 2 1))) Note that the use-value restart was specified twice. I’ve mentioned that this is possible, but it is meaningless unless we use a debugger. Here’s what will be prompted to the user, if this code is executed: text Debugger was invoked on unhandled condition: attempt to perform arithmetic on a nil value" restarts (invokable by number or by name): 1: [use-value ] use-value 2: use-value (impostor) 3: [do-something ] do-something 4: [do-something-else] do-something-else 5: [throw ] Throw condition as a Lua error debugger>> Note that the first restart is indeed named use-value, but the second one has no name, only its docstring is displayed. This restart can be manually called by typing 2 in the prompt - it will ask for the argument similarly to use-value restart, but will negate it. Other restarts are self-explanatory. ## Ending thoughts This should provide some information about this library’s internals. Seeing how many pcall calls it does to transfer control flow around, and how it manages the dynamic stack, can make you think that it has not a very good performance. And in fact, I would agree, the performance could be better. But it’s not that bad. For example, here’s a 10000000 invocations of Lua’s error function inside a pcall: fennel (macro time [expr times] `(let [c# os.clock s# (c#) r# ,expr e# (c#)] (print (.. "Average time elapsed " (/ (* 1000 (- e# s#)) ,(or times 1)) " ms")) r#)) (time (for [i 1 10000000] (pcall error 10)) 10000000) ;; Average time elapsed 0.0002025262 ms And here’s an equivalent using condition system’s error and ignore-errors macro: fennel (local {: error} (require :condition-system)) (time (for [i 1 10000000] (ignore-errors (error 10))) 10000000) ;; Average time elapsed 0.0051841138 ms There’s one order of magnitude difference in performance here, meaning that ignore-errors is ~10 times slower than pure pcall. But the benefits, that the condition system provides, out-weight this cost, in my opinion. Also, a note on why I didn’t port the rest of Common Lisp’s condition system. While this library is not a full port, I think it has all the necessary parts in the context of the Fennel language. What I mean by that is, that since Fennel, for example, deliberately chose to avoid Lua’s return special, I don’t think a condition system should provide block and return, or go specials. I don’t have strong reasoning behind it, but I feel that the error handling part in Lua is what’s needed to be improved, thus this library is only aimed at fixing this part of the runtime. --- 1. In Common Lisp there are more ways to recover from an error from the handler bound with handler-bind, but fennel-conditions library doesn’t implement all those ways. ↩︎ 2. Tables, tables, tables, tables, tables… tables, and tables - that’s Lua at its best. ↩︎ Comment via email