The hardest program I've ever written (2015)
journal.stuffwithstuff.com
journal.stuffwithstuff.com
I think I'd still prefer to see a formatter attempt to preserve any formatting which is already 'good enough' to pass as an output threshold. Code isn't just a recipe for a computer to do something, it's a language for explaining to other programmers what that thing is and what's important to the structure of accomplishing it. The choice of where to place a break can matter for cognition and can be almost as important as the printed characters for organizing thoughts.
I would not be surprised if a lisp formatter following the same philosophy as the dart formatter would be similarly complex, since the complexity comes from optimizing line length, not parsing.
https://jyp.github.io/pdf/Prettiest.pdf
It's based on introducing choice between vertical and horizontal stacking. It avoids combinatorial explosion by pruning away strictly suboptimal choices. With just a few extra rules for Lisp forms, the results are quite good.
I don't handle comments though, since I only need it for pretty printing values so far.
That feels like Prettier sometimes. I think it leaves some objects on multiple lines or one line depending on how you leave them. But I’m not too sure.
I've been thinking lately of a formatter that would resolve this tension. It would be a local, individual formatter, complementary to the global, opinionated, "Prettier" formatter.
When the source code was committed to remote for review, the opinionated formatter would do its thing, making sure the code was formatted properly according to whatever the team agreed. But locally, the code would remain however the developer liked it.
i think maybe your only hope is to somehow “version the AST”, and let formatting be a style sheet or something. But i’m 1. talking out my butt and 2. sure i’m missing something.
That's a good expression of the idea. If merge conflicts happen, we're doing it all wrong. Version control shouldn't even really be aware of this.
One problem though is all of the non-code elements, aka comments and how they intentionally align up with the code (mixed with spaces and tabs), so the ideal format is the one the original programmer wrote it in, and the second programmer using a different stylesheet can't edit comments and have them end up pretty for the first programmer. (The solution, obviously, is to not comment any code >_< )
But isn't this the nearest argument why we should start publishing negative results papers?
The most important thing that I have ever learned was a announcement made by Google about how their industry consortium seeking to trade print magazine advertising inventory, failed with reference to the nature of this failure. No matter how lacking in detail this notice was, no matter that I was sent down a seven years solitary and very lonely path of mercifully ultimate discovery, on the heels of my startups exit collapse due to my cofounder tragically dying but we'd have been sunk by the problems indirectly revealed in that Advertising Age news item. However painful, and I'm talking about therapy for years after I emerged from reclusion, and real health issues giving up running and my diet turning to junk energy hits...I have learned more from being told "no can do Z because of y" and I seriously think that we /seriously have to start already getting over the reasons why we don't talk about our failures/. I mean holy smoke wouldn't we ever run out of good conversation ever again if we could have a good old banter and brew over our cockups? I'm thinking that this is how women are so much more successful in reproduction if they actually want to. What could we do if we tried?
E.g.
(+ 1 2) ; Add two numbers.
(comment (+ 1 2) "Add two numbers")
Another would be
(+ ; Adding
1 ; one
2 ; and two.
)
((comment + "add")
(comment 1 "one")
(comment 2 "two"))
In my opinion, different people have different formatting preferences and forcing someone to use the "wrong" one will lead to slower reading speed and the risk of overlooking errors.
That's why I believe we should treat it like font or color choices. The IDE should display source code in the viewer's preferred style, so that each person sees what they expect. And then the actual source code formatting becomes irrelevant.
Go already goes a good step into this direction by making a language AST tree part of their core libraries. And opening Go source code in JetBrain's GoLand will show you additional annotations, spacing, etc. based on the parsed source code tree (and not based on the source code's text).
Everyone can just set up a pre-commit hook to automatically format the code back into the official style, then everyone is free to put the codebase back into their preferred style.
Otherwise, if developers reformatted code to their preferred style regularly, it would create massive diffs.
In this explanation about Go:
https://golangdocs.com/golang-ast-package
the text following "and we get a nice structure" is what the source code management tools would be working on. It's an abstract representation of the source code's meaning, but not tied to how things are formatted or indented.
The method I described does exactly that. Most modern formatting tools parse code into an AST then rewrite it based formatting rules. So as long as you format both files to the same rules before diffing, only AST changes will show up.
I genuinely feel that focusing a lot on the detriment of someone else's or an established code style is a sign of a lack of team work.
Not if everybody using the language is strongly forced (as with Go), since then those with "different formatting preferences" will eventually (and soon) just get used to the enforced style.
[1] Incidentally the only people I know with carpel tunnel are folks who are entrenched in command line text editors. Maybe there’s some benefit to moving your arms around? Similarly maybe there’s benefit in reading code in a different format, you might actually read it slower and hence comprehend it better.
You got that right -- check out this epic GitHub thread on optional semi-colons [1], and the author's own comment on the subject in an HN thread [2]
The language is fairly complex syntactically and that definitely adds some cost to formatting.
But I think much of the complexity comes from two things:
1. A lot of idiomatic Dart uses function literals in a block-like way, as in:
test(() {
expect(1 + 2, 3);
});
someFunction(
() {
expect(1 + 2, 3);
});
It's not as good as a human (mainly because it doesn't have semantic context) but it had to be pretty close or people wouldn't have tried it.
Now that it's well established, I think it would probably be possible to simplify how it formats while still making users happy. Possibly happier because the results would be a little easier to predict.
> Code isn't just a recipe for a computer to do something, it's a language for explaining to other programmers what that thing is and what's important to the structure of accomplishing it.
An automated formatter will never be as good as carefully crafted artisanal formatting. In particular, automated formatters don't know what stuff means. A good human might choose to line break a function call like so:
setColor(red: 123, green: 54, blue: 26,
alpha: 45);
But the value proposition of automated formatting is not just "how nice is the resulting code to read". You have to look at the total value proposition of completely yielding formatting to a tool versus allowing human control over it. When it's completely automated:
1. You can run it on generated code that contains absolutely no whitespace and still get nice output.
2. Humans can do large-scale refactorings, format, and get output that is consistent with the existing state of the codebase without having to understand any local style preferences.
3. Humans never have to spend time deciding how to format. Further, they don't even have to spend time deciding if they should format.
4. When reading a random codebase, it is likely to be formatted in a style you are used to even if you have zero communication with that team. This is particularly important in open source.
5. The code looks familiar to you wherever you encounter it: IDEs, plain text editors, code review tools, blog posts, StackOverflow answers. As opposed to letting everyone pick their own style and relying on users to apply their preferred style locally, it's just always in a familiar style.
6. Like any automation, the tool doesn't make mistakes. Even very careful humans hand-formatting make more mistakes than they realize. (I know because I've looked at their code). Those mistakes can be distracting for readers.
7. It gets people out of the mindset of being nitpicky about style. It encourages them to stay focused on the structure and naming of their code, which is what really matters.
8. It eliminates style arguments in code reviews. Those take time and, worse, cause disharmony, for next to no benefit.
I think it's very worth excepting some small loss of overall formatting quality to get those in return.
I realized there was a tradeoff I never considered — the amount of time it would cost me to manually format only 10% better than the black formatter is not something I am willing to invest.
And this is me talking about ideal situations where I calmy try to write beautiful code, when you consider that often you are not in the mood to spend a ton of time on formatting, because it distracts you from focusing onto the actual data flow, logic and so on a formatter as good as this is just beautiful.
This is not something I would've said before I tried the thing btw.
The Hardest Program I've Ever Written – How a code formatter works (2015) - https://news.ycombinator.com/item?id=22706242 - March 2020 (125 comments)
The Hardest Program I've Ever Written (2015) - https://news.ycombinator.com/item?id=17271963 - June 2018 (76 comments)
The Hardest Program I've Ever Written (2015) - https://news.ycombinator.com/item?id=15063193 - Aug 2017 (48 comments)
The Hardest Program I've Ever Written - https://news.ycombinator.com/item?id=10195091 - Sept 2015 (76 comments)
I'm still hoping that Rust will eventually get such a formatter. Unfortunately the people responsible for rustfmt seem to have a strong preference for the "ignore line-breaks the user inserted" approach.
So far my magic number is three columns: three code files, or one/two code files with a terminal and/or browser window thrown in the mix. Or any of these columns can be split into two vertically stacked boxes for a total of six things.
I’m also a (nonultrawide) single screen coder (after many forays into the multi screen world) which has undoubtedly guided my preference.
Any other strategy is like trying to design a chair without thinking about butts. You may come up with some sort of elegant Bauhaus mathematically perfect work of art, but no one will want to sit in it.
It’s like, I could scan the code easier and understand it better without you doing that, thank you!
A similar thing is match statements — some arms using braces vs statements.
There’s also a bunch of heuristics in rustfmt that are complex to the point that I literally couldn’t format the code the way it does without building some sort of decision tree annotated with uneven column limits (e.g. “70% of the column limit”). If I can’t and therefore wouldn’t format the code the way the formatter does, there’s an issue.
Formatters are for consistency, I think, and sometimes they work against that.
TeX famously does line breaking in a perhaps decent way - but it and text shaping become more of an art than a definite list of rules to follow.
That's…how he does it.
With the code formatter, to format the part after a certain character, you need to keep track of the indentation depth of all the expressions that have not yet terminated at this point — because you presumably want parallel expressions to be formatted with the same indentation depth, for closing parentheses to match their corresponding opening parentheses, etc.
For example, in this example:
experimental = document.querySelectorAll('link').any((link) =>
link.attributes['rel'] == 'import' &&
link.attributes['href'] == POLYMER_EXPERIMENTAL_HTML);
experimental =
document.querySelectorAll('link').any(
(link) =>
link.attributes['rel'] == 'import' &&
link.attributes['href'] == POLYMER_EXPERIMENTAL_HTML);
This is what the author alludes to in the post:
> A line break changes the indentation of the remainder of the statement, which in turn affects which other line breaks are needed. Sorry, Knuth. No dynamic programming this time. […] For most of the time, the formatter did use dynamic programming and memoization. […] It worked fairly well, but was a nightmare to debug.
> It was highly recursive, and ensuring that the keys to the memoization table were precise enough to not cause bugs but not so precise that the cache lookups always fail was a very delicate balancing act. Over time, the amount of data needed to uniquely identify the state of a subproblem grew, including things like the entire expression nesting stack at a point in the line, and the memoization table performed worse and worse.
In TeX, paragraphs have each line of the same width (simple case) or can have a \parshape (in general), but these are "global" constraints that don't depend on what breaks you choose.
If I recall correctly Knuth uses Dijkstra on a graph with nodes of 'line break at position x' and I don't see why you couldn't use a graph consisting of 'line break at position x indentation y' or something similar.
One of the hardest parts of programming is understanding what's happening from reading code. And if you abstract too much "the traditional way" then it is just even harder to understand.
And if you think that formatting is a problem to understand the code, let's get real: this is the smallest of the problems. There are tons of other things that make code complicated to read, like variable and function names, the particular style of your code, how you split it into classes and files, the algorithm you're using, and so many other, more important things. I can guarantee you that if a piece of code is well written, you can understand it independent of where you put braces or the number of spaces you're using.
Auto-formatting (esp. when used as a pre-commit hook) means that changes people make to the style are ignored/reverted (and/or, that places where people introduce a different style in new code, are auto-formatted back into the existing style immediately, rather than that needing to be an additional commit later on.) Thus, no spurious diff lines from formatting. Thus, not having to wade through a bunch of "noise" diff-lines, to get to the "signal" of semantic changes at code-review time.
Also, having auto-formatting on both your main branch + development branches, makes merge/rebase conflicts less likely to happen. (Which basically boils down to "fewer noise diff-lines" again.)
In other words: auto-formatting makes code more machine-legible to syntax-blind parsers; which in turn allows tooling like diff(1) to be more helpful.
(Yes, we could just have language-syntax-aware semantic-level diff/merge/etc. tools. Not sure why nobody ever made these. I bet this is one of those things where Lisp users have had it for ages but using their own parallel world of abstractions that doesn't exist in C/POSIX.)
If you add a line between two existing lines, and then insert after it a new blank line to serve as a sort of "paragraph marker", is that a "pure formatting" change?
If you add a constant in a group of constants, whose name is longer than the existing ones, do you pad the spacing of the values of those constants so they line up with one-another?
For that matter, if you fully-qualify a previously-unqualified and potentially-ambiguous identifier, is that a "pure formatting" change? Some auto-formatter tools do this, after all.
These are things that people may or may not do in code-bases, that "fly under the radar" of even the most stringent of human code-reviewers, because they're so irrelevant to understanding the code. They're "fluff." But because of this, how people introduce that fluff is essentially random, and so the cause of a lot of diff noise. These are the things that auto-formatters can "lock down" to only happen a certain way.
But I think you're missing the forest for the trees, as I mostly wasn't talking about pure formatting changes. What I'm talking about is more like:
You add a formal parameter to a function. Before, the function's clause head was less than 80 characters. Now it's more than 80 characters. Do you break the formal parameter list onto the next line? If so, how far do you indent it? Do you split the formal-parameter list up so that each parameter is now on its own line? Etc.
Done by humans with no strict standard, these sort of one-off judgements made arbitrarily will add up to "syntax rot" — not something you observe with your eyes, but a sort of "potential energy" of un-made formatting changes, that means that any given semantic change by a sufficiently-motivated human might become the impetus for a manual reformatting during that semantic change, such that that reformatting will happen at a random time, inflating a patch where affecting that additional code wasn't strictly necessary. (If you ask the programmer why they did it, they'll say they needed to "clean up the code they were working on" so that they could understand it well enough to apply the fix.) Which is horrible for both code review and merge predictability.
On the other hand, an auto-formatting tool will apply that transformation exactly when it becomes necessary; and will pick some way of formatting the additional lines and stick to it. There's no "potential energy" there. At all times, the codebase is "at rest", with no chance of anyone introducing "arbitrary" (but actually left-over) formatting changes.
Human formatting is like a sequence of DML statements in an RDBMS. Auto-formatting is like a sequence of operations against a CRDT. Given a bunch of changes run in a random order, the output of human formatters will be arbitrary, while the output of auto-formatting will be deterministic. Which is what you want, if you're doing complex things involving e.g. long-maintained stable branches for 1.x that cherry-pick changes from 2.x.
Counterpoint: When using a formatter, I stop worrying about formatting. It's a job for a computer, done by a computer. Humans are bad at consistency and discipline, computers are great at it. I want to concentrate on the things that matter, and formatting isn't one of those.
Especially in larger teams, consistent formatting is just nice. No conflicting styles in the same file, and more meaningful diffs.
Moreover, using an automatic formatter will not fix it, because, guess what, there is no universal code formatter. All of them have different results and a long list of parameters. Determining the best way to use one will create more work for you as you manage your team, and will inevitably add a new step to your already complex building process. Just stop worrying and use that time in more productive ways.
I don’t know. I write JavaScript at $DAY_JOB and setting up Prettier on our repos took all of ~30 minutes, with an additional ~15 to determine which options to use. (There aren’t many because Prettier is fairly opinionated.) I have seen far more time wasted quibbling about code styling in code reviews.
- Not having to format my code manually at all, just letting the formatter do it for me, is a significant productivity win. I write code as fast as I can, with the minimum number of key strokes, in a way that would normally be super ugly, and it comes out the same. I have my editor setup to auto-format the current file on save, so it’s just type a bit of code with zero formatting, cmd+s, then it’s instantly perfectly formatted
- For a codebase with 10s or 100s of devs working on it, uniform formatting does significantly help readability. Sure I can still read it if there’s dozens of different formatting styles going on, but I can read it faster if the formatting is always consistent
- Re: the above, yes you can keep consistent formatting without a code formatter, with a style guide that everyone learns, and that you enforce in code reviews. But that’s a waste of time both for on-boarding new devs, and a basically neverending waste of time during code reviews. Also a waste of time writing and maintaining the style guide itself
The first point helps me write faster, the second helps me read faster, and the third keeps code reviews and the like quicker.
Code formatters are such a clear, easy win, especially with large teams, that it’s hard for me to understand why anyone would opt out of them. It’s not a MASSIVE win, but IMO it clearly makes for a more productive development environment, and they’re generally dead simple to setup.
Furthermore, not worrying at all about indentation, spacing, brace placement, semicolons or not, etc. lets me write code faster, not just read it faster. Type it out with zero effort expended on formatting, save, editor auto-formats.
It’s not that any of this saves crazy amounts of time, but it does make all of code writing, code reading and code reviews slightly faster. When it’s so easy to setup, why not do it?
The only argument I can see against auto-formatters is that people like to put their own artistic touch on the code they write. I get that, but it wastes time, especially when everyone starts doing things in their own style.
I’ve been working professionally as a dev for 9 years, also on teams that use auto-formatters, and teams that don’t. I think they’re a small but clear productivity booster. The only time I’d consider not using one is in languages where the formatter itself is super slow. But fast ones, where you can setup nearly instantaneous “format on save” (go fmt, prettier, etc.), no brainer.
Auto-formatting tools in editors exist, and they're very common, and they're not always configured properly, so people change formatting on accident. Sometimes formatting changes can cause code reviews to take more time than necessary. Having tabs in code can cause actual problems, for example, since tabs aren't the same size everywhere.
This is not just a code understanding problem, and shouldn't be written off as trivial, IMO.
I just want to write the tokens without ever thinking about where they go on the page, periodically save and let my formatter deal with it.
Take a look at how complex the code is for something like vim-ruby to get a feel for what I’m talking about
I have a working regex (passed by many regex online testers) but in bash yet, NO!
To graderjs of HN, the Author of Dart code for matter, you got mad respect from me.
And the creator of prettier/plugin-ruby worked on a pure-Ruby implementation https://github.com/ruby-syntax-tree/syntax_tree
The best code formatter is you.
But I never worked in a big professional team before, where git's features shine.
And git != code formatter.
The biggest advantages (to me at least) are that they almost entirely eliminate formatting from code review, and the consistent style makes it easy to read and edit code written by different people.
Git's features shine just as brightly, IMO, if you work for yourself.
Granted a good number of them are often not very relevant to a lone developer, but if you work for or by yourself on any projects over the long term, you should build git (or some other distributed source control, but probably git) into the way you work.
Not to excess, by any means. I use a very small subset of git's features and in practice many of my simpler projects don't even use branches, because it's not in the nature of the changes I am making to those projects or the time management I do.
But for example you should consider git an essential step between dev and live -- using it to deploy -- and you should look at how you could use it to facilitate staging and testing.
Combined with a changelog and relentless use of comments and notes, git helps "structured forgetting", which as a freelancer is pretty crucial; sometimes you work frantically on a thing for a month, get paid and then it comes back to you years later.
> And git != code formatter.
No, obviously, but the needs of the former are supported by the benefits of the latter.
That said, I use one for golang but not, in general, for PHP. I should find a code-formatter I can bend to my will for PHP, but after 17 years of increasingly complex lone PHP development, I know what I need from my own formatting requirements in order to manage projects.
Granted when I am developing for myself, most/all commits are just going to master but even so.
Apparently some actual code changes are in there but we’ve never found them.
One of the best trends recently.
I've never understood this point. Programmers will always find something they can nitpick about. Ultimately you want a culture which focuses on the critical parts: Correctness of code, good test coverage, big-picture architecture which has long-term impact. Bringing an auto-formatter into the picture may reduce some senseless nitpicking, but you haven't actually done anything to solve the real culture problem. If your team was getting blocked because people were arguing about formatting you have bigger problems that won't be magically solved by adding an auto-formatter.
To give some examples:
A while back I worked with one person on a project where we had both Prettier and super strict ESLint, and I would still get PRs rejected because they wanted the code to be slightly refactored in a way which was entirely subjective and had no impact of the correctness (e.g. "flip this negation") .
And right now I'm working on a team where we explicitly tag some PR comments with "nitpick". This will not block the PR from getting merged, but instead it's a way of saying "I prefer it this way, but it's not that important in the bigger scheme of things". This is also a signal that it's not something that we want to start a bigger discussion around.
(We use auto-formatters and linters as they are very useful.)
* Prevent arguments on which formatting convention to adopt,
* Save a peer reviewer from shallow comments if formatting conventions are broken,
* Prevent fill-in white space commit from filling in the git history, and
* Decrease the risk of senior developers imposing weird styles on the code base.
Formatters might help you write better code by freeing you from worrying about one aspect of coding, but—much more importantly—they help create and maintain a better culture around your code.
1. Make programming more boring
2. Prevents me organizing related thoughts on single lines
3. Prevents me doing meaningful and more readable indentation in specific contexts (such as align on equal sign etc.)
I don't like them, nor I like any of the style checkers. The equivalent is aS if somebody wrote a book, and you give it to GPT-3 to make it more readable for entire world. Fascinating BS.
This is a terrible argument for anything to do with programming/code. It is pure opinion and preference and therefore is not falsifiable.
If you’re on a team of one, go wild. But if you’re on an actual team trying to get things done, please don’t bore the other people to death with the “interminably soul crushing debates over code formatting” as the article aptly puts it.
The team wants to see exciting results, not “exciting” code. Code is a means, not an end.
And especially if you’re on the job, you are not being compensated with excitement, but money. Go seek excitement in your personal time.
Formatters are not comparable to GPT because the code semantics have not changed, only the form. You just don’t want to retrain yourself to read code that isn’t written exactly to your liking. That’s laziness.
It can greatly affect readability, lead to/prevent unnecessary merge conflicts, and aid in/stand in the way of using nice VCS features (blame, revert, bisect, cherry-pick, …).
Many automatic code formatters ignore and do not optimize for these metrics.
Git history trumps code style IMO, I would not e.g. add a formatter to a legacy codebase and then reformat everything, or do so after changing a rule. Only diffs get formatted.
1. They didn't have X, so obviously we don't need X now.
2. There were the ones who created X, so clearly they felt the absence of X was a problem to be solved.
I'm inclined to believe 2 comes into play more often than 1. The present was created by those living in the past.
Let's not think of the spin-off: FaaS.
My experience is that people prefer formatting that is:
1. Unsurprising.
2. Nice looking.
3. Simple.
In roughly that order. Using ML might increase 2 but at the expense of 1 and certainly 3.