Hacker News Reader

C/C++ macros are particularly horrible as they are essentially text replacement on the source code. Other languages (such as Rust) have macro systems which work with ASTs and can used in isolation without interfering with other stuff.

For example there's a library in Rust which provides a `json` macro with syntax like so:

    json!({
        "foo": "bar",
        "qux": baz
    });

where `baz` can be a Rust string or integer. Note: this will typecheck, so if you provide something weird as `baz` then it won't compile, but otherwise the syntax is identical to standard JSON syantax.

Whats the canonical way to deduce the types in Rust from JSON? JSON Schemas?

This particular crate, serde_json, lets you create any type that implements Serde's Deserialize trait when you're parsing.

... and it also provides a dynamic interface too, if you need to work that way. You probably don’t...

That still doesn't address the issue of macro syntax being foreign/not easy to refactor. See the nom library for an example.

Rust procedural macros are just regular rust that runs at compile time. You get vector of tokens in, and return a new vector of tokens out.

And how does that address this issue?

Procedural macros and macros are different things; nom is the “normal” macros (“macros by example”).

The initial procedural macro release is in 6.5 weeks. Nom has worked on stable for years.

I know procedural macros are different in how they work and more flexible, but the problem remains the same: you can't really look at any macro definition and go i know how to use this.

A vector of tokens is not the syntax a programmer can parse without the grammar definition that the macro implements. It's still foreign unless it is well-written to match existing language constructs.

Sure; to be clear, I'm trying to point out what I think your parent is getting at. I don't necessarily agree.

Oh, I thought you were talking about the issues as a macro writer. Now that I reread it, I think you're talking about as a macro consumer. Is that right?

That's a good point. The nom library definitely does exhibit a lot of the problems described above. There's definitely a ton of macros in the rust ecosystem that don't though.

‘Most programmers can’t be trusted with macros’ is a sentiment I read a lot, but at one point in time people might have said the same about functions, and arguably they could say the same about pointers or recursion today.

I think that being able to think symbolically about one’s code is a vital skill for a programmer (and, more generally, that the capability of thinking abstractly is a vital skill for a learnéd human being). Thus, if someone lacks that skill, perhaps he should probably not be a programmer? That may sound revolutionary, or elitist, but I think most of us would agree that someone who can’t effectively reason about a depth-first tree traversal will have trouble as a professional software developer; mayn’t that also be the same of someone who can’t effectively reason about computational structure?

Your comment re. macros & fragmentation makes me wonder if you’ve ever used Lisp’s macros: I think that they actually mitigate against fragmentation, as e.g. CLOS started out as macros atop Lisp and ended up being standardised as part of it. Lisp’s packages help manage namespace collisions, and thus prevent fragmentation by making differences manageable.

And Lisp tooling typically handles macros very well indeed.

the unique problem with macros (at least in the C/C++ world) is that they can be used to invent new syntax. Now when you join the company, not only do you have to learn the code, but you also have to learn the language extensions at the same time. If there's a lot of them, you're effectively learning a whole new language with likely minimal intentional design, with all the potential problems that a new language comes with (unintentional/hard-to-understand side effects, performance issues in the output, poor domain fit, etc).

For all the flack Java gets, it got something right on this topic imo. Take the lowest common denominator of language features (for a time), and have the solution to almost everything be code written with those very, very few constructs. Yeah, it ends up verbose as hell, but even though I haven't done Java since the early 2000s, I can look at our backend code (which is all in modern Java), and short of a few new things like annotations and lambdas, I can read it just fine.

The problem space is very, VERY well understood. The patterns have been bikeshedded to death almost 20 years ago. It's super boring, there's not many interesting left, and it's awesome because of it. Of course, software engineers hate this in general because they get way too much enjoyment out of figuring out how to save 2 lines of code for their own good ;)

There are two things about Java syntax that are my sore points.

Not having had the guts to make override a keyword, forcing the dumb @Override annotation. It is a matter of taste I know. But they could have made use of contextual keywords, thus keeping backwards compatibility.

I really would like to have a typename kind of statement instead of making empty classes to simply typing with generic code.

I think the point being made is about incurring unwanted technical debt, rather than about the skills of programmers. A macro enthusiast can leave behind a codebase that requires substantial expertise or siloed knowledge to work with.

It's not that you can't trust people to make good decisions or invest in understanding what they're doing. It's that in real life people take shortcuts. Time constraints, deadline pressures, a brief skimming of a StackOverflow answer with no followup or further exploration - any of these can lead to code that does the job, is ignored until only one person is left who understands it, and needs to be parsed out and refactored. I think you can only trust programmers with macros (or functions, or pointers, or object-oriented programming) if you have good practices in place to handle technical debt (clean naming, clean separation of concerns, adequate testing, etc.).

As a side note: the other big part about macros is YAGNI. Macros are fancy, but I've yet to come across a use case for it that can't be replaced with a simple function call. I'm also curious about how one would test a macro.

> I'm also curious about how one would test a macro.

The same way you test any other kind of function call?

> Macros are fancy, but I've yet to come across a use case for it that can't be replaced with a simple function call.

For starters, macros are mandatory for out-of-order execution; anywhere you need arguments to be manipulated before evaluation must be done in a macro. E.g., a short-circuiting "or" function is impossible without being either a macro or built-in to the language.

Another common use case is efficiency. Any substantial computation you do at compile/eval-time never has do be done at runtime. I've worked on a Clojure router that compiles its nested route structure into efficient string-matching/regex code before it ever gets turned into machine code. Other languages either have to re-parse the routes, or find a way to cache a more efficient representation as part of an extra build step.

I'm coming around to the same feelings about dependency injection that many others express about macros.

In broad strokes, I really like that composition is increasingly being favored over inheritance. But I've also inherited some deeply Byzantine amalgamations of decorators and abstract factories that certainly do eliminate a lot of boilerplate. But they do it at the cost of sometimes making it very difficult to understand where certain behaviors are coming from.

The worst is, most of them are ones that I inherited from my past self. The specialized knowledge it takes to maintain an application that's been written this way apparently fades quite quickly.

> [...] but at one point in time people might have said the same about functions, [...]

I really like your example, because it nicely highlights how much of a tooling problem this really is.

To call a subroutine, you'll have to arrange your argument values on the stack, then do the call instruction, after which your target procedure pushes the return address to the stack...

Except that almost no one does it this way anymore. I usually never think about all this stuff when I call `foo()`.

Writing a macro in C/C++ is the equivalent of doing the whole legwork to call functions yourself.

What I find really surprising, though, is that we haven't ever arrived at the point where we think that "most programmers can't be trusted with arbitrary mutability". It's everywhere, and since the C/C++ days, it got worse with many languages. I absolutely hate the house-of-cards code I regularly deal with in C# because people would have to write a couple of additional lines to encapsulate the internal state of their types properly. So instead, you get mutable public members or properties everywhere... * sigh *.

It seems quite intuitive that the complexity of code scales with the amount of build time logic it contains, whether that be C macros, JS transpilation, or whatever else.

I think if a lot of developers programmed with this mindset we'd be in a much better place. The best example I can think of off the top of my head is people choosing Vue over React "for it's simplicity". You're really just taking the little bits of ritual and reliance on JS fundamentals that React contains (which isn't that much in the first place) and moving that complexity into a DSL with a big transpilation footprint.

It makes sense from a "I want to play around and build features quick" perspective, but it's a horrid decision in terms of the complexity of the software and the number of moving parts it contains. Unless like many you (foolishly) believe the law of leaky abstractions doesn't apply in JS land for whatever reason.

I couldn’t agree more. I almost want to call it the iceberg metric. If 90% of your complexity is hidden underwater it’s still there, you just don’t notice it until your boat crashes into it.

But this is also how civilization is built. We need that 90% to be underwater just to build the remaining 10%.

Proper macros make that complexity much more accessible. That is, if things work correctly, you still don't notice, but if they don't, you get to easily fix them (though you still have to pay the cost of understanding it first).

Macros can be seen as incidental tooling reduced to only essential complexity. For comparison, a typical JS transpilation toolchain seems more like 80% of incidental complexity.

I definitely agree with this. I think there's obviously a point where the complexity savings from using a macro outweigh the inherent disadvantages. But I'd also guess that a large percentage of the programmers using them wouldn't know where to draw that line, and probably don't spare a single thought on it.

My point was more around instances where the supposed benefits of moving that logic to build time are superficial, which as you alluded to, is rife within the JS ecosystem.

I hate to pick on Vue too much, because for the most part it's a pretty good tool, but in their docs they claim one of the motivations for building Vue and using it over React was that it's more familiar to people who have built traditional web apps, because it leverages more html/css knowledge rather than JS knowledge (it doesn't, other than in superficial ways, by the way. But that's a discussion for another day). This seems insane to me. Just as insane as if you used a ton of C or Lisp macros to make the language look like Python, just because your devs were used to Python. I feel like in those communities you'd get called out for adding a bunch of accidental complexity and told to suck it up and just learn C. But in JS land it's just par for the course.

You also have hdom & co. coming from the i-am-not-a-framework side - https://github.com/thi-ng/umbrella/

Production size and evaluation times are a thing with react apps.

>template meta programming mixed with a lot of C macros...

The author cites Lisp macros. C macros, or C++ template metaprogramming, are to Lisp Macros as a 19th century locomotive is to a state-of-the-art MagLev train.

>I would actually argue that macros might be why Lisp didn’t become mainstream; it creates a lot of fragmentation.

This has no substance. Please take a look at how macros are used in Lisp.

>this is a nightmare for tooling

Not in Lisp.

You say “in lisp”. But which Lisp? Common Lisp or Scheme? And then which implementation of scheme? And which object system? See, that’s kinda my point. If I talk about Python or Java or C# we’re not even having this discussion, because they’re monolithic languages and features are core to the language, not built on top. On the other hand I can’t really move my code between Lisps.

And I don’t buy the argument that it’s just because Lisp has been around a long time. Python is almost 30 years old but there’s one canonical version. And seriously, I bet there are a lot of pythonistas that would change things about, for example, the object system if they could (explicit “self” comes to mind), but if they did that the community would fragment with some people using the original system and others using an alternative and it would be hard to integrate new libraries and know the rules of the module you’re working in. It’s a trade off no doubt, but it seems like most people have voted that a rich ecosystem of libraries without compatibility issues is more important to them.

That's why Lisp has a main standard. Common Lisp.

Most other languages, derived from Lisp, are incompatible - like Ruby is incompatible with Perl. Thus Scheme is called Scheme and not Lexical Liwp - it's now a new language family - the family of Scheme dialects - which are grounded in RXRS reports as language standard.

It's good that there is only one/two Pythons - but Python competes with a zillion other scripting languages - or they compete with Python. And they are not compatible at all.

Mmm, I think it's a huge stretch to say those languages were derived from Lisp. They certainly borrowed/stole a lot of features and were heavily influenced, but that's not the same thing. For instance, it would be way more accurate to say that Ruby is more derived from Smalltalk than it is from Lisp. (And not coincidentally, Smalltalk is a lot like Lisp in that it's a very minimalist language where most of the useful constructs are built on top of a very small core. And similarly it's also very balkanized.)

By the way, I don't mean to imply I don't like these minimalist languages with powerful extensibility like Lisp and Smalltalk. They're very elegant. I just think that there's a very large tradeoff to going that route. If I'm writing code for myself I love using Lisp or Smalltalk, but if I'm working somewhere and I have to debug someone elses code, I kinda want something boring and unsurprising.

> Mmm, I think it's a huge stretch to say those languages were derived from Lisp

I did want to say that. I said 'Most other languages, derived from Lisp, are incompatible' - that means the languages which were derived from Lisp - like Scheme, Logo, Dylan, Clojure, partially JavaScript - those are incompatible. Like Ruby, which was either influenced or in the same application niche (web programming, scripting, ...) like Perl is incompatible with Perl.

There is a similarity:

Lisp -> lots of incompatible dialects

Scripting languages -> lots of incompatible languages

But Lisp has with Common Lisp as standard, which has enable many different implementations with different capabilities, but a large shared core language.

Languages, macros, libraries, and apps are in descending order of how many people will use them and how tightly coupled they are to their problem domain. Thus they should be designed and created in descending order of experience and domain expertise. Very few people should create languages, a few more macros, etc.

I agree w.r.t. languages -> libraries -> apps, but I would say that macros are part of a library, along with functions, methods, datatypes, classes, interfaces, etc.

A typical library should usually define more functions/methods than datatypes/classes, more datatypes/classes than interfaces and more functions/methods than macros. The exception is libraries which only provide some particular macros or interfaces or whatever.

Macros are a kind of function that have the ability to hide a lot of things, including control flow, and often do, which often makes them unable to be composed. That's why I put them between languages and libs. They're a little less flexible than regular functions/methods/classes and they usually have tighter lock-in.

> ability to hide a lot of things, including control flow, and often do

They're still local transformations though, so I would think they're either:

- Choosing between their arguments somehow e.g. like `(unless condition body)` expanding to `(if (not condition) body)`, or `(with-foo body)` running `body` inside some setup/teardown boilerplate. In such cases we just need local reasoning (possibly after expanding the macros; but like functions, if you have to keep reading the definition to see what it's for, the abstraction it provides has been lost)

- Invoking other code (macros and functions); but functions can do this too, so such non-local reasoning is essentially the same.

A phrase like "hiding control flow" sounds scary, like we're implementing "GOTO" all over the place (or `call/cc` or something). Yet macros themselves can't do that; they could only do GOTO-like-stuff if there are GOTO-like constructs in the language, in which case (a) that's what's causing the underlying difficulty and (b) functions can presumably do similar shenanigans.

I must admit that I do most programming in Haskell these days, so I take it for granted that I can implement my own control flow using ordinary functions (thanks to laziness). That's just as well, since Haskell's macros (Template Haskell) are pretty gnarly; they're very useful on occasion though (e.g. I'm currently working on a script which reads in and parses a bunch of JSON from a filename (taken from an environment variable) at compile time, building a datastructure that I can query at runtime).

> which often makes them unable to be composed

Composed in what sense? I get that we can't e.g. pass macros to higher-order functions, and things like that; but I wouldn't say that's especially due to their use for control flow. I'm more frustrated by the general asymmetry between functions and macros, which can cause us to write the same functionality multiple times (I have the same frustration in languages which distinguish between functions and operators, functions and methods, statements and expressions, objects and "primitives", etc.).

> this is a nightmare for tooling

I self-inflicted some pretty awful debugging woes due to C++ macros a few years back. VC++ is incapable of stepping into/through macros - so a logical error in my macro had to be solved with good-old printf debugging. I learned my lesson about using macros sparingly.

This can obviously be alleviated with source maps. This presents a new set of dilemmas: does the developer want to debug the macro in its macro form, or its expanded form? Does the developer only want to debug a specific macro in expanded form, while keeping other macros expanded? How do you even debug a "case macro" in macro form? How do you debug the case macro code while it is expanding?

I am not so sure about the tooling situation, at least with Lisp macros you can run the macros and work on the underlying, generated code.

Exactly this. Given how a majority of JS devs tend to not learn from history and other languages, I would trust them with macros about as much as I'd trust a baby with a sportscar.

>but I wouldn’t trust 99% of programmers to be judicious with these things

The evil elitist in me wants to qualify that as "99.99999999% of JS programmers".

You've obviously not met many C programmers. I have a new general rule: if it confuses clang-format, it probably shouldn't be a macro in the first place.

The amount of time I hit that pulled out of my butt rule in daily code reviews is above and beyond what you would expect. Also what attribute of JS programmers would preclude them to abusing macros or templates more than c or c++ programmers already do?

> I know there’s the contingent that wonders why you’d ever want anything more than vim or emacs …

Well, we have tools that work. Why would we use tools that don’t work? It just seems weird to me, when a solution to a problem exists, to not use it.