Frustrations with the OCaml ecosystem while developing a synthesizer library

> which despite all the hate for them are always the most reliable build way.

Based on the amount of totally broken Makefiles I find on a regular basis as a package maintainer (logic bugs, recursive make, invalid deps = no parallel builds) suggests to me this is not the case at all. Doesn't even touch on how awful these things are often written, even if they work reliably enough. Insane idioms in the name of one-liners, implicit dependencies on GNU specific features, incredibly inefficient shell pipeline commands, overly large rules that mean many intermediate steps aren't cached, subtly broken rules and invocations based on spaces or non-utf8 chars, etc.

I worked on the Glasgow Haskell Compiler and the truth is that the Make-based build process was only reliable because a massive amount of effort was spent on it. I suspect it's probably the same with OCaml, and not because Makefiles are special. There are degrees to it all, but they certainly don't have good syntax (too terse), aren't easy to read (batshit insane escape rules that were fixed by Plan 9 mk in 1991), and make expressing many types of dependencies really awkward by design (I want a dependency on an environment variable?) They're just really easy to get started with and you can write something broken that looks like it works zero effort, honestly.

It should be a crime to mention "Make" and "just works" in a single sentence.

I like it because:

1. Well, I made it because I was dissatisfied with what things were like at the time, so it does almost exactly what I want. To be fair, dune was in its infancy when I created pds but even now I think pds is easier to use for most projects. 2. It outputs makefiles so you have all the power of makefiles if you need it. pds itself is really quite small. 3. pds could have different backends, such as a dune one. This doesn't quite work in practice because complex repos that I have end up using makefile features, but it wouldn't be a huge reconceptualization to make it support other backends, but it would make it more complicated. 4. Because the pds input file is TOML, I have easily built other tooling on top of it. For example I have hll which takes a pds config and outputs an opam package. hll needs a small amount of configuration but otherwise it pretty much just works. I also created merlin-of-pds which produces a merlin config of your pds build. 4. pds can support very complex builds, for example I have kqueue bindings using ctypes which has to go from compile and run ocaml, which produces c code which then needs to be compiled and run, which then again produces ocaml code, which is then the library interface to kqueue. There is no magic to make this happen although it does twist the brain a little bit to implement.

The Ocaml thought leadership in terms of builds is focusing pretty hard on making Dune the answer. Personally, I don't like dune. I think it's complicated, it's too ocaml focused, it's too "cute". In that sense I find it similar to opam, which I do use everyday but again, I find it too cute. The dependency version solver always does "the right thing" given its input which many times is not "the thing I want to do".

My biggest complaint, as things are now, is that sometimes the thought leadership makes decisions that makes the life of people outside that specific toolset harder. It's not that often though. The biggest painpoint I had in the last few years was switching to ocamllsp, which was designed to depend heavily on a dune world. It took a lot of asking to get supporting merlin files added to ocamllsp.

And I also think this is what I enjoy about the Ocaml community. I feel fairly confident I can build my own tooling in a way that serves my needs and I can use the larger ecosystem where necessary. Ocaml has a culture of rugged indivdualism, IMO. I think it will slowly converge as new people come in, with expectations, or us people that care more about using Ocaml than about using other peoples' tools age out.

P.S. I really wish opam was built on top of Nix. But I literally wish that about every bespoke package manager, which just end up making the same mistakes over and over and solving the same problems (poorly) over and over, and nothing is compatible, and it just sucks. Nix everywhere, please.

The art of writing a reference manual that actually explains how the software works is almost lost, it seems.