Hacker News Reader

Comment by MontyCarloHall | Hacker News Reader

I get that this is mostly for fun, but this is as good a place as any to bring up an issue I rarely see discussed with any post about a new language.

Well-established languages have tons of widely used, highly vetted libraries implementing functionality that doesn’t exist in the new language and would be totally impractical for an individual to implement themselves. For example, if I’m doing scientific computing, I need a good linear algebra library like Eigen or Armadillo (or Numpy/PyTorch/Tensorflow), all of which would be impossible for me to implement myself.

Therefore, for a new language to catch on, it needs a good foreign function interface. Yet FFIs are almost never brought up whenever new languages are discussed on HN. (Again, I realize that this is just a fun project and widespread adoption is not a goal.)

View on HN

We've been investigating coupling syntactic and low-level FFIs [1]. Our most recent work will be published at the upcoming Scheme Workshop [2], where we integrate Gambit Scheme with CPython. We share the concerns you have and have had quite good success with the Scheme<->Python interface! The syntactic interface makes it really enjoyable.

[1]: https://zenodo.org/record/4711425

[2]: The paper should be up in a few days I suppose.

Totally agreed about FFI. I wanted to make it easy to interop with C code and write expressive bindings.

Check for example the language bindings to LLVM's C API (fairly low level) and Tree-Sitter which is used internally (a bit higher level bindings)

https://github.com/tibordp/alumina/tree/master/libraries/llv...

https://github.com/tibordp/alumina/blob/master/libraries/tre...

I think UFCS makes it quite nice for bindings, since external C functions can be used as if they were methods if the object is passed as the first parameter. So in many cases there might not even be a need to write wrapper structs for bindings that feel native.

Of course, it's still a manual process and since Alumina is just a compiler and stdlib for now (no llibrary ecosystem, no compiler driver), it's a bit cumbersome. But I like the approach Rust has with bindgen and cc crates, to automatically create bindings for C and C++ code.

UFCS making C bindings a hell of a lot nicer is one of the things I love about Nim, so I’m happy to see another language try it!

We are building a new language that is not for fun and we have thought of this important issue. We solve it with a Package Ports and Package Adapters. The Bitloops Language (BL) is a transpiled language with support for only TypeScript at the moment. Nonetheless, if it picks up we will be adding support for Java, C#, and C++ so that people can leverage their investments in their existing packages.

https://github.com/bitloops/bitloops-language

What's an example of a language with "good" FFI?

Rust's seems pretty good. I've been doing some Rust/C interop and it works very smoothly

Nim is quite enjoyable to use. I do a decent amount of scientific computing, interfacing with various c/cpp libs

Calling C functions from zig is very nice, partly because zig uses llvm to parse C header files and uses that information.

LuaJIT has a pretty great FFi which was later ported to standard Lua; Python’s ctypes provides a decent FFI.

Nim, Zig and Rust do too, but they have semantics much closer to C, so it’s almost free (especially when they can all use llvm directly for code generation, and Nim’s preferred way is to compile through C in the first place)

With the D programming language, to interface with C's stdio.h:

    import stdio;

    void main() { printf("hello from D!\n"); }

is all that's necessary, as D has a built-in C compiler that reads stdio.h, compiles it, and presents its interface to the D code.

I hear about D often on HN but I never see it being used in the wild so to speak. When I looked at it, it seemed, like C++, a hodgepodge of features built up over time rather than being a more deliberately focused language.

I hate to compare and contrast but Rust comes to mind as one of the latter, probably because it has a somewhat more intensive and extensive RFC process for adding new features.

D has made its mistakes (there's no way to evolve a language over 20 years and not make a mistake here and there), but we also have a deprecation process to remove them.

The people who use D tend to like it very much.

This is even more impressive than LuaJIT's FFI in my opinion, which I previously thought was state of the art. You basically pass in a preprocessed header, and LuaJIT does the rest.

All this to say, I feel that any FFI that requires you to manually write bindings is rubbish.

We lived with manual bindings to interface with C for years. Building in direct support for C itself is a huge leap forward in usability. In several ways it's even better than C++'s support for C.

How does it deal with the macro-in-macro-in-macro soup that a non trivial amount of headers are? c2nim does a decent job but struggles with some of those types. gcc -E on the header works around it, so I wonder if c2nim could be made to operate similarly to what you’re describing.

Libraries are a farce. Stdlib or die. I 100% mean this and live this.

Libraries are how software is shared and reused.

Do you really intend to write your own library for a given task when there is a perfectly good and mature library that does that task and more freely available?

How long would it take for you to write your own Sqlite or your own Nginx?

That's a strange take. Do you mean that you would rather call external binaries directly, like Bash does, or that you would rather re-implement all functionality you need, however complex?

To me, there is value in the process and craft. It started as a one year challenge to use only stdlib in all projects, for work and personal code, and became a way of life. Though it may not work for everyone, or all languages, or all skillsets, it was a revealing experience.

In retrospect, it made sense given that in my community we make our own furniture, instruments, tools, foods from whole items. Code seemed like the next logical step. During travels, once there was a man who had a word in his language for this. He said it was an aphorism loosely translatable to "the beauty of struggle". As he explained it, this is the positive benefit gained in return for the time and effort to do something as an act of appreciation of the craft, and how the value in the experience surpasses the debt of time and sweat.

To put it in a more modern and eastern philosphical context, think of it like Kata.

https://en.wikipedia.org/wiki/Kata

I too suffer from obessively overcoming challenges. I will say this, in a snarky and friendly way. I hope you understand.

If developing with high level abstractions wasn't challenging enough, your vision was too small. Surely you can think of a problem to solve that even Copilot would be of little use.

Imagine what you could build if you got a little more help from third party libs and a little more ambition?

I appreciate you for relating your experience. It's not ambition or challenge that is lacking, we are what's commonly called 'Amish'. This is just how we do things.

https://en.wikipedia.org/wiki/Simple_living

https://en.wikipedia.org/wiki/Nonconformity_to_the_world

I'm sure this creates more questions than it answers, however, I hope it lends some context as to how we've both arrived at our own respective happy milieus.

I can certainly identify with this.

Having done enterprise java for so long, I strive to be as reductionist and minimalist as I can be in order to avoid dependencies outside the JDK.

Vying for "you aren't going to need it", trying to keep things as simple as I can.

But it also comes down to not overcomplicating stuff we've already written.

Obviously there's an urge to reuse your own libraries, but you have to strive to not overcomplicate them as they're applied to new use cases as you drive the peg in to a rounder hole.

It's very easy to FactoryFactoryFactory in Java, but flexibility and configurability leads to complexity and black hole of combinatorial testing.

Almost better to fork the earlier library and hammer it to fit the new space without regard to its old use in the other system. But that leads to file replication (notably files of the same name doing subtly different things). And there it's a challenge to go "gee which one should I use" when you're working on version 3, when what should be happening is "pick the best one" and keep hammering.

We used to have our general purpose "catch all" library which, in the end, indeed, "caught all". "Oh look, somehow I have jars from Clojure, Groovy and Scala, when all I wanted was a URL Builder."

That's a whole lot of words for describing NIH syndrome [0].

But sure, yeah, there's value in practicing your craft and making all the things.

[0] https://en.wikipedia.org/wiki/Not_invented_here

Why even use stdlib? Or an OS? Just write everything every time down to the bare metal.

Chuck Moore, is that you?

Would that include things like encryption? No exceptions for your rule?

The parent will never "need" something as "complicated" as encryption as you can't write anyway any serous software with the stated mindset.

The whole point of modern tech and science is that everybody is standing on the shoulders of giants. Without that all you probably can get at max is low tech form 200 hundred years ago…

Reply on news.ycombinator.com