Comment by fwsgonzo | Hacker News Reader

Comment by fwsgonzo | Hacker News Reader

Parent Full thread

Yes, precompilation is not optional. I've used many languages in sandboxes over the years developing both this and other emulators.

I really like Nim the most. It's Python spiritually, but with types. Types just really really necessary, and it does have FFI support so you can forward arguments either from a wrapper or use {.cdecl} directly. It's still not a barneskirenn (as we say in Norway), but definitely the most fun I've had using another language in my various sandboxes.

Nelua and Zig are close seconds. Both are just so easy to work with C-based FFI. Nelua is probably not safe to use, as it's still a work in progress. At least last I checked. Zig is very much ready to use. Maybe not your cup of tea, though.

Golang has a complex run-time and I don't recommend using it. It's definitely possible though as my sandbox does have a full MMU. Just expect integration to be long and ardous. And the ABI changes potentially every version.

Rust is one of the easier ones. I didn't find it fun to work with though. It has by far the best inline assembly, but too much fighting with the compiler. I know that people love Rust, and it is fully supported in my emulator.

C/C++ has the benefit of having the ability to have their underlying functions overridden by native helper system calls. Eg. replacing memcpy() with a system call that has native performance. It's too long a long topic to talk about here, but Nim and Nelua also falls under this umbrella along with other languages that can compile to C/C++.

Kotlin is definitely possible to use. A bit hard to understand how the native stuff actually works, but I did manage to run a hello world program in several sandboxes with some effort. I'm not 100% sure but I think I managed to convert a C API header directly to something kotlin understands using a one-liner in the terminal. I would say it scores high just on that. Again, just a bit hard to understand how to talk to Kotlin from an external FFI looking in.

JavaScript is of course possible with both jitless v8 and QuickJS. v8 requires writing some C++ scaffolding + host API functions, and QuickJS requires the same in C. Either works.

And I think that's all the languages I have tried. If you think there's a missing language here, then I would love to try it!

This is a really interesting list, thank you for replying!

(the following are entirely undirected musings on using your technology for my game engine, mostly in case anyone finds them interesting or has something to suggest that I haven't thought of since I'm new at this)

At least for my use case C, Zig, Rust, and even probably Nim are out of the question because I'm not just using scripting for sandboxing capabilities and such, I'm also using it because I want people to be able to program games and write mods in a high-level language, so using a systems programming language as my scripting language kind of feels like it defeats part of the purpose and I might as well just use dynamic linking with a C ABI or something crazy like that (I'm new to this so excuse me if that's nonsense lol). JavaScript and Kotlin are intriguing though, because they aren't systems programming languages, so I'll have to think long and hard about those;

I've been considering C# for scripting lately (because I like it well enough, it's widespread, and known in the gaming world) and I wonder if you can get natively aot compiled Kotlin to work, if you could get natively aot compiled C# to work too... there would probably be similar complications due to the large and complex runtime, but I know you can strip it down so I wonder how that might play in. I also wonder what the performance would be like compared to just hosting the dotnet runtime, which is what I was intending to do before I saw your post. Maybe at some point I should set up a benchmark to compare! Although I have to say the pre-compilation thing is a bit of a deal-breaker for me, since I really want people to be able to put plain text things directly in the game folder and see those changes in the engine without having to do any kind of build process, which is something that is achievable with the regular dotnet runtime using a precompiled assembly bootstraps the rest of the sctipts using dynamic assembly compilation and loading to collect everything else in the script directory.

Hm, if you don't actually need a sandbox then I think just using the C# run-time makes a lot of sense. I also like C#, but I've never been in a situation to try the fairly new AOT support. Sounds like a good idea, though. C# is a very good language.

C# runtime is HUGE and not ported to everything. The compilation step to native code takes a significant amount of time.

Good to know. That solidifies my plan to embed the dotnet runtime in my engine and load JIT code, not compile C# to AOT and then run that through an emulation layer

Do you mean compiling the .NET runtime (CoreCLR) itself or AOT compiling C# code? Because I recently tried AOT compiled C# and that builds quickly. And you don't generally need to compile the .NET runtime when embedding it because you can just load the builds they ship.

Simply running a DotNET dll. I just tried a basic "Hello World" program compiled to a CIL .dll and to RV64 machine code, on my x86 Linux machine (original ThreadRipper 2990WX) and then running it through different JIT/interpreters. Wall time:

DotNET: 0.061s

qemu-riscv64: 0.006s

Spike: 0.031s

Qemu is JIT with Linux syscall layer built in (in native code).

Spike is a RISC-V interpreter with Linux syscall layer provided by interpreted RISC-V "pk"

So the DotNET JIT has quite a high overhead for startup, or one-time code.

For very compute-intensive code DotNet has an advantage. e.g. on my own primes benchmark (https://hoult.org/primes.txt, https://hoult.org/primes.cs)

DotNET: 3.5s

qemu-riscv64: 10.2s

gcc: 9.7s (i.e. defaulting to -O0)

gcc -O1: 3.2s

DotNET beats emulated RISC-V here, but the RISC-V emulator (with RISC-V code compiled with -O1) is pretty much as fast as a lazy person compiling C to native x86 gets.

Oh, so you meant startup time for the runtime, JIT, etc. then. That's not exactly relevant to usage in game engines here because it is only done once.

If you want to compare startup times you should be using .NET's AOT compilation so it doesn't need to load the full runtime or do any JIT compilation. See here: https://learn.microsoft.com/en-us/dotnet/core/deploying/nati...

I'd be interested to see Scala (native).

Reply on news.ycombinator.com