Comment by nostrademons | Hacker News Reader

Comment by nostrademons | Hacker News Reader

Parent Full thread

The big question is "Would you want to use this in a mission-critical project?" Emscripten lets you write webapps in C, for example, but outside of sharing common libraries across server/Android/iOS/web clients, few people use it.

Pretty much every professional job I've had used polyglot programming of some sort - when I was in financial software it was mixed Java/C/Fortran numerics, when I founded my first startup it was polyglot Javascript/ActionScript, and at Google it was a large server that was first Python/C++ and then Java/C++. The boundary between languages is always problematic. And the reason for that is because you choose memory layout to make certain trade-offs around the access patterns for that data. Do you inline data structures in a vector, or chase pointers? Do objects carry their type information with them so you can manipulate them dynamically, or do they throw it away at compile-time for greater efficiency? Do you get O(1) string indexing or native UTF-8? Do you allocate on the stack or the heap? Do you copy, borrow, move, or COW?

And then when you go to switch representations to call into another language, you pay a cost to convert all the data structures you might be touching. In many cases, that cost could be more than you saved by using optimized representations in the first place.

That's exactly how Truffle's cross-language-interface works though. Instead of paying the high cost of conversion, data stays in its existing representation and it is the interface code that is recompiled to fit.

An example is JRuby+Truffle where in its C extensions, pointers to Ruby objects act to the C code as if they are pointers to MRI data structures, but behind the scenes Graal compiles code that accesses them like the JRuby objects that they are.

The C-memory to Java GC transition doesn't work as well, but you can get around that by implementing C-memory using the Java GC like the Managed-C paper does, alternative implementations only have to keep the same behaviour, not the same underlying data representation.

Interesting. What happens if the native memory layout doesn't support a feature that the extension code requires? For example, what if a JRuby+Truffle script calls into a C library, which allocates a struct and invokes a Ruby callback with it, which then wants to access the fields via reflection? C doesn't normally include type information with its structs, so how would the Ruby code know the memory layout of the object?

What if it's not the C extension in your project that does the allocation, but a closed-source third-party library that your C code calls into? Do you need to recompile all source code to generate the appropriate typemaps?

Yes. Sulong assumes that at least the LLVM bitcode including debug information is available. It does not work with third party machine code binaries where no source information is available.

The idea of Truffle language interoperability is to avoid switching the representation at the language boundary. Objects carry their type information with them that includes the semantics on how to access their properties. This also allows Truffle to fake up the existence of an object without actually performing allocations (e.g., a table in raw buffer format can be interpreted as an array of objects). This clear separation of logical and physical layout enables efficient data representation in the context of higher level languages that typically suffer from pointer chasing and object header overheads.

Right, and my point is that this won't work unless you dictate a common memory layout for all Truffle objects. And if you do that, then you lose the ability to make trade-offs about object representation that depend upon access patterns that only the programmer can know. The whole reason we have different programming languages is because not all domains of computation face the same trade-offs.

It works without common memory layout between Truffle languages. It even simplifies the ability to use diverse physical memory layouts within the same language. The programmer specifies the logical layout based on the semantics of a language. The runtime decides how to map this logical layout onto the physical hardware. It can take into account the trade-offs the programmer decided to choose.

Right, and my point is that the reason people continue to use C++ or Rust over JVM languages is because there are some use-cases where the JVM's decision about how to map the logical layout onto physical memory causes unacceptably high memory usage and/or cache misses, or prevents them from taking advantage of clever serialization formats (Cap'n Proto, for example, loses much of its performance benefits on the JVM without the use of sun.misc.unsafe). Will Truffle allow the programmer to override Graal's decisions on this and manually specify memory layout? And if so, how are conversions between different language formats specified?

If you already buy the JVM's premise of "just let the compiler do it, we'll figure out the most efficient representation", then this is a non-issue. But there are still programmers out there who believe that Rust or C++ or Go or CPython or whatever presents a better memory layout for the tasks that they wish to accomplish, or language designers who think they can do better than all of the above, and these are the users that Graal+Truffle is trying to win over. What's the story for them?

The default when running static languages like C++ or Rust or Go via Sulong is to stick to the memory layout chosen by the programmer. There is no conversion to typical JVM memory layouts. Nor does running on the JVM has major restrictions for the framework (we are working on a version that does not run on the JVM in the SubstrateVM project). What Graal+Truffle allows is for library writers and language designers to invent new representations without any restrictions on the layout and have them interact with the rest of the system. We want programmers to use the best language for the task. And even combine multiple languages within one program. Foreign objects can be passed around freely as parameters and local variables, but there are restrictions when building combined data structures - e.g., if you have a highly optimized native data structure, it is not possible to install a pointer to a JavaScript object in it without performance loss.

Interesting. So, if I'm understanding this correctly - Truffle allows a language designer to specify a certain logical layout for data structures, along with hints for how this will get converted to a physical memory layout. Language implementors also get the full power of Graal for lowering their AST to machine code and other compiler tasks. On cross-language boundaries, it generates automatic accessors for other languages to access that data, using the logical layout to identify how particular fields need to be pulled out and manipulated, but not requiring that the full data structure be converted across the foreign-call boundary. One consequence of this is that nesting & embedding of data structures may require an explicit conversion, since if an object is a hashtable in Javascript but a packed series of fields in C, it's obviously not going to fit.

Sorta like SWIG++? If you could do SWIG but never require that an end-user write a typemap or debug a crash themself, there'd probably be a big market for that.

Yes, this is an excellent description.

> Cap'n Proto, for example, loses much of its performance benefits on the JVM without the use of sun.misc.unsafe

Not actually true. `ByteBuffer` lets you read integers from arbitrary offsets, which is all Cap'n Proto really wants.

Reply on news.ycombinator.com