Comment by aeturnum | Hacker News Reader

Comment by aeturnum | Hacker News Reader

Parent Full thread

I have really enjoyed working with Lua when it's come up and I think it's an extremely good language. In particular, as Noë says, its interface for embedding into C/C++ is clean and flexible. There are performance concerns, but considering how widely Lua is used for game logic in high performance video games clearly those concerns can be addressed.

That said, I think there are other warts that can be confusing and make the language difficult. The Nil-Terminated Arrays section points out one oddity about `nil` - but there are many others. There are a number of confusing corner cases (or were - I haven't used it in a while). As an example, if you have a list of values and you pass that list into a new table, the resulting table may be different than if you add each value in that list in order.

I also think that the flexibility of embedding means that you are often learning the Lua <=> C interface as much as you are learning Lua. So jumping from deploy to deploy is more confusing than most languages.

Nevertheless I fully agree the language is underrated!

> but considering how widely Lua is used for game logic in high performance video games clearly those concerns can be addressed.

It’s ok, but should be much better.

Once upon a time Supreme Commander used Lua. Perf on consoles for SupCom2 was catastrophically bad. We switched to Kore which was effectively Lua but way faster (10x?).

Kore was bought by Havok and rebranded into Havok Script. Havok got bought by Microsoft. I’m honestly not even sure if you can still buy it today. It’s not listed on their website.

These days I think you’d use Luau? I’m not sure. https://github.com/luau-lang/luau

LuaJIT is pretty good, but some platforms might not allow it. iOS technically didn't allow it when I used it before, but Apple either didn't _actually_ care or just looked the other way.

> you are often learning the Lua <=> C interface as much as you are learning Lua.

One time I had a memory leak in a game, something like exactly 4 bytes per frame.

It turned out I was not popping the Lua-C parameter passing stack properly after each frame.

I found this out after some unhelpful person suggested that "all programs just leak memory" lol

The Lua C API is fantastic from a design perspective, but as you have discovered, it is rather error prone. Fortunately, there are many wrapper libraries that take care of all the stack manipulation. For example, I'm using https://github.com/ThePhD/sol2 to interface with C++ and the API is incredibly ergonomic.

How is it fantastic, it’s just barebones stack interface thrown in your general direction.

Any integration code that doesn’t use an ad-hoc helper library looks like assembly listing (not even in disguise, cause it basically is bytecode but in C).

> it’s just barebones stack interface thrown in your general direction.

I do not do a ton of low level development, but my impression is that you do not want a demanding interface. You want an interface that is straightforward, doesn't have complex pre & post conditions, and doesn't crash. Leaking 4 bytes per frame is bad but crashing would be worse! You may think you want a library that refuses to function at all unless its called perfectly, but once you're in the trenches you may change your mind.

To my mind, it's a very good sign when there are popular "adapter" libraries. It means the underlying code (Lua) can be used many ways and people have found best practices for different approaches. When I used it I was not working on a game and it would have been worse for us if Lua's interface was "built for games." Being very bare bones and low level allows flexibility and that flexibility ensures a healthy ecosystem.

  #define T lua_gettop(L)
  …
  int t = T;
  …
  assert(T == t + expected_inc);
  …
  lua_pushvalue(L, -2); // note: this is b
  lua_remove(L, -3); // old b
  // eventual realization that
  // -index is fragile
  // and rewrite to
  int start = T;
  int a = ++start;
  int b = ++start;
  luaL_checkstack(L, 10, "…");
  …
  lua_pushvalue(L, b);
  lua_remove(L, b);
  lua_call(L, …);
  luaL_ref(L, …);
  …
  lua_settop(L, t - 1);

If you have ever done this, idk how that feels flexibility. I’d rather get handles to a and b and directly call a function with them as like:

  LFN *func = LGETFN(L, LARG(1));
  LTAB *retvs = LCHKTAB(
    LCALL(L, func, 3,
      LGET(L, LARG(2)),
      LGET(L, LARG(3)),
      LGET(L, LUPV(1))
    )
  );
  me.retvs = LREF(L, retvs);
  // everything but ref’d
  // things disappears here:
  return;

And avoid all this stack-in-your-mind and stack-in-your-int nonsense, which is completely unnecessary in C and I fail to see where exactly it is flexible, compared to the short snippet above. Stack is just how vm sees it, it is not any special “for you” thing, it’s literally implementation shining through.

I’d like to see a specific example (low-key, I left Lua) on how its stack provides more flexibility or anything, because in my humble experience it does not, absolutely, do anything like it. It just makes you play these off-by-one games all day for no good reason.

Like I said I am no low-level C guy, but my impression is that the existing structure is allowing you to break up the setup however you like across functions / calls / etc (just keep the context) instead of keeping many different contexts. The approach you roughed out is fine as long as you can 100% swap any arg for any other when you make the call - but I think often that's not true? Or if it is you need to pass all the info. So I suspect you are drilling down on a simple case that elides the problems that will arise in the complex case.

You can, ofc, write your Lua wrapper like your example - all in one place. It reduces the risk of misuse. But you could also do it differently - which is the flexibility that I think is a strength.

> How is it fantastic

See my other reply: https://news.ycombinator.com/item?id=42519033

> Any integration code that doesn’t use an ad-hoc helper library

There are mature bindings for most languages. You only need to deal with the C API directly when you

1. Create a new language binding

2. Write a Lua extension

3. Call Lua from C

Typically, Lua is embedded as a scripting or configuration language into another project, and in this case you wouldn't even be aware of the C API.

The problem is, if we are talking about Lua API, typically you are on the other side of “embedded in another project”, as an embedder. My reply shortly above goes into some details.

(from elsewhere) It is a low-level API for creating bindings to other languages and as such typically isn't used directly. (Well, unless you are writing C :)

Yeah, unless. But even under this assumption, wrapping handles to values would be much more easier in a language-specific wrapper, because it wouldn’t have to watch out for stack overflows etc and solve the exploding stack problem in general in naive api-user-side loops. This thread started as Lua stack API is better than Python’s, iirc. But Python API already looks like a high-level wrapper doing exactly that, so… what was the point again?

I’m a little concerned that everyone talks high matters here without providing examples or at least looking into “final particles”. Cause that’s where claims get tested against reality. I spent too many years in deep Lua and its mailing list to miss something obvious that everyone knows.

Returning handles would introduce subtle memory management issues because the garbage collector wouldn't know which objects it should keep alive. With the stack-based API the user can only ever interact with objects that are on the stack and thus are kept alive by the garbage collector. The Lua authors explain this in the following article: https://queue.acm.org/detail.cfm?id=1983083

I would agree that the Lua C API itself is not very eegonomic, but the wrappers/bindings it enables absolutely are! For example, integrating Lua in C++ with sol2 is an absolute joy.

Yes it could pin accessed objects in some arena, like idk... "stack", for the duration of the call. And release it after an "invocation" returns, or if the host requests it explicitly. It's almost like my example already mentioned it near "everything but ref’d things disappears here" (/s).

I understand how gcs work and their nuances. My only problem was with the "fantastic" part. Lua stack is not "not ergonomic", but absolutely shit to work with, and "there's wrappers" is not an argument, cause they are not a part of distribution but external projects ought to fix the mess.

> And release it after an "invocation" returns, or if the host requests it explicitly.

This would only work when being called from Lua, but not when the host calls into Lua (on the top level).

An API cannot be “fantastic” if it’s error prone.

It is a low-level API for creating bindings to other languages and as such typically isn't used directly. (Well, unless you are writing C :)

What I find fantastic about the stack-based API is that it exposes very few interpreter internals. As a result the authors may drastically change the implementation without breaking the public API. Most importantly, they hide the internal object representation and as a user you only ever deal with plain C types, like integers, floats, C-strings and user data pointers. As a consequence, you don't have to think about memory management because all types are passed by value! In fact, someone could theoretically write a Lua implementation with reference counting and the API wouldn't have to change. It really is a very smart and elegant design.

It might be fantastic in other ways; there are multiple ways an API can be good. For example, it could be very expressive and flexible, or it could be especially amenable to formal proofs with things like TLA+ even if it's error-prone in human hands, or it could facilitate very high performance. Even on the error-proneness axis, an error-prone API might still be less error-prone than the alternatives.

You can only make a C API for interacting with a precise garbage collector that ergonomic. And, to be honest, I wouldn’t call the situation described by the GGP particularly problematic—you messed up memory accounting and you got a memory leak. In most garbage-collected runtimes, if the runtime messes up memory accounting, at best you get immediate memory corruption, at worst you get memory corruption much later in some unrelated part of your heap. So this is pretty mild all things considered. Of course, if you’re working in a language with greater metaprogramming capabilities than C, absolutely do abstract the stack into something more manageable.

Reply on news.ycombinator.com