Lua and Python (2020)
lwn.net
lwn.net
If you are whipping up a command-line script because what you want is too complex to attempt in Bash, use Python. Lua is terrible for this.
If you are making an application from scratch using just the stand-alone interpreter, your scripts and a bunch of dependencies from the internets, use Python. People work really hard to make this semi-viable in Lua, but it is a path of frustration.
If you have your own C/C++ program and you want an easy path for configuring and composing features that are written in C/C++, deciding between Lua and Python depends on your goals.
Do you want MAXIMUM POWER for the scripter to whip together any crazy feature they might dream of? No concerns about footguns, hanging ropes or corner-painting? Memory and performance are secondary concerns to flexibility? Go use Python!
Do you want minimum power for the scripter so as to keep their creations constrained within a defined box? Is there a pre-defined role that the script will play? Are there definite restrictions on what should and should not be done in scripts? Are you going to be responsible in practice for debugging some monstrosities that the scripters build for themselves? Are performance and memory primary concerns above flexibility? Then Lua is well suited for embedding into your application.
It's unfortunate that Lua doesn't work well for this. The Lua implementation's light weight, portability and robustness [0] seem ideal for building these applications - especially for building applications that can be distributed to end-users (a scenario which is not a focus of Python [1]).
Obviously Lua is never going to include a large standard library because that conflicts with its embeddability. However, even in the Python community, “batteries included” is no longer considered entirely beneficial [2]. Instead, I think Lua would benefit from a set of officially recommended external libraries. This would improve the situation for people trying to write standalone applications in Lua, and I believe it would also help those who are writing scripts targeting embedded interpreters.
[0] https://www.lua.org/bugs.html
[1] https://twitter.com/gvanrossum/status/1347045076401147904
That said - the Lua ecosystem in 2021 is not the same as it was in 2005 or whenever people compare it to. There is a healthy amount of Libraries and choices of libraries for solving most tasks on Luarocks. And I would suggest you produce a better end project by selecting which libraries are included in your project that uses Lua for scripting. You can even further remove your choice of the libraries that _are_ included ootb.
[1] - https://github.com/rjpcomputing/luaforwindows
[2] - https://luapower.com/
[3] - http://luvit.io/
Back in games we put the whole game state (including ai routines written by designers using coroutines) in a 400kb preallocated block on a system that had only 8mb of system ram. Worked like a champ.
We'd also manually run a fixed number of GC steps per frame in our games. The higher average CPU use was worth the much lower variance.
I would rather use Tcl for that purpose :)
You can also get to a middle-ground state between "no batteries" and "just enough batteries" to get a large portion of the expected work done if the problem space is understood and has some bounds. E.g. including a library to make HTTP requests, parse JSON and work with an embedded SQL database will be enough for a lot of use-cases in the "user-defined glue logic on an existing application" space.
> "But no one is suggesting that a set of batteries become a requirement for Lua—even the existing Lua standard library can be compiled out for use cases where it is not needed"
It's great at filling that niche but if I want something like Python I just use Python.
If you have in-house coders who can't agree on what language to use, I guess WASM could allow them to agree to disagree. I'm not sure if that's a better path than using the languages' native tools and focusing on improving the compile-debug iteration time.
If you have external coder customers who will demand differing languages, WASM sounds great!
If you have external, semi-technical users who don't flat-out demand JavaScript-or-Die, standardizing on either Lua or Python is probably better for them both in terms of complexity and consistency of documentation and community help.
In general, for semi-technical users, see my above post on power levels.
I personally DO use Lua sometimes to code general purpose stuff.
Still, I don't see Lua ever having "batteries", the language wasn't invented for that purpose, and the original sponsor of the language wouldn't want it changing in that direction (I dunno if they still sponsor it though).
Lua was invented by PUC-Rio (Catholic University, Rio de Janeiro campus) at the request of PETROBRAS, its purpose was "describe" stuff, and replace another language that they already had, plus being turing complete, because they got tired of hacking things endlessy whenever they needed to describe something "new" in their systems.
So to "describe" stuff, Lua doesn't need libraries.
That said... currently Lua most popular use is add some kind of scripting environment to other software, still that often means a library would be useless, and even then sometimes that usage is still to describe things, for example the game "Crusader Kings 2", use Lua to read a file named "defines.lua", and all it does is set some constants, another use of Lua that I find interesting is "xmake", Lua programs in it are very descriptive, using Lua syntax features to "tell" things to the system, instead of looking like anything resembling a normal imperative program.
EDIT: about community size... yeah, it is tiny :( I never found a Lua job, companies that use Lua often hire a programmer of other languages and teach him Lua, I am also currently using PUC-Rio official GUI library (iup) and never found anyone else using it to ask questions about it, I've been in PUC-Rio during a tech event, and when I asked around if anyone knew where Lua authors were, the people that study and work there, didn't knew they had anything to do with Lua in first place!!!
Lua is a very good choice for config files. Wezterm uses lua for configs. Lite editor uses lua only plugins and configs. And many more. Maybe someone/group of people can step up and declare an universal config format using lua for everyone.
As for batteries, I would love to have that but wouldnt that again defeat the whole purpose of Lua as embeddable? But of course people would love to have it.
Speaking of general purpose application programming, you might be interested in my new(ish) project Teal:
https://github.com/teal-language/tl
It's a statically typed dialect of Lua, which is compatible with all modern Lua versions.
The core compiler itself is one tl.lua file which can be added to any Lua project, and includes a Lua package loader that makes Lua's `require()` support .tl files, and there's also a tl CLI for generating .lua from .tl files.
Maybe you can clarify what "describe" means. It reminds me of the subtle-possibly-false dichotomy between a program and a configuration. Turing-completeness could be one way to think about that dichotomy it, but in this case Lua is definitely Turing complete.
In any case, if there's commonality to the stuff being described, then that's the kind of thing that is "batteries". Sequences, associative arrays, etc. come up all the time, and that's available. Surely there are some things that come up a bit less often when describing systems, though I don't have a concrete example on hand.
My main question is, since any program is a description of a computation, then what kind of distinction are you making?
https://andregarzia.com/2021/01/lua-a-misunderstood-language...
I'm speaking from the point-of-view of someone who used Lua for games, hardware, and web, in there. I am not an expert in the language, but I feel that many people here are approaching the language with the wrong perspective, and thus are unable to see what makes it great.
All I write there is my own personal opinion of course, I'm totally OK if you still hate Lua after reading it, but I'd encourage those here who have criticism about it to double check my article and reach out if they want to talk more about it.
One of our motion equipment suppliers had an existing set of products for industrial automation which were originally programmed in C++, but also offered a more standards compliant version for doing PLC programming.
My employer negotiated very early on ( pre 2000 ) for them to provide a simpler interfacing layer, and we got Lua libraries as a result.
Now we do the bulk of our automation equipment programming using a fully featured Lua library, which interfaces with the core controller/amplifier/motor/sensor/IO libraries already present on their original product.
A sizeable portion of our equipment now are programmed purely in Lua, and we work closely with said supplier so that they update the library when a feature deficiency is noted.
Wish I could say more about it, but you can count me in the group that enjoys programming in Lua, and does it daily.
I'm not using it for general purpose software, its for a highly specific use case that is tied very tightly to proprietary hardware unavailable for purchase without purchase contracts, so this is very different from what normal programmers might have to deal with.
There error messages aren't great, the 1-indexed arrays, the syntax is super weird.
I understand there is good technical aspects to the language, but I am really thrown off by the syntax. I get that that could go away with time, but I have no reason to believe that "learning lua" would actually bear any fruit.
Part of me is convinced that Lua is liked simply because it's unpopular, and that if it was used in production more, people would hate it. Kinda like we hate javascript and c++. Maybe it's that we hate our jobs, and blame the language, and then we go home and screw around in lua, we enjoy ourselves and blame the language.
EDIT: Just to piggy back on my comment here:
I've been optimistically looking at the wren scripting language. That seems like the "correct" way to make a scripting language, at least from the outside. I should make it clear that I am not speaking from a position of authority, but I am sharing my opinion of LUA as a pessimistic outsider, which I think is very common view.
[1] https://wren.io/
In terms of syntax it’s basically JavaScript with stupid array indexing, which is enough to turn people off I’m sure.
Unfortunately, the language itself is "quirky" at best. Its has unusual syntax, variables that are global by default, and pervasive use of `nil`. I personally also dislike build-your-own-object-system and the language's conflation of lists and maps.
I too have been looking for an alternative to Lua - I want a language like Python with an implementation like Lua's, which I could use both for embedding and standalone command-line applications. I've even considered building something like this myself - either from scratch or, perhaps, as a fork of MicroPython.
No language has _great_ error messages. But comparing the Lua message for e.g. `print(nil+5)` to the Python message for `print(None+5)`, I somewhat prefer Lua's natural language approach "attempt to perform arithmetic on a nil value" over Python's jargon-heavy "TypeError: unsupported operand type(s) for +: 'NoneType' and 'int'". Of course Lua's message could be better (it could/should tell you which operation and where the nil is in the expression), but "not great" basically describes every language's error messaging, and Lua's messages are not exactly eye-bleeding like C++ STL errors either.
> the 1-indexed arrays
1-based indexing is actually really human friendly, and programmer affinity for 0-based indexing in languages that don't map directly to RAM pointer offsets is largely excess familiarity with C plus cargo-culting and wanting to believe that everything Djikstra ever said is universally true regardless of context. Indexing by 1 gives many nice human-oriented properties, like for instance that indexing the front and back is symmetrical (first is 1 and last is -1), whereas in 0-based languages you have 0 and -1 which makes a lot less sense (people who haven't been bent to the will of the C language gods don't think of the last element as one before the first one, it's all the way at the end) except as a conceit to the fact that there's no -0 available. It's easier to reason about a list of five things where the first element is the first and the fifth element is the fifth, instead of constantly having to do mental -1 arithmetic to satisfy a paradigm that requires visualizing hardware memory layouts.
And if you really really really want to cycle over an array using modulus operations, the classic "0 is better" example that people love to trot out, Lua also lets you start your array at 0 if you want. Lua doesn't care what you use for table keys as long as they're not nil.
> the syntax is super weird
Is it? I teach Lua to people, and most novice programmers I meet find the syntax quite intuitive with its plain english keywords and operators. Do you think "function" is more weird than "def" or "sub", "or" more weird than "||", "not" more weird than "!"?
Elm and Rust have great error messages.
So instead of "no languages" let's instead say "very few languages, and none of the top 8 languages according to TIOBE". (Rust is now #9 as of like just now when it was #18 a year ago, so mad props to them for breaking into the top 10!)
What's nice about Python's message is how compositional or "orthogonal" it is, which means that the errors can easily be handled in a program. Does Lua have anything like that? From the manual
> If you need to handle errors in Lua, you should use the pcall function (protected call) to encapsulate your code.
This actually helps me understand a bit the dichotomy between "configuration" and "program", and why Lua is so good for the former. As another comment put it, it's good when you need to "describe stuff". It doesn't make a lot of sense to describe something that handles errors in its own description, which motivates relegating error handling to specific contexts.
Any function can error in any way imaginable, but if we're talking about built-ins...
`print(nil.."a")` -> "attempt to concatenate a nil value"
`print(({}).foo.bar)` -> "attempt to index a nil value (field 'foo')"
`table.insert(a, 1)` -> "bad argument #1 to 'insert' (table expected, got nil)"
On the Python side you get:
`None[1]` -> "TypeError: 'NoneType' object is not subscriptable"
`str.split(None, "d")` -> "TypeError: descriptor 'split' for 'str' objects doesn't apply to a 'NoneType' object"
... and in terms of why Blizzard used it in WoW... One of the strengths of Lua is it's so small that its interpreter is extremely easy to understand and debug, while still having just enough power to make most things one wants to do not super irritating. That's the technical-space aspect that boosted its social-space aspect... It's not so bad to use that nobody wants to touch it, but it's so simple to integrate into an existing complicated system that developers aren't pulling their hair out building the integration layer to whatever they're really writing the system in.
The Lua implementation was definitely its strong point, and LuaJIT is reportedly extremely good as well. I say “was” because the circumstances have changed and there are much better alternatives around.
Circa 2000 or so, if you wanted to stick a simple, self-contained language in your app for extensions and scripting, Lua was one of the very few reasonable choices. There are a ton of reasons why you might not want to stick TCL, Guile, or Python inside your application.
So Lua was used not only in the WoW UI but in a bunch of other apps from that era, and gained a lot of social traction that way. Among other things, if you were making a game, Lua let your level designers and UX people write a little bit of code without having to get their hands dirty with C++ (which nobody wants). The language is very off-putting otherwise.
The two alternatives at the top of my list are Gravity and Wren. They are both designed for the same general profile that Lua has—a scripting language, safe to use, embeddable, with a small VM (low code size).
- https://github.com/marcobambini/gravity
The language design choices are nice and familiar to people who are used to other existing languages. Lua is a bit radical.
Two other options are AngelScript and Squirrel, which are both a bit older and more mature than Gravity and Wren.
- http://www.angelcode.com/angelscript/
Finally, it’s much more feasible these days to embed something like Mono, and Guile has gotten a lot better.
That sounds quite editorial - can you list some of the “ton of reasons”? C. 2000 Tcl was already 12 years old, received a major performance upgrade w 8.4[0], and was liberally licensed.
[0] https://www.linux.com/news/tcltk-840-release-announcement/
TCL is a bad choice for almost anything. It is only marginally more usable than using a shell. It seems like the only reason you would use TCL is so you could throw together a GUI in front of a command-line program, or maybe stick a very bare-bones scripting interface inside something else.
I think TCL gets a really bad rap. Once you get past the unusual syntax, it's pretty nice to work in. Almost Python level nice. It supports classes, exceptions, first-class functions, and has some concepts I've never seen in other languages (such as upvar). It's easy to embed, or easy to use standalone. Installation is small and portable. And it feels like if Lisp and Bash had a baby together, conceived with love.
TCL in 2005 might have been a beast. TCL in 2020 is way better than you might expect. And it persists also, with a stable ecosystem as an embedded language in various CAD/EDA tools.
.. is for concatenating strings
~= is not equal to
I like your theory of unpopularity, that statement is always true in some capacity for esoteric things, liked this are also liked because they are popular. My favorite posts are the ones that are like yours BUT they continue to use it because it is the best tool for the job. Thats when you know it is true.
Prototype inheritance is kind of nifty but I would really like an escape hatch to be able to avoid using it, mostly for wrapping foreign objects.
That's like complaining about braces vs begin/end. It's just a non-issue, has been standard on other popular languages, and it's just a convention to get used to.
>the syntax is super weird.
How so? Seems extremely normal...
So, you might think Lua can't compete with Python in education. But we have an ace in the hole: concurrency. Lua can be a great sandbox for learning concurrent programming, with built-in coroutines and a reentrant interpreter on top of an extremely simple semantics and basic type system.
The problem is that nobody is going to want to use an ad hoc combination of Lua and some questionably maintained 3P libraries to teach a course. If Lua is to be useful for teaching parallel programming, it needs a built-in or at minimum endorsed solution for parallelism. Penlight, probably the most popular "Lua batteries" collection, includes almost everything except parallelism!
Another issue in education, though, is 1-based arrays. They're a minor annoyance in practice, but learning to count from zero is important in many CS courses.
By the way, for formatting Lua tables for debugging I like using "inspect". (It's not part of the standard library but oh well, that's the whole topic of today's discussion).
(As for formatting tables, presently most of my Lua code has been written for mpv which has a builtin utils.to_string(v) for this purpose. In the manpage, this is documented as "only needed because Lua is terrible." I think that's a bit too far, but the frustration this must have caused really comes across.)
I have to say, once I learned to appreciate the combo of Lua's builtin pattern matching (no not _that_ kind of pattern matching) and LPEG, I really loathe having to use cryptic regexp's in languages like JS.
Patterns are enough for most stuff, and when you want to get into more complex patterns, a PEG (LPEG in particular) is much more readable and expressive than a regex that would come out looking like a generated password.
Lua is also hands down the easiest programming language to embed. There's just no comparison. LuaJIT's C FFI also makes embedded work ludicrous.
https://www.cs.utexas.edu/users/EWD/transcriptions/EWD08xx/E...
The starting arrays from one thing, while not that big of a deal, can be a bit annoying, especially if you're using lua and c together. You've just always kind of got to be aware of it.
Tables and metatables are great, until they're not.
Lack of true integers(at least when I used it) made at least one task I tried to use lua for unfeasible and I had to use a different language in the end.
In the end though, lua's still probably my favourite scripting language to write in.
http://hedgewars.org/kb/LuaRules - I see someone wrote some hopefully correct guide on that last point.
I think LuaJIT being mostly stuck with Lua 5.1 is a cause for the lack up uptake on more modern versions of Lua.
[1] Mike Pall, creator of LuaJIT, disagreed with the direction that Lua 5.2 went and remained with Lua 5.1 semantics.
[2] I personally found Lua 5.2 a "meh" version and didn't bother upgrading.
[3] Including some functions to pack and unpack integers and floats into strings, in both big-endian and little-endian formats.
Personally I think it’s very capable language. It’s such a comfortable scripting language that I think nearly anybody could pick it up and get up to speed pretty quickly. LuaJIT is also ridiculously powerful and so satisfyingly stable (Lua 5.1 for over a decade).
Heck Lua 5.1 even makes an excellent transpile target so you might not even need to use Lua to appreciate the software that runs Lua.
> "Everyone has different needs so the places where serious money is getting poured into using Lua don't really have much of an incentive to standardize -- it wouldn't give them any benefit."
I really don't think that's true. It _is_ true that a standard library doesn't cater to single developers' or companies specific, custom needs and performance tradeoffs. But that's only true for limited parts of the code. Most of an application's code is not the critical "secret sauce", and typically, we are well served with standard library facilities. We simply don't use the small part where we need something different.
It's like that for basically any high-level language, I believe; and standardization is desirable and appreciated. Are we really to believe that most big LUA users need their own regex library for LUA? Their own sqlite library? etc.
In my extremely limited experience with Lua, the only thing that I definitely didn't like was its weak-ish dynamic typing, so I'd been keeping my eye on tl[1] and wu-lan[2] which both transpile to Lua for a while.
One of my use case is use Lua as an OAuth proxy, run inside Nginx, or use Lua as a S3 Proxy(with sign signature etc) and dealing with XML/JSON/HTTP Request is hard in Lua.
When I installed an external lua rocks, for it to be embeded inside Nginx, I have to make sure lua lib path are sync between luarock/nginx.
I don't know how to solve that problem other than reading the document. But explains all of them to a new comer to Lua/Nginx is hard. It takes a lot of time until they can write code.
Out of several dozen alternatives (the other scripting languages from a list on Wikipedia), my analysis is that Lua offers a very compelling and attractive technical profile.
I haven't been using it long, but so far I haven't seen evidence otherwise. Prior to Lua I would have used Bash, or maybe Python in a pinch.
> "an official list of approved Lua libraries", which could be searched and that would provide information on the strengths and weaknesses of the library
Perhaps something to emulate could be https://djangopackages.org/ which has repeatedly helped me with "how do I do X" questions in the context of Django web projects.
When embedding a language in another system, you have to worry about the abstractions the standard library enforces upon the rest of your system if those abstractions are deep. Far better, for our purposes, to use a language with an absolute handful of primitive types and build abstractions atop it.
I personally like lua because it's conceptually very simple. It has just eight basic types and that's all. Also it does not dictate you how to use it. It provides mechanisms, not policies.
There's a significant mental lift for me to be productive in Lua, there are tons of weird gotchas, and I find it to be super verbose. Even basic stuff like parsing JSON I end up having to go rummage around for support for.
These days, I'll use javascript or python for my smaller needs, and I'd much rather have a JS runtime embedded than a Lua one for any game scripting I'm doing.
First and foremost there is no easy guide how to even get started. Do you need MSYS2? TDM-GCC? Should I use WSL? Are there any precombiled binaries? Is there an editor?
Then by sheer luck I found the ZeroBrane Editor [1] and was quite excited. Simple installation, everything included, autocomplete and debugging! Awesome.
The first thing I had to do was trimming a string. Oh Boy! [2] I mean really.. As a beginner, this completely wrecked me. I have a task to do, why should I worry about which Trim implementation is best?
Switched to Nim (which I had not used before either) and never looked back.