I do agree it'd be nice with a more general purpose language and a lib like you say, but should this lib be implemented in rust/c so that people can easily integrate it into their own language?
Many unknowns but great idea.
In comparison, if I learned Github Actions syntax, the only thing I know is... Github actions syntax. Useless and isolated knowledge, which doesn't even transfer to other YAML-based systems because each has its own quirks.
Lua would be a good candidate for the latter, but its standard library is minimal on purpose, and that means a lot of the functionality would have to be provided by the build system. Interaction with the shell from Python is needlessly cumbersome (especially capturing stdout/stderr), so of those options my preference would be Ruby. Heck, even standard shell with a system-specific binary to call back to the build system would work.
Easy shelling - check.
Easily embeddable - check
Easily sandboxable - check.
Reasonably rich standard library - check.
High level abstractions - check.
If you're still guessing what language it is, it's Tcl. Good old Tcl.
It's just that is syntax is moderately weird and the documentation available for it is so ancient and creaky that you can sometimes see mummies through the cracks.
Tcl would pretty much solve all these pipeline issues, but it's not a cool language.
I really wish someone with a ton of money and backing would create "TypeTcl" on top of Tcl (à la Typescript and Javascript) and market it to hell and back, create brand new documentation for it, etc.
That said, you're right, at least at first blush, tcl is an attractive, if easy to forget, option.
They did. Larry McVoy of BitMover created Little, a typed extension of Tcl [0]. Didn't do the marketing bit, though.