Show HN: Backstage – Three-speed scripting language and task automation tool
github.com
github.com
I'm Alex, a tech enthusiast who likes to build stuff from scratch.
The more time we spend doing something, the more comfortable we feel and the less dependent we are on an external learning resource. Since I spend a lot of time in a text editor, I can spend days writing Python code without checking an online resource. On the other hand, since I spend so little time in the command-line shell, I often found myself looking online for commands to build and publish Python packages.
Considering the text editor and files as the software development stage [1], the backstage [2] would be the tasks performed in the background to manage the project. These tasks would relate to the different stages of the project lifecycle (build, test, publish, et cetera).
I thought it would be both cool and useful to have a tool that would serve as the backstage entry point for any of my projects. This tool would be called 'Backstage' [3] and would allow me to quickly discover the available tasks (including their descriptions) for a given project. Tasks would be defined in a file named 'backstage.tasks' (preferably located at the root of the target project) and would consist of collections of commands to be executed sequentially. To execute a task, I would just have to type in the command line its name possibly followed by arguments. For example: 'backstage build' or 'backstage release 1.0'.
INI File [4] or TOML would be the perfect file format candidate for the 'backstage.tasks' file. But unfortunately they are designed to handle sections with key-value pairs when I need a simple way to define sequences of commands aligned under their respective task name. So I built Jesth (Just Extract Sections Then Hack) [5], a new file format that looks like INI file but lets me decide how to interpret the body of a given section.
After a few days of programming, I successfully built the first iteration of Backstage and placed a 'backstage.tasks' file in my projects directory. From that day on, releasing a new version of a project was stress-free.
This first iteration served its purpose as I expected until one day I forgot to enable internet connection while using Backstage to publish a new version of a project. Project documentation was generated successfully, version management, build and 'Git Commit' were done automatically, but 'Git Push' failed and the task was aborted. This specific task was supposed to run without interruption because each time it is executed, unique and timestamped events occur. Enabling internet and re-running this task would update the target project version and create a new superfluous version.
From that day I understood that I had to improve the syntax of the 'backstage.tasks' file to avoid this kind of annoying situations in the future. The most obvious solution was to introduce variables and control flow. I took the time to anticipate what I will need in the future. Then I decided to add stuff like data types, namespaces, multithreading, test support, builtin commands, et cetera. Eventually, the syntax of the 'backstage.tasks' file evolved into an eponymous (Backstage) three-speed scripting language.
The three-speed scripting language concept is inspired from the three forward gear ratios of early automobiles transmission system [6].
In first gear, a `backstage.tasks` file is intended to store a list of tasks related to a specific project, each task exposing a list of commands or subtasks to be executed. Here, a command represents a process or pipeline of processes to be spawned. Environment variables can be used in commands via variable interpolation [7]. No other logic is involved.
In second gear, the `backstage.tasks` file not only stores the tasks like in first gear, but here logic intervenes, variables are defined, control flow is used, built-in commands are called, et cetera. Basically, in second gear, Backstage unleashes its power and allows the programmer to anticipate problems, make sophisticated combinations of subtasks, in short to write a real script to automate things.
In third gear, in addition with whatever can be done with previous gears, the programmer can directly from the `backstage.tasks` file, call Python functions with arguments and get the return ! Thanks to this third gear, any too complex or overly verbose calculation can be written in Python and called from Backstage. This functionality alone proves that Backstage is all about making the programmer's life better, not pretending to replace existing mature solutions that actually work.
These three speeds make Backstage versatile enough to act either as a simple task runner or as a scripting language with a built-in bridge to the powerful Python ecosystem. Not only are tasks defined in 'backstage.tasks', but their documentation and tests can also be embedded there. The command line interface allows the user to discover available tasks, read their documentation, use a glob-like [8] syntax to search for a task by its name or by a keyword that is part of the documentation of the task.
I use Backstage a lot and want it to stay cross-platform and relevant long term. So, as said before, Backstage has data types, namespaces, multithreading, test support, built-in commands but also regular expression support, timestamp and date/time conversions, support for here document [9] directly in 'backstage.tasks' file, automatic normalization of path strings, intuitive variable persistence mechanism, et cetera.
Backstage allows the programmer to spawn and pipe processes, capture their output, push data into a process's input, redirect STDOUT and STDERR. File and directory manipulation is intuitive, so one can create, move or copy files or directories with built-in commands, browse an arbitrary directory and access properties such as creation time or size, consume the contents of a large file line by line or in bulk.
There is more to say, it's a work in progress and I'm excited to share this project with you. Backstage is available on PyPI and you can start playing [10] with it from now. You are welcome to share what you think about this project.
[1] https://en.wikipedia.org/wiki/Stage_(theatre)
[2] https://en.wikipedia.org/wiki/Parts_of_a_theatre
[3] https://github.com/pyrustic/backstage
[4] https://en.wikipedia.org/wiki/INI_file
[5] https://github.com/pyrustic/jesth
[6] https://en.wikipedia.org/wiki/Manual_transmission
[7] https://en.wikipedia.org/wiki/String_interpolation
[8] https://en.wikipedia.org/wiki/Glob_(programming)