Show HN: Automation the KISS way. No YAML involved

Keep working on this. It’s a splendid idea. Would be cool to eventually have a recipe where you import the crate, then program the deploy in rust, invoking your tool which builds my-build.rs, or a proper cargo build.

The following not true apples to apples, but pretty close. Rather than another crummy stringy DSL, the ocaml community said “we want to program our CI pipelines in OCaml”. So, they created https://github.com/ocurrent/ocurrent. You build your pipeline in an incredible language—then the CI server simply invokes your pipeline code, assuming you’ve implemented a basic interface.

Thank you for your feedback! :)

> Would be cool to eventually have a recipe where you import the crate, then program the deploy in rust

This is actually one of the goals and already somewhat possible since v0.5.0 (i'm at v0.8.0 now, working on the test suite to reach 1.0.0) if you don't mind writing shell commands as strings in Rust :P

The currently available tasks are:

  - info: to get what's in the inventory (and later infos about the remote host)
  - exec: to run a (templated) shell command
  - upload: to upload files or render templates
  - download: to download files

With this, you can already upload archives, extract them, create systemd files, start the service, etc...

It's a bit more manual than with ansible/puppet/... but it works :)

I think, at the moment, a more correct comparison would be with Fabric[0] in Python. I have some ideas for post-1.0.0 relying on TCP/IP tunneling through the SSH session (docker builds on remotes, querying remote services hiding behind an SSH bastion, ...), I'd love to hear more ideas though :)

  [0] - https://www.fabfile.org/

I couldn't agree more with you about Ansible... I've started using Ansible in two projects, and I think it was enough for me... Ansible is perhaps great when you have granular tasks, that maybe don't need to communicate between them, or perhaps that don't have too much logic (or worse loops). However, as soon as your Ansible "scripts" start becoming actual "scripts", everything gets out of hand... (Also the performance is terrible due to the fact that each task is "bundled" on the host, copied remotely, extracted, executed, deleted, repeat for each and every instance of the same task...)

----

Now about your `tricorder`, by looking at the readme, I gather it just allows one to run the same command over multiple hosts (filtered by name or tag), but nothing beyond that? How could one run for example a pipeline of tools on the remote hosts? (I assume `bash -c 'tool-1 | tool-2'`, but with careful escaping?)

Perhaps a first step towards actually being an "Ansible replacement" would be to bring back in some "scripting" ability. For example instead of giving each command as arguments, there could be a "library of scripts", and the user can choose one of those to run on each remote host. Then those scripts could be written in whatever language the user chooses (`bash`, Python, Ruby, etc.)

Also, with regard to the "library of scripts", one thing that Ansible got right is the possibility to put multiple different tasks in the same file, thus one doesn't need 20 files for 20 small tasks, but instead these can be put inside the same YAML file.

----

For example this is how I did it in my own "Ansible replacement", <https://github.com/volution/z-run>, like for example, one can write something like these snippets: <https://github.com/volution/z-run/blob/development/examples/...> (see `ssh / launch / simple` that runs locally which just calls `z-run ssh "${_target}" ':: ssh / remote / main' "${@}"`, which is in itself a non trivial `bash` script, one that will run remotely.)

In the end, what I think we need, is a tool to easily run "scripts" on a remote system, but make it as easy to use as it would be on the local machine. (Sort of like the RPC but across SSH and for scripts.)

At first look, your z-run remind me a helper[1] I use sometimes.

For massive remote executions, I often use dssh[2] program or similar.

I think that tricorder belongs to that later category and isn't a true library of scripts dispatcher.

[1] https://github.com/lucaswerkmeister/cheats

[2] http://sigpipe.io/dssh

Thank you for your feedback.

> I gather it just allows one to run the same command over multiple hosts (filtered by name or tag), but nothing beyond that?

For now. I intend to carefully extend it to other features, like in said in the original post:

- mimic the "gather facts" feature from ansible so you can query it with `jq` - provide an API in a few programming languages so that you can choose whatever fits your needs to "script" the logic (and with Rust/Go, distributing a single binary as a 'setup.exe' should be plenty enough) - probably a plugin system similar to kubectl (as in subcommands look in PATH for a binary named `tricorder-subcommand`)

I'd like to see it becoming a real replacement for YAML based workflows, but I started small because I don't want to repeat the same mistakes. I'd like to see how people would use this before building on it.

> For example instead of giving each command as arguments, there could be a "library of scripts", and the user can choose one of those to run on each remote host. Then those scripts could be written in whatever language the user chooses (`bash`, Python, Ruby, etc.)

For running scripts from a library, I've used ByJG's automate.sh[1] that call'em recipes. Unlike z-run, they're bash only and there's no select menu and looks simpler.

[1] https://github.com/byjg/automate

This is interesting. Is the tag mode an or match or an and match? Ie does “server,backend” in the example match those with both tags or either tag?

Thank you for the feedback. This is an OR, if either tag is present, it will be selected.

I'm not set on this, maybe I will add a simple parser for something like `(A|B)&(~C)`. What do you think ?

I think the ability to target more restrictively would be a fantastic addition, largely because I could imagine the tags being used in a way the example doesn't cover: DC/AZ/Region tags.

I don't know that a full parser for Boolean logic is necessary - even having a complimentary `-T` option that takes mandatory tag names might be enough.

I'm curious about the ability to upload stuff too. Right now I mostly use a tool I wrote in shell several years ago (unimaginatively called `remote-install.sh`) that basically tars up a directory (minus exclude rules), sends it to the remote host, untars it to a tmp dir, and then runs the specified command (via sudo if specified).

But it's very much a one-at-a-time process.

Meanwhile a deployment helper script I wrote is very much a multiple-at-once process, and so it's multiple background SSH processes.

I could see both of these likely being candidates for using tricorder.

One interesting thing I don't see on your roadmap would be some way to have per-host data sent (i.e. a way to perhaps set some args to a command/script on a per-host basis, or perhaps a way to define environment variables to set per-host, and the script invocation could reference those)... Maybe this is part of the 'send data' stage, and it just has the ability to substitute a few vars when determining what so send. So e.g. `--send-file "%id%.env" --send-file "script.sh --send-file "data.foo"` would interpolate the `%id%` substring with the target's ID from the TOML file, allowing to send a customised file per-host.. Anyway. Just a thought!

> largely because I could imagine the tags being used in a way the example doesn't cover: DC/AZ/Region tags.

I'll add this feature to the roadmap then :)

> I don't know that a full parser for Boolean logic is necessary - even having a complimentary `-T` option that takes mandatory tag names might be enough.

I truly dislike command lines with hundreds of arguments. IMHO, A boolean logic would be clearer on the intent than a combination of -t -T -not-T etc...

> so it's multiple background SSH processes.

I'm not sure yet if parallelization should be the default or not. Also I've not yet learned Tokio and Async programming in Rust :P

> some way to have per-host data sent

Yes, just like Ansible evaluates very variable per host. At the moment you can do it manually:

  res=$(tricorder -i inventory -- echo "available")
  host_ids=$(echo $res | jq "some jq black magic")

  for host_id in $host_ids
  do
    tricorder -i inventory -H $host_id -- echo "specific data sent to $host_id"
  done

Maybe i'll make use of the `tinytemplate` crate and add a -D option for data that would be similar to the inventory (TOML static file or program/script outputting JSON).

I'm not sure yet on how it should look.

I mean to keep tricorder KISS so it can be integrated easily in a bash script as a standalone command, and not a full-on folder structure like ansible playbooks.

> I truly dislike command lines with hundreds of arguments. IMHO, A boolean logic would be clearer on the intent than a combination of -t -T -not-T etc...

I get that. I guess its a case of where the complexity is, which is going to be subjective. One has potentially confusingly similar CLI options, but with simple value form for each option. The other has a single clear CLI option, but the value form it takes is more complex.

`tricorder -i foo -t 'RoleA&(DC1|DC2)'` or

`tricorder -i foo -t 'DC1,DC2' -T 'Role'`

Ultimately if the capability is there, that's a great positive IMO, regardless of how you choose to expose it.

> I'm not sure yet if parallelization should be the default or not. Also I've not yet learned Tokio and Async programming in Rust :P

Right, I hadn't actually got to looking into how tricorder handles calls to multiple nodes - if it's not concurrent it actually wouldn't be able to replace the deployment helper I mentioned, but that doesn't mean it's bad or wrong. Every tool doesn't have have solve every problem.

> how tricorder handles calls to multiple nodes

Sequentially, one after the other, then it aggregates the `Result<(exit_code, output), Error>` type into a JSON.

Once I take a look at async programming in Rust, it should be trivial to add the concurrent option: just start them all, wait for them, and aggregate as before.

Still not sure if it should be the default or not, should it be a `--sequential` or a `--parallel` flag? IIRC, Ansible does things sequentially too.

My approach to stuff like this is basically semver: the product is out, so any new capability (e.g. parallel operation) can't change the default at least until a major version change.

This is true for post 1.0.0 versions. 0.Y.Z versions are experimental and the API has not been stabilized yet.

Ah right. I didn't even look at the current version number I must admit (and my own practice is that stuff starts at 1.0.0), so I didn't consider the pre-1.0 clause.

I always start at 0.1.0 until I figure things out :)

>> how tricorder handles calls to multiple nodes

> Sequentially, one after the other,

> […] Ansible does things sequentially too.

No, call to nodes aren't sequential, but by parallel batches where tasks are sequential. Here's an interesting reading on those aspects: https://ansiblefrombasics.blogspot.com/2017/09/serial-limit-...

> I think the ability to target more restrictively would be a fantastic addition, largely because I could imagine the tags being used in a way the example doesn't cover: DC/AZ/Region tags.

> I don't know that a full parser for Boolean logic is necessary - even having a complimentary `-T` option that takes mandatory tag names might be enough.

Here's how the targeted concurrent is doing it: https://docs.ansible.com/ansible/latest/user_guide/intro_pat...

Hey, just FYI, v0.8.0 is out with the tag expressions "serve & backend" etc... :)

On my way to the 1.0.0 by implementing the automated test suite!

Clickable link: https://github.com/linkdd/tricorder