So You Want to Ship a Command-Line Tool for macOS (2022)
becca.ooo
becca.ooo
If this is aimed for developers, going through all of that to provide an installable and clickable thing feels a bit unnecessary...
Alternatively, since you're already using nixpkgs, that contains some wrappers and examples for signing Darwin binaries. May be easier than doing it from scratch.
Because I don't want to install some tools on my PC just to build _your tool_. A packaged application is less complicated for the final user to use it.
In this, Windows and some Linux distribution have managed to create a good experience (MSI, AppImage, Flatpack)
Developers suffer what they must, while the user suffers what they will.
And the Mac has equivalents such as the standard installer, DMG images etc.
You’re conflating packaging with building. They’re different steps. You package for distribution after building.
Yes, yes, you can have a package manager build the tool and whatnot, but that’s not the case the article is discussing.
"""So, what can we do with all of this? We have a few options, but none of them are particularly appealing. ...
Download the file with curl. Unix command-line tools like curl and tar don’t add the quarantine bit to the files they create on macOS, so it’s pretty easy to run software downloaded with them.
Unfortunately, the repository where this tool is developed is private, so unauthenticated curl downloads won’t work. I’m seriously considering asking the CTO if we can make the repository public solely so we can use curl as a distribution mechanism. I could even write a platform-detecting shell script! """
The "curl ... | sh" technique could get pass through the authentication information for a subsequent curl call, but then authentication becomes a facade.
Have you thought about publishing the install.sh and final zip file through GitHub pages? Or Cloudflare pages? Then the repo stays private, but the built application can be accessed publicly, and behind a hard-to-guess url.
> in an internal, trusted environment, why not post a "curl ... | sh" command?
It's internal code, so we can't use a public (e.g.) GitHub URL, and developers running this tool may not yet have SSH keys or the VPN set up yet.
> point them at the master/main.zip of the repo
Basically the same issue -- developers would have to install a Rust toolchain and compile the code themselves.
> Alternatively, since you're already using nixpkgs, that contains some wrappers and examples for signing Darwin binaries.
It would! But this tool is what developers use to install and configure Nix in the first place, so it's not a great fit here. (Although we do have it packaged in Nix for users that already have Nix installed.)
Why? The current script installs Rust for them, so the downloaded repo snapshot can do the same thing. Whatever is distributed in the signed package can be in that zip file instead.
Is it WSL2 ? Because you can distribute Mac CLI apps exactly the same way.
The issue here is with the code-signing process which is optional but recommended just like on Windows.
Do they? Between Homebrew, MacPorts, the hundreds of thousands of Mac App Store apps, indie dev apps distributed outside MAS, not to mention all major proprietary apps like Abobe Creative Suite, MS Office, Pro Tools, and so on, probably more software devs steer around Linux than Apple..
> I've been through the same hell and there's basically just one way to publish software for macOS with no hassle for the user: Completely lock yourself in their ecosystem. That means working on a Mac, building in Xcode using Swift and publishing in the official app store.
Your highlighted portions are false.
All those other devs, including tens of thousands of MacPorts and Homebrew ports, manage to not dump any hassle to the user...
To be honest though, if your making a cli tool, you’re requiring the command line. And anyone comfortable with the command line can install homebrew
I build a cross-platform app using Tauri and have never had to touch Xcode, Swift or the App Store.
And if you want to a similar experience to Linux then just package your app using homebrew.
In which case, they should do the same thing other platforms do: user scripts.
Much like bat files on windows, macOS allows writing AppleScript which would sidestep their issues completely, or command files for shell scripts.
Otherwise they don’t need to sign command line tools. macOS doesn’t block tools run in the terminal to the best of my knowledge.
If they want to make an app, that also doubles as a command line tool, it’s pretty simple as well. You can check for command line options when running the main executable. I’ve done this several times. Apps aren’t magic.
They can self sign that app, and their end users can bypass the unknown developer warning, or just launch the command line part.
But again, just use AppleScript or command files.
This is incorrect. You do need to sign command-line tools as otherwise they are blocked from running even in a terminal, at least on Apple Silicon.
xattr -d com.apple.quarantine Foo
The author of the post even mentions this BUT they’re operating in a very weird space where they want to make a command line tool but don’t want people to use a terminal to access it.
To me it’s a nonsensical issue. I suspect the author got flustered with a set of ideals and didn’t take the necessary step back to reevaluate their situation
it's sad that we need third party tool to be able to codesign and notarize effectively (and from other OSes and CIs), but it works now
but maybe I miss something that the author needed and I didn't. The hardest part was cross-compiling for macOS but we did it too with these images from elasticsearch
There is a class of macOS applications which can perform such functions and they're called Packages or Distributions (it's the wizard you see sometimes when an app needs to make changes to the system before you can use it). That's the way to deliver such applications from a downloadable source. Packages can be signed and notarised and none of the crazy workarounds listed in the post will be necessary.
Plasma looks pretty good but is not nowhere as good as window's.
I heard that they fixed power management but haven't tried myself.
My last point is that hyper-v got soo good with GPU partitioning and can be fully managed using type-safe Powershell.
I don't think this is true, actually. I've been notarizing command-line builds of unxip by shoving it in a zip and uploading it to notarytool for a while: https://github.com/saagarjha/unxip/blob/main/release.sh. (You will note, amusingly, that I am suffering from the same -parse-as-library bug as the author.) After this is done I just chuck the zip file and upload the binary directly to GitHub. If you download the file from your browser and chmod +x it, you can double click on it and it runs. Obviously you can skip the chmod +x step by sending people the zip (which preserves permissions inside of itself).
Also, FWIW, Apple's tooling sucks but if you check Console while it evaluates the code sometimes it will tell you why it doesn't like it. Most of the evaluation code is open source, too, so you can look up log messages and errors there: https://github.com/apple-oss-distributions/Security
In most cases that isn't an issue, though it won't work if your internet connection is down or if you're blocking syspolicyd to keep if from phoning home to Cupertino. ;-)
Incidentally, the other day I discovered that the Arc web browser neglected to staple the notarization ticket to their app.
Shipping a command-line tool for developers is easy. Put it on Homebrew or NPM or something, or just give them a URL for curl.
macOS is Unix, it's just that the Unix bits are carefully hidden away from most users.
Or MacPorts, which was created by Apple employee and inventor of the FreeBSD ports system Jordan Hubbard, has more packages than Homebrew, and has a much more correct design. Homebrew is a bit of a joke in comparison, not sure why it gets all the attention.
One possible minor advantage of Homebrew is that it works on both macOS and Linux. I recently found this very handy for setting up a project with developers on Macs and CI on Linux. Almost no OS-specific build steps were needed.
A couple more points, from a quick scan of the respective docs:
"Homebrew won’t install files outside its prefix and you can place a Homebrew installation wherever you like." The MacPorts docs are much less clear, I can't tell if it can do that.
Homebrew gives "brew install --cask firefox" as an example of installing a GUI app. MacPorts doesn't seem to have Firefox.
I'm curious what makes MacPorts "much more correct", if anyone wants to clarify that.
Where did this coddled sense of entitlement come from? Are they creating their own SSH keys? Do they know the basics of the git cli? Or are these "magic commands" too? Do they know how to do their job?
I sympathise that Apple's tooling could be better, but the conclusion and tone is ridiculous given what could charitably be described as 'unique' requirements.
Our internal CLI tools have all largely been hassle free regardless of devs OS. We do use GoReleaser which abstracts away the multi arch builds, signing, and publishing to GitHub and Homebrew, which can be criticised sometimes as a bit of a crutch. But it's also been zero hassle for all of our Mac devs (the most junior of which are still expected to be able to download and execute a binary from a terminal.) and lets us get on with our day without prioritising 'requirements' that aren't even worth spending any time on.
Why would you even ship a command-line tool if your apparent need is to avoid the command line?
Windows isn’t a cakewalk either (you now have to use a hardware security module to store your certs, which means USB stick based signing or using a cloud service that supports HSMs, plus certs themselves are a racket in terms of pricing).
- “Windows protected your PC” with some tiny blurb no-one seems to read and a single button labelled “Don’t run”.
- If you do read the tiny blurb and spot the “More info” link, you can click it to get a summary about Windows Defender SmartScreen having saved the day by protecting you from this app by an “unknown publisher”, and then a second button becomes available that says, “Run anyway”.
It’s pretty hostile and it effectively forces devs to throw 300-600usd/year at a cert provider plus spend time to configure signing via CI.
There are also two levels of certificate, and the better one is only available if you’ve been in business for several years with an office address and public phone number (cert providers need to call you at your public number to validate your business ID).
There are some ways around it (I think games distributed via Steam’s installer still don’t need to be signed) but those aren’t an option for many.
Distributing a command-line tool for Windows is a complete nightmare in my experience, far more frustrating than on a Mac.
No npm or docker needed.
For some reason they want to distribute this command line script as a double-click installer which doesn't need to be signed either. Users will simply need to override the "this is untrusted and you shouldn't open this" dialog.
The user gets a double-click “installer”, and like you said, you don’t even need to sign it.
> There’s a long guide on Embedding a Command-Line Tool in a Sandboxed App, so I followed that, and then slowly, painfully, factored Xcode out of it, so that I wouldn’t have to figure out how to get a 10GB Xcode install onto the CI machine (remember, you need to be signed in to an Apple ID to download Xcode, and there’s no way to do it from the command-line).
You could actually solve this with Rust and no Xcode whatsoever. cacao [1] and cargo-bundle [2] will produce an app bundle you could sign/notarize/distribute without needing to ever open Xcode.
[1] https://github.com/ryanmcgrath/cacao [2] https://github.com/burtonageo/cargo-bundle
You wouldn't need to tangle with Swift, would only need enough Rust to hoist whatever script you're trying to run.
In general though I find the article a bit odd - there are also quite a few examples of signing/notarizing/etc for GitHub Actions out there already, I went through this exercise a week ago and it wasn't that bad.
Edit: Hell, you could also potentially just bundle your scripts up into a `.pkg` installer
The distribution mechanism was to build from source[]. Once compiled, all other dev tasks were handled by the tool.
If you compile it yourself you don't run into any of these issues. This approach is only viable for developer tools of course, but that's what the post is about.
[]: This was a Go project, so building from source was relatively straight forward.
Make a pubic repo that is just a facade, generates an ssh key, and calls GitHub (or whatever) to push the ssh key.
curl -H "Authorization: token **********" --data '{"title":"test-key","key":"${key[...]}"}' https://api.github.com/user/keys
On success you can then call the private repo and execute the cli installer. If the bootstrap is piped then it doesn't ever have to be installed, and you don't have it junking up your system.
Extra points to pull down existing registed public keys and check if they are already there.
in any case I think there are plenty of apps that do this. Some are distributed as packages instead of apps.
I'm not a Mac dev expert at all but I have used enough apps to know it's possible. Many of those apps are open source maybe look into them if you want to solve this issue.
Electron apps have tools for automatically signing on GitHub actions so looking though what they do might help?
sorry I don't have concrete suggestions. I'm just confident there are solutions.
I'm totally sympathetic to the fact they it's a pain in the ass and Apple, probably intentionally, doesn't document the process well or at all
These only sign/notarise the dmg or app bundles (or the binaries inside them), so I don’t think that will help the OP unless they’re willing to wrap their CLI app in an Electron app, unfortunately.
Xcode-select —install
git clone…
make
...from what I'm seeing on macOS, downloading and running a cmdline tool (built with the Xcode cmdline toolchain on GH Actions) inside the terminal still works without all the hassle described in the article - of course I wouldn't be surprised if that stopped working too one day :/
PS: I kinda wish that operating systems had builtin WASI runners for this kind of bootstrapping work.
Perhaps the nix packages configuration could reference a script with everything properly debugged and then the snake can eat its own tail.
It seems like a provisioner to install and configure stuff, in which case one obvious method to install is copy pasting bash code. Have we really reached an era where folks expect to use CLI tools without using a CLI?
Can nixpkgs install from brew? Maybe that’s the answer, use the annoying stuff and wrap it with something to make it less annoying. Or, you could wrap your CLI with Tauri and install with Cargo. Might be more robust but not in the comfort zone of a nix shop.
Also, while this only sometimes helps, it is worth looking for more error information in dmesg as that is where verification errors go.
Yeah, I'm starting to understand that more and more. I'm not a big fan of the PWA SPA-approach, but given what's going on I'm going to have to change my tune.
Selecting the "Add to Dock..." option in Safari for YouTube Music has been kind of eye-opening. It works really well.
I'm not sure what they're thinking here. Are native desktop apps done and over? It kind of seems like that's what they want based on behavior.
Wait I'm confused by this part (never used MacOS).
Which part of the chain mechanism mentioned above can be bypassed by just downloading the file via CURL?
I didn't seem to see the author mentioned (not explicitly at least) which steps added these "quarantine bits".
But if this blog post gets Apple to improve their code signing process, then it does have value :)
But I think they have a really odd set of requirements in that they don’t want their devs to run a CLI tool from the terminal, but want to run it from Finder?
1. New team member gets new fresh Mac 2. New team member downloads the tool and runs it 3. New fresh Mac is now set up with a development environment suitable for the company, including a working Homebrew installation (or Nix or whatever)
go run github com/mikefarah/yq/v3@latest
Assuming go is installed this works on all common platforms and architectures.I wonder if a stub program that packages go with something like cosmopolitan could make it a true universal cli
I think you should be able to use the new-style scoped GitHub access tokens to generate a token that'll allow access to the one resource.
https://example.com/aeluaeualeraciuwefaljwefnalsduhgsdgasu.s...
Rotate the URL from time to time.
Sadly I can't find it in a zounds of articles containing macOS and build