Memory-safe, clean implementation of classic Posix "BC" calculator | Hacker News Reader

Memory-safe, clean implementation of classic Posix "BC" calculator

github.com

33 pointsjgarzik52 comments

Am I the only one who thinks we need less short-lived tools written in memory-safe languages an more system infrastructure written in said languages?

Don't take me wrong, I like when someone creates a better version of something existing. For example, ripgrep is fantastic. But seeing the recent trend of rewriting whatever in Rust just because, I can't help wondering why I should bother whether bc leaks or not...

Rust does not guarantee absence of memory leaks, does it?

I think this is the general "rewriting solves all problems" fallacy, which is rarely true, but who wants to maintain existing code, if there is something new and cool? So we get rewrites which largely just add new projects that then have to be maintained in parallel, because they rarely are to completely replace the old (if at all) and at some point the new is not cool anymore and that it is just another thing. I like some key ideas of Rust, but overall I still like many things about C much more (stability, fast compile times, simplicity), so I would much prefer people improving memory safety in C projects.

I like Zig's approach much more than Rust. When you get fixated on one aspect and sacrifice everything for that, you get things like Perl, Java or Rust.

So, massive success ?

Major adoption across the industry,being taught at university courses, bootcamps, shipping in commercial OSes?

Rust does not guarantee the absence of memory leaks, that's correct.

Maybe we are not seeing rewrites of large OSS projects in Rust yet (eg. Linux kernel, Blender, LibreOffice) because the language is not (yet) ready for projects of that scale and complexity.

What is the lack of a Rust equivalent to all those projects is because they already exist and work well, and Rust programmers aren't as mad to rewrite things as the stereotype suggests?

I disagree that they work well. A quick search of the CVE database will find many vulnerabilities that could have been prevented by rewriting in a memory safe language. Ideally, it should be done. However, rewriting those large projects would be a very difficult task, so it's probably not the best use of resources when there are still relatively easy security improvements available, such as implementing OpenBSD's pledge/unveil system.

Oversimplifying, there are two metrics: Bugs, and features. You're pointing out that the existing ones have bugs, parent comment is pointing out that the existing ones have features. Rewrites tend to have less of both.

> so it's probably not the best use of resources

Cool, so what useful software are rust programmers writing with all that free time? Where is it?

Where's the "killer app"? Whatever it is, nobody is talking about it.

There’s millions of lines of production Rust at tons of companies. It’s beyond one single killer app. Basically every major tech company relies on at least some Rust at this point.

I’d sleep better knowing the existing bc can’t write to ~/.profile than if it’s written in rust or not.

Those will take time, nobody will re-invent Linux, likely it will come with the OS systems knowledge advancements from the last 30 years.

Redox-OS is one such effort.

Most utils are 25+ years old. It is worth revisiting, dropping old cruft, entrenching UTF-8 support, and other areas.

Smaller and fresher.

Sounds like second system syndrome.

I don't think that's necessarily the case. Maybe the original version was designed around limitations and assumptions that made sense back then, which aren't there or true anymore. Maybe there were design decisions that turned out to be mistakes in hindsight but can't be dropped due to compatibility reasons.

Adding UTF-8 support in 2024 is hardly second system syndrome. Note that these re-implementations generally try to be compatible with the original rather than starting with a completely new vision, another sign that we're not suffering from SSS here.

> dropping old cruft, entrenching UTF-8 support, and other areas.

I couldn't possibly disagree. My point is that focusing on ownership, lifetime, etc. (the most prominent features of Rust) shouldn't be goals for a tool that's invoked to perform a few quick calculations and shut down immediately after.

> It is worth revisiting, dropping old cruft, entrenching UTF-8 support, and other areas.

I certainly won't say anything against supporting UTF-8 absolutely everywhere, but "dropping old cruft" is IME usually what people say right before dropping useful features or breaking platforms that are actively in use.

As the author of the default bc in macOS and FreeBSD, good luck.

Also, while mine is in C, I have tried very hard to eliminate all memory safety bugs, and I think I have done pretty well.

https://git.gavinhoward.com/gavin/bc/

> good luck.

Do you see this as an unexpectedly challenging task? If so, what do you think makes it challenging?

Not terribly challenging, but harder than expected.

The reason: parsing.

The bc language is poorly designed for a REPL, which it needs to handle.

I got sick of fixing parse bugs, though, and my source code got a sarcastic comment: https://git.gavinhoward.com/gavin/bc/src/commit/4b83bf07b97f...

Amazing work. Thanks and kudos. I definitely tested some early stuff vs your bc.

Lost as to why there is a tests folder yet the test function marked with the #[test] attribute is being added to /src: https://github.com/rustcoreutils/posixutils-rs/pull/132/comm... edit: Since the change was to the grammar saved in a separate .pest file, which is separated from the procedural code, then would that make the test an integration rather than a unit test as the "parse_program" function is public?

In Rust, unit tests are marked using the #[test] attribute and typically live alongside the code under test.

Integration tests live under a top-level “tests” folder - in this case, you’ll see that the “tests” directory contains test that call the CLI command and verify its output.

See: https://doc.rust-lang.org/book/ch11-03-test-organization.htm...

The `parse_program` function is public inside the `bc_util::parser` module, and the parser module is marked public inside `bc_util`, but in the `calc/src/bc.rs` file the `bc_util` mod isn't public, and can't be accessed from a test inside the `tests/` folder, which only has access to the public API exported by the library.

I am still waiting for Rust people to do something new and innovative. I acknowledge that Rust itself is innovative, don't get me wrong, But where are the true proof-of-concept projects such as the Linux kernel, qemu, git, postgresql, vim, etc. of the Rust world? Rewriting code is a waste of time of time, fragments the eco system, and introduces new bugs. And with a new focus on memory safety and analysis tools and sanitizers evolving, a lot of C code will be made memory safe in the future without needing a rewrite in Rust.

Aren't most of the standalone wasm runtimes written in rust?

No idea, but according to this site: https://github.com/appcypher/awesome-wasm-runtimes there a wasm runtimes written in all kinds of languages. This does not appear to be a very challenging thing to do, but I haven't look at this closely.

Writing bytecode runtimes is a common thing since 1958 (see UNCOL), only a novelty for those selling WASM as some great invention never done before.

Only this year I started seeing WebAssembly conference talks actually acknowledging the history of bytecode based distribution.

There's tons of stuff of the kind you're describing in Rust, more than I care to list but I'll offer you some [1-3] that I think would interest you (based on this comment alone). If your perception of what people are doing in Rust is driven by what people post to HN, it may not reflect everything that is going on in that ecosystem.

[1] https://os.phil-opp.com/

[2] https://gitlab.redox-os.org/redox-os/redox

[3] https://oxide.computer/

Fair enough, I may not have the full picture. But "writing an OS in Rust" sounds also seems the "innovative" part is that something is written in Rust. I do not need a new OS in written in a different language. I could use a good open-source CAD tool, or a good open-source FPGA tool chain (maybe those exist, I need to do some research). A language and projects written in it do not impress me if its own novelty is the selling point, but if the language can be used to solve problem I have. But yes, maybe those projects do exist somewhere...

You might be interested in https://mattferraro.dev/posts/cadmium

People are building cool stuff, and happen to do it in Rust.

They do, you may just not look very much into the ecosystem.

OSes - redox and many other smaller projects

git - pijul is novel, gitoxide is a pure-Rust git reimplementation

vim - Helix, based on vim and kakoune

>a lot of C code will be made memory safe in the future without needing a rewrite in Rust.

Sanitizers don't do what Rust does, and static analyzers cannot make C memory safe. You would have an easier time of bolting a borrow checker onto a subset of C++ than what you're thinking of, IMO.

Many Rust projects have done cool new stuff. Alacritty is possibly the greatest terminal emulator on the market today. I'm not into cryptocurrencies, but Parity and ZCash are doing cool stuff in that space thanks to Rust.

I'm also the author of Pijul, a much simpler and more scalable (yes, both! why choose?) version control system, and of Sanakirja, an on-disk transactional allocator to write persistent datastructures (like B trees, ropes, radix trees, HNSW…).

1) (new and innovative) and (safer, written in rust) are orthogonal attributes.

(2) Rewriting isn't necessarily a waste of time. There are rewrites of some wretchedly vulnerability-prone (and poorly extensible) core infra like ntpd, sudo, etc., that are sorely needed. Plus, rewriting them in a safer language will make it easier for future innovations, because it won't be as terrifying to try to have on them without introducing yet more vulnerabilities.

(3) Same applies to using rust for future components within the kernel and major browsers.

(4) Sanitizers are great, but they're not as strong as what you get from a memory-safe language. Most importantly, sanitizers only check where your tests cover.

(5) ripgrep. It's not a rewrite of grep, it's a rethink, and it's amazing.

- DC to convert between bases and if you like RPN.

- Calc it's better if you want to cover complex numbers: https://github.com/lcn2/calc

- Qalc from libqalculate in order to solve some equations in legacy machines. It's lighter and smaller than Maxima, but not as complete. - -

- Maxima for anything else, the biggie one. This will cover calculus. Gnuplot will work fine as a plotter for the previous tools.

Seems there are multiple Rust coreutils implementation projects? I have heard about uutils, but this is another one?

Quoting the repo:

> The goal is to create clean, race-free userland utilities that are POSIX compliant, maximizing compatibility with existing shell scripts while minimizing bloat.

> It is not a goal to be compatible with GNU utilities, which are sometimes viewed as bloated and overloaded with rarely-used options.

> A similar project with the aim of GNU compatibility is https://github.com/uutils/coreutils

Just added this FAQ section to the README:

Because it is a FAQ, the major differences between this project and uutils are:

1. Wider scope: posixutils is far more ambitious than uutils from a breadth standpoint: posixutils will include bc, m4, c99 compiler, fort77 compiler, a cron daemon etc. uutils is far more limited in the scope of programs covered, mimicing GNU coreutils.

2. More minimalist: Each posixutils utility implementation is intentionally more minimalist, intending to avoid the bloat of supporting rarely-used, non-POSIX features. Our common denominator and baseline is the POSIX spec, then add non-POSIX features that users cannot live without.

3. Transportable: Each posixutils utility should look like normal Rust code, easily stand alone with little-or-no deps, and be used in another project. This project is MIT-licensed, not GPL licensed, to aid in that transportability goal.

If you want core utilities rewritten in Rust …

‘uutils’ is exactly that.

https://github.com/uutils/coreutils

posixutils is something more. (see README.md)

bc(1) is nice to have, but —for my purposes— less useful than dc(1), because it doesn't generate arbitrary binary output.

If you're talking about the `P` command, I added an extension to my bc to do the same thing. It's the `stream` statement.

But it's probably still more convenient in dc.

https://git.gavinhoward.com/gavin/bc/

Exactly; also nice to see https://git.gavinhoward.com/gavin/bc/#ai-free .

Lagniappe: 6581840dnP

Is that a dc quine? Cool! Thank you!

And yes, I hate AI.

I prefer dc because I'm an HP calculator user... we're not the same ^-^

what's next that urgently needs mem-safety? /bin/true?

this must be parody. what exactly is the threatmodel where memory-safety matters for a calculator? did these devs miss the point of popping a calc.exe? surely no bc has ever been used for LPE or RCE.

The entire repo is written in Rust. bc is one of 145 utils specified by POSIX.

The title reminds me of the adds for "Shen Yun China Before Communism" which begs the question: ok what about life after communism ... where's the song and dance show about that?

Similarly, if this is the memsafe bc, where's the wow it leaks memory like a 1980s rock ballad bc code?

Hint: one has an answer, and the other has really never been on the radar safe or no.

Reply on news.ycombinator.com