I'm Rewriting IBM SPSS Statistics in 100% Safe Rust, Alone | Hacker News Reader

I'm Rewriting IBM SPSS Statistics in 100% Safe Rust, Alone

github.com

23 pointskobaltGIT36 comments

Alone? How do you achieve clean room reversing with one person?

You don’t state StatCraft is clean room, but all Artcraft software is.

Talk to a lawyer to check if you’re not wasting your time.

https://en.wikipedia.org/wiki/Clean-room_design

Does safety matter for this kind of software? It’s mostly internal tooling used by researchers and statisticians. I get it rust is cool but safety isn’t the prime concern here I think.

It's not about memory safety per se. It's about not losing an hour of analysis because a malformed .sav file or a zero-variance sample triggered a segfault or silent wrong output. SPSS crashes on bad data. Researchers work with messy data. Rust's Result type makes "handle every error explicitly" the default, not an afterthought. That's why I chose it.

Rust is the prime language for vibecoding because the compiler checks most of the issues for you. This is meant to be part of the ArtCraft suite, so it is intended to be developed and managed with AI agents.

People always seems to confuse correctness as in „it compiles“ vs. „it does what I intended it to do“, not sure how Rust helps with that as it won’t solve mismatch between intent and implementation. Or how could it? Main problem with vibe coding is that you almost always under specify requirements so the agent will fill in the blanks and make choices for you, the language it implements these choices in doesn’t matter much in my opinion, agents will produce correct code from a detailed specification almost always which is why they are so good at translating between languages, they are terrible at interpreting requirements not specified or derivable for complex software though. Rust can’t solve that.

Segmentation faults are super easy to get from AI agents writing C. If you want an agent vibecoding in a compiled language, you have to use Rust. That doesn't mean that the software does what is intended (that requires a review agent and then an orchestrator at minimum) but it makes things a lot easier.

There are MANY safer-than-C alternative languages compiled to efficient native code. I don't vibe code, but vibe coding seems popular with Nim, for example. I'm sure it's a very long list. "Safer-than-C" has been a slogan for PLangs for the entire history of modern computing. { Before people groan about "but, but mem.mgmt", that is often negligible overhead in context (depending on context and overhead). When it is not negligible, the same problems in Rust will usually have "allocations optimization" fiddling and such, so you don't really avoid "fighting memory". Ada people often just avoid dynamic allocation entirely. Java people will spot-optimize certain things. Ocaml, etc., etc. AMM is not even a property of a PLang as much as a runtime since even spartan C has -lgc with Boehm. }

I feel like Nim is too expressive to be a good vibe-coding target. That is one feature that I think is generally good for people writing code and really bad for agents.

You can take a lot more "smart" shortcuts in Nim, that are hard to reason about, and probably even harder when an automated system writes them.

I very much agree at the stylistic and syntactic abstraction levels. One reason I don't do it myself is that I find chatbot "judgement" for such things awful, usually highly contaminated by weaker PLangs. To be fair, there is probably not much sampling data in several relevant directions. Honestly, I think it Araq's personal tastes in tooling drives people to vibe Nim, but I do hear about it a lot.

> not sure how Rust helps with that as it won’t solve mismatch between intent and implementation.

It's not rust specific, but any language which allows you to work on a higher level and with better typing helps close that gap. For example for a long time Linux kernel had an issue with marking user/kernel addresses which opened it up to bugs and security issues. In C those are just numbers. In Rust this is moot and correct by default because user addresses are explicitly UserPtr.

So yes, sometimes a better language allows you make some unintended things explicitly not compile. It won't solve the issue completely, but the scope of the mismatch gets much smaller.

Author's comments are dead unfortunately so I'll repost w fragment from them here:

> It's not about memory safety per se. It's about not losing an hour of analysis because a malformed .sav file or a zero-variance sample triggered a segfault or silent wrong output. SPSS crashes on bad data.

Why not [vouch] the dead comment?

I've done some without any effect. I'm guessing there may be some sneaky moderation in play. I did email the mods though.

A couple days ago we had 0-click RCE in MS Paint. At this point, we must think about safety everywhere, we're entering AI era where vulnerabilities are easy and fast to find, exploit and chain in automated ways, and it only accelerating.

GNU PSPP [0] is what got me into free software after losing access to SPSS in the middle of writing my undergrad capstone.

I’ve long since switched to python & other normal programming for any stats needs. And I only ever scraped the surface of SPSS/PSPP anyways. But, I’d be interested to hear what PSPP doesn’t cover.

Good luck on your project.

[0] https://www.gnu.org/software/pspp/

Sorry but the problem is not primarily software, it’s community, teaching and standardization.

We have R/tidyverse which does 90% of everything and has an amazing community and procedures eg. around documentation. For everything else you have excel or python. So I don’t see the use case.

There are tons of people in the social sciences that still, unfortunately, use SPSS to this day. If you think that R/tidyverse/Excel is a true alternative to SPSS, you definitely haven't interacted with any of these people.

I have, in fact, professionally interacted with, published with and taught these people for almost 20 years ;)

Ah, are you a statistical consultant? That would explain a lot, but you should then know why these people need you. Or maybe you're in a department where the statistical standards are much higher than in mine.

What’s wrong with PSPP?

Or, indeed, JASP or Jamovi?

No experience with or opinion on SPSS;

GNU Octave is like Matlab. Matplotlib is like Matlab but doesn't support the same syntax.

Will it have a DataFrame API?

Before R data.frame and Pandas DataFrame and substrait.dataframe, there were labeled arrays in NumPy.

Pint uses numpy ufuncs to type numbers with physical units.

Here are the notes I had for pgrust a few months ago: https://news.ycombinator.com/item?id=49112449 ; Polars (already has tested SIMD and GPU-optimized routines), substrait interface, and file formats: LanceDB, Vortex, Arrow recordbatch

Here are my notes for the ArtCraft folks about methods for Rust ports (and whether it's necessary to also port Blender to Rust with an O3/Maturin Python API) https://github.com/storytold/artcraft/issues/2489 :

> port the tests [and check figures] first; use branch coverage metrics when testing; use cargo-llvm-cov and rust-nightly to test; make (.sh,) scripts that the .github/workflows/ call so that local builds are easy; run local builds using nektos/act; use rstest to keep tests and test fixtures maintainable; use cargo-insta to snapshot; use cargo-fuzz, and do coverage-guided fuzzing; use TLA+, Lean or similar formal methods and tools to model the critical parts of the app at least

In the ArtCraft issues, there are so many great ideas that any or many or all firms could benefit from.

I am doing python based pipelines, new pipelines were too slow, I must admit, the first that crossed my mind was to rewrite it in Rust:) But later after detailed analysis it was an architectural problem. You may not need Rust:)

>I can't into R

But why?

Probably he has some tokens available to use

I'm (well, my LLM is) Rewriting IBM SPSS Statistics in 100% Safe Rust, Alone (for a given value of alone).

You even let your LLM write the README.

Look, we nearly all use LLMs these days, but the phrasing of "I am... ...alone" is particularly weird when it's obviously not just you, it's you with LLMs.

Maybe title it as "A rewrite of IBM SPSS Statistics in Rust with no unsafe" - the 100% safe claim is something an LLM makes, it's not something a reasonable human would say, as it implies the Rust is totally safe, whereas you're trying to communicate that your Rust code uses no unsafe blocks, and these are two different things.

Offering a "statistical package for the social sciences" is like giving guns to toddlers.

Why not let go of SPSS instead? It has run its course and is basically legacy legacy software by now. But it good fun I suppose? Are you using SPSS as an oracle? (Guessing that the author is/will be in the comments.)

An oracle and fuzzing have been excellent when reimplementing software. Given implausible inputs, the black box shows its colours quickly. Rounding errors and different representations of floats are and endless source of debugging, eh, joy.

I think a lot of people would like to move on from SPSS, but try telling that to the people that have been using it for the last 25 years.

This is one of the things that makes me negative on AI. Before AI, no one would have done this. Because it's a huge amount of effort and a total waste of time. Now we have AI, we have this explosion in people going off and doing all these pointless activities.

What do we gain from a plethora of SPSS-alike tools? Nothing. There's literally no explanation on this page about why anyone who uses SPSS would want to use this. But it's "100% Safe Rust". Yes, that's a feature that someone who likes rust wants, not what someone who uses SPSS wants.

What do we lose by fracturing all academic statistical analysis into a dozen different tools instead of 1? Lots!

It's not even clear the person creating this even intends for anyone to use it. So (a) why are they doing it and (b) how do you even know if it actually works if absolutely no one is using it. In fact the danger now is someone who doesn't know the author is just an AI slop dilletante comes along looking for an SPSS-like tool and makes the mistake of trying to use it.

Is there really any difference between this and the people posting AI slop onto their reddit accounts. It's just raising the noise floor to the point where we're going to have to essentially shut down the public internet, everything of value will move to closed spaces.

Sorry, but this really comes across as demonstrating a substantial degree of entitlement. What is the purpose? Easy. A student is trained to use SPSS for their thesis work. They may be given a sitewise license from their university, but only for one device. Then after their training, they lose access to SPSS, but want to do that analysis in their workplace.

Proprietary software, especially at these prices, cause all kinds of issues. SPSS is one of the software packages that has a sizeable audience of people crying out for a true open source alternative. Just because you have the money to afford these things, does not mean that others do.

Also, if you are referring to the similarly named ArtCraft suite as a "pointless activity" then you are really out of touch.

I'm extremely on the other side - it's not about what we gain, but what we don't lose. If there's even one person with a dataset they can't process easily without a proprietary tool, why shouldn't this exist?

I'm extremely happy every time a closed format falls. Whether it's this one, Sibelius music scores, records from medical devices, data series from scientific measurements, or something else. Every one of them stopped me from doing something in the past (not on a technical level but "it's not worth spending a month on this") and I'll support efforts like this.

The title "I'm...100% Safe Rust, Alone" is obnoxious.

Op, is there any technical merit to this, or is this a step in some process to earn brownie points in some pathetic "I'm a morally superior hero club?"

What'd you expect the rest of us to feel? Chastened?

I'm building StatCraft — an open-source statistical analysis package in 100% Safe Rust, as a clean-room alternative to IBM SPSS Statistics.

I started this because I believe statistical software should not crash on malformed data, should be scriptable by AI agents via MCP, and should not cost thousands of dollars per year.

Current state (11 days in, solo):

- 14 statistical blocks audited against SPSS 24 Algorithms Guide with ≤1e-6 SciPy precision

- Full .sav/.zsav codec, Excel/CSV import, SPSS syntax parser (.sps)

- 37 SPSS-style dialogs, pivot tables, Output Viewer with TSV export

- Built-in MCP server for Claude/ChatGPT/Ollama agents

- Zero unwrap/expect/panic in production; fuzzed on corrupted inputs — 0 panics

What I need help with:

- Testers with real SPSS datasets (the weirder the encoding, the better)

- Rust developers for the expression evaluator (3 known precedence gaps)

- Statisticians to validate output against their SPSS workflows

- Anyone who wants to port a procedure (regression, factor, reliability are in)

GitHub: https://github.com/kobaltgit/StatCraft

Nice to see these efforts. It’s completely vibe coded though, and the agents.md is in a language I don’t understand: https://github.com/kobaltgit/StatCraft/blob/main/.agents/rul...

I think I’ll pass.

Reply on news.ycombinator.com