Hacker News Reader

Comment by whatever1 | Hacker News Reader

Reproducibility is a huge issue in academia and in the software engineering world.

As an anecdote, we were trying to reproduce results for a c++ algorithm for which we had the source code, on the same machine, same compiler, only 2 years after its original creation. Impossible. Our best guess is that some os/hw patch / firmware update had an impact on the random number generation, but we never managed to figure out what exactly caused the discrepancies that we were observing.

View on HN

The way some video game studios do this is simply put a whole computer into the archive with all the software on it, so all the hardware and software are captured in time.

Shouldn't containers or VMs solve these problems?

No since the hardware changes over time, such as the video and audio hardware, and VMs don't prioritize emulating those anyway.

wouldn't that depend on which VM you are using?

I agree. They used to say there would never be an Amiga emulator because of all the custom chips it had.

But there are quite a few Amiga emulators now.

Video games built on consoles have this benefit in that they are locked to a hardware iteration, granted there may be some server communication now but historically arcades and consoles capture the hardware snapshot as well.

I prefer PC gaming as consoles are essentially locked in time in hardware but that does lend better to archival purposes.

What's the expected lifetime of a computer in storage? I'd think that bitrot sets in fairly quickly on storage medium.

That's data rot or disc rot. Bit rot is also known as software rot.

My professor requires the software, data and makefiles to reproduce any figure in a published paper. Then the figure would be erased with software reproducing it.

The motivation was kind of selfish. The professor would update his textbooks every 5 or 10 years and noted how hard it was to recover old figures.

Did you look at your version of libc? What version the original linked to?

It's been some time, but I am 99% sure that we were linking with the correct version of libc, because we never used the system GCC (in CentOS they provide the very old 4.8 version), so we explicitly specified the compiler and the libc that we wanted to use.

That might not even matter, because if you think you're using one version of gcc, but you're using another. I experienced this multiple times over the summer with scl and gcc.

Reply on news.ycombinator.com