Hacker News Reader

Comment by ABeeSea | Hacker News Reader

> Mark Perlin, is said to have argued against source code analysis by claiming that the program, consisting of 170,000 lines of MATLAB code, is so dense it would take eight and a half years to review at a rate of ten lines an hour.

So it’s definitely riddled with bugs. And I can’t imagine that much matlab code following rigorous software engineering practices.

View on HN

I have done a lot of source code review in my time. For security assessments. Our general rule of thumb was about 10k lines per week that we can really get deep on. 10 lines an hour would only be for the most dense code and critical path stuff. They will need a reviewer that knows the domain (DNA), but it’s perfectly reasonable to review that code on a weeks/months time scale, definitely not years.

My exact thoughts. This sounds like a classic example of launching a prototype created by domain specialists (biostatisticians and bioinformaticians) as production software and skipping on the expensive stuff, like sound development practices

There is so little emphasis in production software development in bioinformatics and biostatistics. Despite a lot of groups open sourcing their code it's is nearly unusable and not reproducible due to hard coding, ignoring edge cases, and dumping the majority of the code on a single giant R or python function.

It's a real problem, and I've been struggling with it for two decades, but even so I am legitimately impressed (and not in a good way) if they have 170,000 lines of Matlab code in their production software. That takes a really special combination of productivity and cluelessness, even for academic specialists. Regardless of the facts of this particular case, it should be absolutely horrifying that anyone's freedom is left up to a gigantic pile of unaudited Matlab code. (That said I am almost certain he added some zeros to the number, I have a hard time imagining what they're doing that could be that complex.)

Choosing MATLAB as a language for software that could potentially lead to people dying (in areas where they still have the death penalty) is a gigantic red-flag

I don't know how many job postings ask for a software engineer who knows MATLAB, but I can't recall any

That's my experience as well. My master's is in bioinformatics and I worked for several years in biotech.

I got frustrated because my concerns that my team's development practices were causing issues on a regular basis, were ignored. I was continuously able to predict what issues we would run into, but no-one seemed to care - I even had a manager tell me, that it was good that our software was buggy, since the client would continue paying us to fix it

I've since left the biotech industry. There's a limit to how many times I want to run my head against that particular wall

It also sounds exactly like Ferguson's Imperial College epidemiology model that apparently compelled politicians into imposing hard lockdowns (and was likely wrong by at least an order of magnitude):

- "a single 15k line C file that had been worked on for a decade" [0]

- code review of the model: [1]

- corresponding HN discussion: [2] (including sad appeals to authority: you're not an epidemiologist)

- other HN discussion [3] (including ridiculously blaming programmers for making C++ available to non-programmers)

[0] https://twitter.com/ID_AA_Carmack/status/1254872369556074496

[1] https://lockdownsceptics.org/code-review-of-fergusons-model/

[2] https://news.ycombinator.com/item?id=23093944

[3] https://news.ycombinator.com/item?id=23222338

For reference, whole Tesla Autopilot is 'few hundred thousand' loc.

https://youtu.be/YAtLTLiqNwg?t=947

>170,000 lines of MATLAB code

This is a deep problem. Many scientists don't understand software engineering and more and more need to write bigger and bigger programs. And most of the time they don't open source their code.

Open source science.

That's like an accountant, accused of embezalling, refusing to hand over the ledgers because "there are just too many records to go through". Yeah-no, that's kind of the whole point. We want to find out what you've been hiding in that wall of paper.

I also cant image anyone thinking that would be a winning legal argument... "This software is too complex to look at so just trust us" Really... that is what they went with...

Yeah, that's one of those "defenses" that feels like a confession.

unfortunately, the technical illiteracy of much of the US judiciary has made this effective.

and surprisingly all of the code is of equal importance so you really need to review each line sequentially! Instead of finding stuff that you think is most likely to relate to what you're trying to figure out and debug from there. Wow I would like to see this marvel of engineering myself!

Kind of makes me wonder: if the argument is that the code is too complex to review and understand, does that mean the company is not doing code reviews themselves?

this has actually only happened to me a couple times but it has happened - someone tells me Bryan, go look at the code X did in Y, figure out if we refactor. X would then tell me - that code is really complicated is full of algorithms! I go and look at the code realize that for what it is trying to do can be cut down from 10 pages of printed code to less than 1 and it was incredibly simple what actually needed to be done.

In short when someone tells me the stuff is too complicated because too clever and advanced I tend to disbelieve them.

that said I have of course written my too complicated stuff lots of times, but if asked I don't say it was because I'm clever.

names anonymized so as to not accidentally hurt anyone's feelings.

on edit: actually one time the code was clever but not especially difficult, they just used the algorithms line because they didn't want anyone messing with their stuff.

I think there's a bias towards judging things to be "clever" if they're hard to understand

It's a cliche to have a "what idiot wrote this" outburst, then realise it's your own code, because most of us have written our fair share of "clever" code

My boss explicitly stated that he doesn't want to see any "clever" or "smart" code in our product - write code based on simple fundamentals, benchmark before deciding to optimise, and be respectful in your reviews

I like my boss a lot

If the code was developed in less than 8.5 years, it seems like the answer must be yes based on their previous claims.

Well I mean if something requires 8 man years of work you can do it in a year with 8 people. Not always, but usually.

I would claim that if something requires 8 man years, that it will most definitely take more than a year to develop with 8 people.

Communication takes time, coordination takes time, there is an incremental cost to each news person added to a team. From experience, perhaps with 2-3 people who happen to gel well together you may get close to proportional scaling of output, but with 8 it’s really unlikely in the real world.

On the contrary, such linear scaling would be quite exceptional. I'm speaking from experience but you don't need to trust me; I invite you read any book on software engineering management, starting from The Mythical Man-Month by Brooks.

the mythical man month was first published in 1975, I think the typical applications programmers work on today have changed significantly since then and encompass many different disciplines (to be thought professional) - so many disciplines that one developer is likely to be the master of all. It is true that there is a communication overhead to adding more people so it will not scale linearly, but if a single developer has taken 8 years to build something in our era it seems likely that having 8 people might get it done say 1 and a half to two years.

I would find it extremely unlikely to work that way.

You can do code review on every single commit while not being able to make overall analysis of the massive codebase.

I completely agree, but if the reviewer isn't able to (with some amount of accuracy) predict the impact the committed code will have on overall behaviour, then there's very limited value on doing the review in the first place

In any larger project, the reviewer are not able to predict impact from reading commit.

More importantly, typical reviewer have only small partia area where he has good idea about which commit is bad idea. He however does not understand whole codebase.

Knowing what the whole does and knowing what my module does are two different things.

I again completely agree with you

Looking back at my reply, I think I should have added a bit of background to clarify my comment

My master's degree is in bioinformatics and I worked in the biotech industry until about a year ago. I mainly worked as a consultant for top 20 pharma companies, but also did work on different in-house projects and in academia

From my experience in the industry, I find it very unlikely that the software mentioned in the article is structured in a modular way. I've yet to see good software practices outside one or two academic projects. Most pharma companies still use copying and renaming folders as version control. Naturally I'm sceptical of any code coming from the biotech industry

On top of that, it's written in MATLAB. I have only ever seen this used by statisticians and university researchers, never by software engineers

I'm therefore willing to bet, that when the reviewers open the source code, they'll find unstructured mess of spaghetti code, that has never been refactored, reviewed or tested

So yes - I agree in all your points, but I find it unlikely that they're being applied to this particular project

Reply on news.ycombinator.com