Hacker News Reader

This all sounds like a race to the bottom, where automated testing will drive further kludge upon kludge and the code will get less understood and harder to maintain and write tests for.

Whilst an automated test suite can at least check if the code may have broken - you'll never know if the break is significant or just an inconsequential side effect of a new piece of data. And if both the code and tests aren't understood, as you say the "right" fix won't be done and something will be added quickly to fix the test.

If no one knows whether the code is correct a generated test could easily do more harm than good.

If no one is able/willing to refactor the code into understandable and testable bits, there's not much left to guarantee the correctness of it.

In that situation, why do you WANT tests?

If, for example, someone intends to refactor the code into something more maintainable and wants tests to prevent any accidental changes in functionality, then this is useful.

If the goal is to catch bugs or to facilitate future development or to meet some test coverage standard then I argue you might be better off not having tests rather than having auto-generated tests that have not been carefully reviewed.

I'm intrigued by this example. What's an example of a domain or problem set that produces a function with 100 input parameters and a boolean result that can't be broken down cleanly?

Adding a product with many attributes to a database, where some of the attributes depend on the values of others, might be one example.

The decision is actually 5 different variables. Types are: boolean, a decimal percent and integer being a monetary value. The domain is fintech: making a decision on how much money to give and some other aspects about the loan.

That sounds like no one knows what the correct result is.

> The domain is fintech

What you describe is the opposite of what I expect from (modern) fintech.

* no tests

* god method with no proper documentation/requirements

* no one actually understanding the function deciding how much money to send! Isn't this one of the core functionalities of fintech products?

Just plain scary especially with money if they can't at the very least test the extremes and some in the middle values. Good gravy! What is the name of this company so I can avoid it...

The whole thing does not sound very competent. Like noone could possibly guarantee that the result is not off by a factor of 2 or 3 or perhaps even 10 or more in some cases. What would happen in that case? It also sounds like a domain where mistakes could be very expensive or lead to lawsuits or similar such things.

Things are not that bad. While the whole thing is kind of a monolith, there are at least broad sections that split code into logical sections. With some practice, it is possible to easily narrow down changes and any issues to 10-20 LOC. Later, breaking code by them could be a starting place for refactoring. In the end, the decision is manually reviewed. There's also a log that tells how and why the code had arrived at the numbers. So, if you know that client cannot be X but the log says X then you know that there's a bug for example.

This code was around before I joined. It started small, but more and more requirements were added until it became like that. Not much effort went into structuring it. Now it's my responsibility, and I'm trying to make it better for me and anyone who will have to deal with it in the future.

Does this mean there are at least 2^5 = 32 paths that a procedure can take? Seriously, is it even documentable?

This might not be quite the same, but logic systems -- we might have a logic expression on a hundred variables, and need to check if it is true.

While one would unit test with tiny expressions, often bugs only show up in large integration tests, as systems use many optimisations to cache and speed up behaviour, which can interact poorly

Years ago, in a similar situation, I wrote a tool to capture parameters and results for the algorithm while QA ran their full application test suite. Ended up with a few thousand cases and expected results; results where QA had verified the software worked as expected.

Used that to create a test suite that we could then maintain normally.

Much better than random.

What you're describing is testing without understanding the code. I've not yet encountered inunderstandable code, and can't conceive what it might look like, though I've encountered code that I'd rather not try to understand.

So:

- you don't know whether the code is correct or not

- code is being modified on regular basis... and logic is changed

If someone knows enough about the code to regularly change it, then they should be able to provide some documentation? How else are they able to modify a "correct" function and know that it's still correct?

Can I guess that the code modifiers are quants, or something?