Hacker News Reader

Comment by aomix | Hacker News Reader

I saw someone describe python as “stressful” for this reason and I couldn’t agree more. It’s difficult to have confidence in any change I make or review. I need to sit down and manually exercise codepaths because I don’t get the guarantees I crave from the language or tooling. While with the small amount of Rust code I’ve written lately I could yolo changes into production with no stress.

View on HN

Agreed. I had to work in a larger Python codebase after spending a few years with Go and Rust and the drop in logical confidence around the language was remarkable.

I have, roughly, sworn off dynamic languages at this point. Although I have dreams of implementing a firm typed system over Common Lisp.

Firm like:

  https://coalton-lang.github.io/20211010-introducing-coalton/

?

I need to spend some quality time with Coalton. I don't think it's _quite_ what I want, but its in the same key.

Same, though my trauma was Ruby. Those Rubyists who were apparently born with the language spec in their heads can do amazing things, but I am a mere mortal who needs to be told I wrote bad code right when I wrote it, not told at 2am on a production server.

do you not test your code?

In a large enough code base, there is no "your code". There is the code you wrote today, there is the code you wrote two years ago, there's all the code people are writing around you.

I change a function's arguments, I'm _pretty_ sure I caught every place it's called (spoiler alert: I didn't, because someone calls functions dynamically based on their names with call_user_func_array, maybe), and I ran the test suite - everything works, and I've fixed what doesn't.

Except some of that old code written 8 years ago didn't have good tests, or didn't have any tests at all, or didn't cover the specific code path.

And now it's 2am on the production server.

Now, you could tell me - Pavel, you're an engineer, you should be sufficiently dedicated to your craft to verify that the tests are testing those paths, you should find all those instances where that function is called (a few dozen times across the codebase), and check that those tests exist and are good. And you're not wrong! I should be doing cleanup in the codebase as I go, washing those dishes while I wait for pasta to boil or whatever...

... but now I've given myself two more actual engineer-weeks of work for a change that just needed to add a flag to a function to make sure only premium users get emails when something happens. My manager will be thrilled to hear it. Do you think I'll get a raise?

I’m assuming that Python code base didn’t have thorough type hints. What if it had? Would Go still feel safer? I know these aren’t checked in runtime, but Python type system seems more thorough than Go’s, so shouldn’t a Python code base fully typed be even safer than Go? If so, why not?

(I know Python type checks aren’t mandatory, but for this question assume that the type checker is running in CI)

You mean compiler vs client reporting issues? Never safer.

Yes, but in practice, is the difference significant enough to matter? I’m genuinely looking to see if I’m missing anything when favoring Python type system over Go’s.

One difference is that Python's type system is unsound. You can write code that passes `mypy --strict` but still contains functions that, for example, claim to return an `int` but actually return a `list` [0]. AFAIK it's not possible to circumvent Go's type system in such a way.

[0] https://news.ycombinator.com/item?id=43508152

The biggest limitation of Go's static type system vs. Python's type hints system that I can think of would be the lack of union types, with nullability a close second (and it's just a specialized application of union types in Python). An interface in Go can be anything under the hood, though you at least know that, whatever its concrete type, it does implement the methods exactly as defined.

You can definitely circumvent Go's type system with unsafe operations (with FFI also recognized as inherently unsafe) but that's a clear boundary that you know you're crossing (albeit not quite as clear as e.g. in Rust).

It matters the most in practice, actually. Much less production all-hands-on-deck situations.

The fact that there still exist people to whom this is not painfully obvious is despairing me.

You're hand-waving the insane amount of effort that would be required to type every bit of Python code that gets executed. You get that by default in statically typed languages.

Even then, you're reliant on the correctness of a 3rd party tool vs a 1st party compiler. It's never going to be as good of an experience.

Since Python type checking is functionally optional, it implies that somewhere, somehow, someone has opted out, or has used a type checker that differs subtly but unpleasantly from the one you're using. Python's type checker systems are fundamentally a _linter_ more than something like Java or C...or Rust.

It's not about feeling. It's about the theorems you can make about the codebase. With Python the answer actually is ¯\_(ツ)_/¯.

> I need to sit down and manually exercise codepaths

Isn't that exactly what unit tests are for?

Except now you're writing and maintaining twice the amount of code instead of relying on the compiler and/or type checker to help you catch those errors

Sorry, but I don't agree that static typing is a replacement for unit tests. I can see static languages having fewer unit tests, but it's not going to eliminate them.

Static typing is a replacement for unit tests aimed to catch type bugs. Also, no one has 100% code coverage so might as we get some guarantees for granted.

I understand that statically typed code doesn't mean bug-less code, but I always find it odd that dynamic language enthusiasts feel like they need to stretch the benefits. Dynamic languages are great for many things. Strictness is not one of them and that's fine.

It's obviously not a replacement. You can get the types right but the values wrong due to faulty logic.

Sure, but only some of the unit tests are about values. Others, at least in my experience, are any handing various aspects of the implied types of values, and those can largely be removed.

Moreover, a good type system can also force the values to be right by enforcing certain invariants directly in the type system. For example, let's say I have a function `deleteProjectAs(user, project)` that deletes a project, but it only works if the user is an admin, otherwise it throws an error. I can write a bunch of tests that validate that this function checks that the user is an admin in all sorts of different cases, but with a type system I can write something like `deleteProjectAsUser(user: Admin, project: Project)`, and this guarantees at the type level that no non-Admin user can delete a project.

The point here is not that types can replace all tests, but that well-designed types can get rid of a lot of them.

I agree. My post was unnecessary; I did not realize the response was talking specifically about type bugs.

You're tearing down a straw man. Your parent poster did not say it eliminates tests. He said static typing mean less tests. He didn't say static typing is a replacement for tests either.

Why are you doing double misrepresentation?

More honest discussion tactics, please.

Yeah, that's a common argument for dynamic typing. You're writing tests anyway (right?), and those will catch type errors and many other kinds of error. Why bother with a separate level of checking just for type errors?

I personally believe it's a valid argument (others will disagree). IMO the main benefit of static types isn't for correctness (nor performance) - it's to force programmers to write a minimal level of documentation, and to support IDE features such as autocomplete and red underlines. Hence the popularity of Python type hints and TypeScript, which provide these features but don't fully prove correctness nor provide any performance benefit.

Fortunately my compiler writes a large number of unit tests for me, that run at compile time! I call it the "typechecker".

When using dynamic languages, either minimize code dependencies / function calls and complexity or ensure high test coverage.

Do you believe that Rust's type system is as flexible, powerful, and easy to maintain as unit tests in Python?

One of the big advantages of Rust's type system is that, if you decide you want to change something (add a parameter, give a type a lifetime, rewrite an entire module), you can just do that, and then follow the errors until they're all gone. Often, once the types have been fixed again, the result will work first time when you try and run it.

In this regard, Rust (and other languages where lots of data invariants can be encoded in the type system) is very flexible and easy to maintain indeed, because you can easily make changes, even in very old or poorly-maintained code, without having to worry about the consequences. Moreover, rather than writing all the unit tests yourself, it's as if the compiler is writing the unit tests for you.

In fairness, you can't encode everything in the type system, so you still need unit tests in top of that, but in my experience you can get away with far fewer. In general, I would say that Rust's type system, when combined with unit tests, is far more flexible, powerful, and easy to maintain than dynamic Python with only unit tests.

No, it's more flexible, more powerful, and easier to maintain than unit tests in Python.

I write and test a lot of both rust and python, so I can say quite confidently:

1. Of course a type system is not as "flexible" as arbitrary test code.

2. Compiler-enforced type safety is many orders of magnitude easier to maintain than the equivalent unit tests

3. Defining rigorously enforced invariants with a type system is far, far more powerful than hoping you remembered to test all the important cases.

Your 2-3 points remind me of the quote by Ludwig Wittgenstein: “The limits of my language means the limits of my world.”

Of course if you limit yourself only to problems that can be effectively solved by type system, then it may work. It is like limiting yourself only to those text processing tasks where regexs work. Yes, some text processing tasks may be much more effectively solved using a regex. But it is obvious that at some point in a more general case grep can’t replace python. It may be less apparent for compiler vs. python case but the general logic is the same.

Nobody is "limiting" themselves to anything. That's not a counterargument you're making.

We all deploy to production, dude.

Strong static typing (Rust), even only dynamic strong typing + pattern matching (Elixir), are leagues better than Python. Literal 3x - 7x less code and less production errors both in Elixir and Rust, over the course of 5 years.

You're the one limiting yourself. Expand your horizons.

So you asked a question, didn't like the answer, and decided to just start insulting people instead?

No disrespect intended. I’m making purely technical point.

If using python with type annotations, linters like ruff and mypy do a great job at identifying issues. It's no substitute for tests and nor will it give you the same guarantees that rust will at compile time. But I think it improves the base quality of the code.

The thing I find annoying with MyPt is trying to tell it I'm doing variable shadowing.

E.g. X is a list of strings Translate X to a list of indices Translate X back to a list of strings.

In that paragraph the input and output types are the same, but not complains about the second line.

I always have to introduce a variable with a new name.

Yeah I see what you mean, you can always disable specific features, but I think that's a habit mypy tries to enforce. They consider redefining a variable bad practice.

Even in rust you have to be explicit about doing the same thing with an extra "let" statement.

Yeah that's fair enough, and I've lived with it because of how good MyPy is. But I always end with weird variable names like uuids_as_list as a result!

Reply on news.ycombinator.com