Hacker News Reader

> My stack is pycharm, mypy, pydantic, sqlalchemy stub, several mypy plugins, black, isort, tabNine completion, and a bunch of custom scripts/Make to automate the common tasks.

Christ. What a mess. With a stack like that, it's a stretch to say you're "writing in Python" anymore.

> Most of their main arguments (...) evaporate as soon as you start using the correct tooling.

You silly! Just use these carefully curated list of 28 different tools (that I got to after years of trial and error) and you'll have a somewhat acceptable tooling and type support in Python.

Anything that can use an LSP server will give you most/all of those benefits. Your favourite editor + LSP, that's the tooling. It's very much worth it.

That list is an editor, a type checker, a package to enforce type checks at runtime, 3rd party type annotations for a popular SQL ORM, some plugins for the type checker (probably the pydantic one), a code formatter, an import sorter, and (optionally) a Github co-pilot alternative.

To be fair, I don’t think you need that entire stack. Just using VSCode + pyright (which is a one button install in VSCode) + type stubs for libraries your using without native hints, gets you 99% of the benefits.

If you really want, you can even throw in a little pre-commit hit hook that runs pyright and prevents you from commuting code with type errors.

Wow gatekeeping types out of python are we? Of course the language looks different when typing is enforced, let it evolve.

That's such a disingenuous argument.

First of all, Black and Isort are literally code formatters that have nothing to do with type hints.

Second, the whole point is that you can actually develop powerful dev tooling for Python now, whereas in the past it was difficult or impossible.

I think the person you're responding to is just trying to be thorough in case it is helpful to someone else setting up python tooling; the only parts of what they wrote that are relevant to type checking are mypy and pydantic.

And pydantic isn't really about type checking either. In fact it says that in the first few lines of the it's docs https://pydantic-docs.helpmanual.io/usage/models/ :

> pydantic is primarily a parsing library, not a validation library. Validation is a means to an end: building a model which conforms to the types and constraints provided.

Anyone whose used Typescript or even strong typed languages knew what to expect from Python + mypy. No surprises there. But pydantic's use of type hinting to create a remarkably dense serialisation library, and then fastapi's leveraging that to create REST interfaces along with automagic documentation generation - well that wasn't when I was thinking would happen when I saw the type hinting proposal.

It's more so "this is my toolchain I use for my own ergonomics, and what you can achieve with the right kit".

Who cares if I'm "writing in python" at all? I'm solving problems. Honestly a good IDE (pycharm or vscode) will give you 80% of the ergonomics.

As long as you don't have to rely on 3rd party libraries, sure, type hints are fine.

Many popular libraries, however, don't have type hints or use them inconsistently (looking at you numpy, pytorch, tensorflow, scipy, ...).

Always great fun to see function signatures that use type hints for the parameters and return Any.

So it really depends what kind of code you write - web frameworks might be fine, data science and ML is definitely not.

You can write type stubs for other libraries. Over past year my team had a 100k line codebase that we’ve gotten mostly typed. But we still use a lot of other untyped libraries like tensorflow. So in past couple months we’ve started adding type stubs to the parts of the api we use. While tensorflow/numpy/etc are massive libraries most people only use a small subset of api. You can write type stubs for the subset you use. It definitely takes time but I find it very helpful to have useful type checking of tensorflow so I bit bullet and started writing stubs. I explored upstreaming them to type shed but got lazy on dealing with CI issue although I hope to revisit it later.

https://github.com/python/typeshed also provides community maintained stub packages for packages that are lacking upstream type hints

Type hints are good if you stick to the std + your own code that mostly relies on std.

Otherwise, don't bother.

If you expect type hints to be perfect and completely disallow you to write type unsound code then this is right, if you expect type hints to be useful when developing then this is very wrong. The automatic type hints produced by pyright on completely untyped code are fantastic.

I was curious of seeing if pydantic was mentioned, thanks! We use most of the same tooling as you, I'm always interested in improving our tooling, would you be willing to share your mypy plug-ins names or maybe most useful tools for inspiration?

My thoughts exactly. Obviously, Python Type hints are an attempt to add type-check to a language "after the fact" and keep it optional. So it's unfair to compare that to a static typed language and conclude that it's bad.

I work with large code bases (100K-1M lines) and can't look back to the time we didn't use Type Hints. Would maybe dismiss it only for scripts or very small projects.

I use this very strict config with mypy and it's effectively a typed language. I rarely have any errors in runtime now, at least from type-related problems like assigning 'vals = 3' and later trying to call 'vals.append(4)'. I sometimes use 'Any' because it's basically required for things like loading JSON or retrieving the results from a GET request, but I have several strictness settings that ensure I never pass around wild Any values and I try to keep their use to within a single function or method and always return a well-typed value. Sure it's all in my IDE right now, but if I import 1 or 2 packages suddenly my well-typed Python code starts to act more like a typed language in runtime, too!

    disallow_any_unimported = true
    disallow_any_expr = true
    disallow_any_decorated = false # true if you never plan to use Any
    disallow_any_explicit = false # true if you never plan to use Any
    disallow_any_generics = false # true if you never plan to use Any
    disallow_subclassing_any = true
    disallow_untyped_calls = true
    disallow_untyped_defs = true
    disallow_incomplete_defs = true
    disallow_untyped_decorators = true
    no_implicit_optional = true
    warn_redundant_casts = true
    warn_unused_ignores = true
    no_warn_no_return = true
    warn_return_any = true
    warn_unreachable = true
    strict_equality = true
    strict = true

What opensource project is closest to "doing python the right way", in your opinion?

I'm in "python is not suitable for production" camp, would be happy to change my mind.

> I remember the dark ages, before type hints. It was rough. I lived in the REPL. Write 3 lines, run, Exception. Write 3 more lines, run, NoneType error.

That's why I switched (back) to typed compiled languages and never went back.

Python types just work. Just use these 30 libraries, spice up your IDE with settings and voila /s.

This is garbage and moving backwards as a programmer. I loved python when it was easy to use with any editor and with few plugins. Now I'm just going to switch to a statically typed language where I don't have to install a zillion plugins to get first class typing.

This is the crust of the problem.

Python type-hints bind you to a lot of other sister projects of Python, which makes you question if they are even official addition to the language.

I think that's also what the OP's article tries to convey.

Python type annotations in the language are nothing more then allowing you put any Python expression in the "type slot" and have it be syntatically valid. They don't do anything. All the functionality is 3rd party packages taking advantage of their existence.

My codebase is more complex than str, text, dict, int, bool.

So developers are creating complex custom types with union[custom_type_1, custom_type_2] where each custom type could have more unions of other custom types. These custom types then get imported everywhere. It is utter garbage.

I don't follow. So some 3rd party library will define a type like Routes = Union[RouteSet, List[Union[str, Route]]] and this is bad? The complexity is already there, you just don't want to see it?

No, my own devs will write insane nested types at the detriment of all developers who end up looking at that code later.

Routes = Union[GooglyRoutes, MetaRoutes]

Where, GooglyRoutes=Union[List[CloudyRoutes], Sequence[AdseyRoutes]]

Where CloudyRoutes = Union[RouteContainers]

And then they'd import these types all over the python repo creating all kinds of import issues. Fml.

What’s the alternative? You have some method like

    def register_route(route): …

That gets called with all the types in that nested union but now the type checker can’t help you if GoogleRoute and Metaroute have different attribute names or that .append isn’t a method on sequences but is on lists.

Like that unwieldy nested union already exists in your code, adding the type just documents it and the type checker makes you handle all the cases.

Is there something specific about imports that don’t work with type alises?

In a language with a sensible typing system, I would create

class Route:

    _route = [] # can be GoogleyRoute or MetaRoute

And import Route everywhere. That import would help in creating instance objects AND be used as a type.

But we can't do this with the spaghetti that python type annotations are.

You know you can use type aliases, right?

So now you are going to create complex types and then create aliases for them? What are we even developing at this point?

Some people love this. They spend more time screwing around with types and related tooling than writing actual working code.

As many already said, 99% is just a modern IDE with a one-click-install plugin.

Also, I don't know the grip with productivity tools.

But a language relying so heavily on IDE means that the language is severely lacking.

People not using said "modern" IDEs are just stuck out of using the language. Is this progress? Even with modern IDEs, there is a difference between what different IDEs do. So now the language is stuck with different IDEs doing different things. Is the progress?

It also means that anyone developing a future IDE can no longer support these languages without support. Is this progress?

God forbid someone tries python on a new architecture where new the entire complex toolchain is not going to be available. They just can't use this language. Is this progress?

I'm pretty sure the IDE uses either mypy or pyrite under the hood.

I don't know how this is a sign of python lacking. Works just fine in jupyter notebooks, too, and that is hardly an "IDE" (though it does have Jedi autocomplete).

> I don't know how this is a sign of python lacking

If a language cannot be written or used effectively by devs without IDE, it is an incomplete language.

This has happened in the past is Java. Within a few years, Python will bloat up like Java and everyone will move on to greener pastures.

I think the difference is largely a culture problem. Most python devs think of typing as a “nice to have” extra, whereas someone using a typed-at-compile-time language see it as an essential feature of good code. You can see the results of this in the fact that a lot of popular 3rd party python libraries still don’t have type hints.

Even with typing, how often do you see Any in python vs Object in Java?

> I think the difference is largely a culture problem. Most python devs think of typing as a “nice to have” extra, whereas someone using a typed-at-compile-time language see it as an essential feature of good code

I love statically typed languages. I also love python. But pythons "type annotation" system is garbage. It is neither here, nor there and developers spend an inordinate amount of time figuring out types.

While other languages typing is a productivity booster, pythons type annotation absolutely isn't, especially with complex types.

Definitely types, but it's still weaker than in other languages. Just the other day found a maddening bug, where Django ORM query accepted `date` type comparison against a datetime field, and of course that resulted in crazy things in production. Other languages would have caught this compile-time.

Have you tried pytype? Curious if anyones compared it properly To mypy.

Otherwise the exact same stack as yours, we use fast api as well. Also a big fan of using dataclasses everywhere.

You’ve never used a compiled statically typed language. I’m sorry to say it but your comment would sound naive to you if you had. The article is spot on.

I used tabnine from early on but found its capability pretty well subsumed by GitHub copilot.

What has kept you on tabnine?