Comment by TZubiri | Hacker News Reader

Comment by TZubiri | Hacker News Reader

I'm sorry but how do you jump from 1. Polling to 2. Asyncio

There's so many solutions in the middle, I have this theory that most people that get into async don't really know what threading is. Maybe they have a world vision where before 2023 python just could not do more than one thing at once, that's what the GIL was right? But now after 3.12 Guido really pulled himself by the bootstraps and removed the GIL and implemented async and now python can do more than one thing at a time so they start learning about async to be able to do more than one thing at a time.

This is a huge disconnect between what python devs are actually building, a different api towards concurrency. And some junior devs that think they are learning bleeding edge stuff when they are actually learning fundamentals through a very contrived lens.

It 100% comes from ex-node devs, I will save the node criticism, but node has a very specific concurrency model, and node devs that try out python sometimes run to asyncio as a way to soften the learning curve of the new language. And that's how they get into this mess.

The python devs are working on these features because they have to work on something, and updates to foundational tech are supposed to have effects in decades, it's very rare that you need to use bleeding edge features. In 95% of the cases, you should be restricting yourself to using features from versions that are 5-10 years old, especially if you come from other languages! You should start old to new, not new to old.

Sorry, for the rant, or if I misjudged, making a broader claim based on multiple perspectives.

As of 3.14 running without the GIL is optional, but the default still has the GIL in place. 3.13 had it as experimental, but not officially supported. 3.12 and back are all GIL all day.

Python's asyncio library is single threaded, so I'm not sure why you are talking about threads and asyncio like they have anything to do with each other.

Python has been able to do more then one thing at a time for a long time. That's what the multiprocess library is for. It's not an ideal solution, but it does exist.

> Python's asyncio library is single threaded, so I'm not sure why you are talking about threads and asyncio like they have anything to do with each other.

Ok, not OS threads, but it de facto creates application/green threads.

>That's what the multiprocess library is for. It's not an ideal solution, but it does exist.

Philosophical argument but, I'd say multiprocess is not python doing many things, there would be many python runtimes (each doing A Thing), and the OS would be the one doing multiple things / scheduling.

It absolutely does not create green threads. Green threads can be preempted and switched by the runtime. Go does this for example.

Python's asyncio is tasked based, the event loop cannot switch out a task until it reaches a yield point.

Or in short, green threads like Go uses are preemptive multitasking, the task based model asyncio uses is cooperative. A CPU bound python task can block the event loop forever if it never yields, goroutines generally can't.

It's not a philosophical question at all. A single python program can use the multiprocessing library to run multiple chunks of work in parallel. It's heavier weight thanks to the need to basically run a full python interpreter in an os process, but it provides the functionality. And the fact that it's scheduled by the OS is irrelevant. Plenty of languages use the underlying OS threading capabilities to manage threads instead of their own runtime. Both Java and C# for example (though I think Java is adding green threads now).

You're conflating several different distinct ideas. It's a common mistake I see at work all the time. Took a good amount of reading for me to untangle them all.

Edit: I left out stackless coroutines vs stackful thread like runtime. That would be more accurate then the preemptive vs cooperative stuff, but either way the gist of my comment is correct.

>the task based model asyncio uses is cooperative. A CPU bound python task can block the event loop forever if it never yields, goroutines generally can't.

I think I was wrong then. The difference between:

a = threading.Thread(request,url1).start()

b = threading.Thread(request,url2).start()

a.join()

b.join()

With similar asyncio code is that async code has to explicitly signal when the task is allowed to switch. So async is used for when greater control is required over when the scheduler can work on the two different tasks, presumably to avoid race conditions. It would be used in cases similar to where semaphores would have been used.

I do still think that it's unpythonic in that whatever you can do with async you can do without, (two ways to do things), and most of the usecases of this will be people coming from node, and people who don't know of more basic concurrent techniques.

I'm looking at the Original Article again, and it just looks like they are implementing a more complex pub-sub control flow system instead of using if statements (because that's too boring?). The traditional solution would use a socket which is essentially a thread, and the states of the connection would just be managed by TCP instead of recreated at the application layer.

I just can't shake the notion that the vast majority of cases async code in python is bad code. Of course the devs are not idiots, but my thesis is that they are bored and have to implement something, and it has an intended use case, but the bulk of usage will be for the 'wrong' reasons.

>You're conflating several different distinct ideas. It's a common mistake I see at work all the time. Took a good amount of reading for me to untangle them all.

If you have a source material to recommend on the subject (assuming I am already aware on traditional OS scheduling of processes, threads and green threads) I would be interested in reading that. It seems that even if my thesis is True, I need to understand for what precise usecases BDFL and company are developing this async thing into the language itself.

I think they were already in the async world and needed message passing -- the polling code was also in python async.

Reply on news.ycombinator.com