Hacker News Reader

Comment by RadiozRadioz | Hacker News Reader

Semi-related question: Why did the ML people choose Python and make it so intertwined with ML? It seems like a bad choice because it's slow and bad at integrating with hardware, the actual hardcore ML stuff gets offloaded to C, which creates a weird system with two languages and a weird build step to achieve one thing. If I knew nothing about the ecosystem of available libraries and was asked to pick a language to use for ML, Python would be fairly low down the list.

Is it just because it's easy, somebody picked it a while ago, made something influential, and now everybody uses it because everybody uses it?

View on HN

IMO, it makes more sense to think about Python or another high level language as glue code for all these C/Rust/Cuda/OpenCL kernels. Why would you choose something like Rust to also write the glue code in? Most people will be slowed down quite a bit by the language overhead, and the people writing this code need results last month.

Because modern ML is a descendant of data science/stats work being done in systems like MATLAB and R. Numpy and scipy were great libraries, and Python was a high-level language that statisticians could use, but also a reasonable enough choice to code a backend for a service in. If the field descended from systems programmers doing high-performance computing, a different language choice would probably have been made, but prior to deep learning becoming popular, compute was not nearly an obvious blocker to improved model performance.

I went to college starting 2007 (right when CUDA was released) and recall a few things about python that made it attractive in both the comp sci and math departments at my school.

  - python integrated well with lesson plans instructors had
  - python integrated well with the software math departments were using (like MATLAB)
  - python had a very easy to use interpreter environment that worked well on linux, mac, and windows
  - python had (for the era) a massive standard library, making it way easier to consume and process 
    common data file formats like XML or CSV without having to install any extensions 
    (installing extensions usually required admin permissions and... very few at school had those for very good reasons)
  - exposing c/fortran libraries into python was pretty straightforward to do and was relatively stable compared to alternatives.

Around 2007, the main competitors in the scripting space had a lot of issues:

  - tcl - somewhat popular for embedding but didn't really have a standalone interpreter that made it easy to interact
    with and the syntax didn't really look like C, and really not much of a library to work with

  - lua - similar to tcl, popular for embedding but didn't have a popular standalone runtime and interpreter.
    syntax was closer to C though. didn't have much of a library to work with

  - ruby - had a runtime for experimenting but a less stable API for integrating, plus the language was pretty
    different from C. the library was large but had a larger reliance on installing extensions than python at the time.

  - perl - both unstable and had a horrible reputation by this point as a standalone language that outside of some obtuse
    systems scripting, most people didn't want to touch this with a 12-foot pole. standard library was small and relied a lot on extensions.

  - php - similarly unstable (at the time... i think 5.3 had just come out) and had a horrible reputation... also not even
    sure it had a runtime that worked outside of an apache/nginx context... exposing a c library to PHP at this time was pretty
    straightforward though. relied very heavily on extensions at this point.

  - javascript - node.js hadn't come out at this point... so the idea of running javascript outside of a browser window wasn't really a thing yet.

If one were looking to expose CUDA to programmers who didn't want to work in C during this era, I'd argue that python was just the best option at the time.

I believe that led to a first-mover advantage that has held over time.

The usability without extensions part is underrated. Whether I'm a beginner in school or a pro trying to hammer out a quick script, I would prefer the thing that doesn't make me jump through hoops to deal with CSV JSON XML etc. Far more important than the language syntax or other things.

And the little scripts I write at work tend to grow and then turn into full-fledged services in a shorter period of time than it would've taken to set up the C++ or whatever boilerplate.

> Perl [...] unstable

Huh? Please elaborate. The backwards compatibility is excellent. The core has been rock solid since the 90s. Especially compared to Py -2-to-3- thon.

In 2007 Perl (to outsiders at least) seemed deep in Perl 6's "redesign and rewrite everything" phase, with no end in sight and no predictable future.

Perl6 is no Perl it has been renamed to Raku. The rest of the Perl5 environment it's ultra-compatible with previous releases. Albeit compiling PDL it's rough case.

You look like never touched Perl seriously, Raku it's still a hard deviation.

No one used Perl6 on real life environments. It's the Duke Nukem Forever of the programming languages.

Using Perl6 as an argument on Perl5 it's like bashing the simple and clean ANSI C giving C++ as the example and its the crazy redundant features as a rant.

Read the first two words of my comment again.

We're talking about the choice of Python for ML when CUDA came along, nearly 20 years ago. Upthread, alaties said:

> Around 2007, the main competitors in the scripting space had a lot of issues: [...]

> Perl [...] unstable

RadiozRadios replied:

> Huh? Please elaborate.

I did.

In 2007, Perl 6 was not Raku, it was intended to be the next major version of Perl. But it had been in development for ~7 years already, couldn't be relied on or predicted, and had become a bit of a joke. Since Perl 6 was still supposed to be the future of Perl, in 2007 Perl very much seemed unstable and unsuitable as a foundation for ML research.

I agree on Perl's choice. But Perl5 was granted as an stable platform, kinda like AWK, pretty much unlike Perl6, which wasn't the future of the current Perl5 at all, except for the hype.

Remember Perl 6?

scheme/guile?

Didn't have a popular standalone interpreter. Also unstable.

Well, it's as true of Guile as it is of Tcl and Perl...

Python is a great language for ML because it allows gluing stuff together quickly. Using a 100% statically compiled language would slow things down, and the typing wouldn't help much because most languages can't properly type check tensor operations, and ones that can (like C++) tend to have awful compile times. The state of the art is tracing the execution and compiling it to a ML-specialised compiler (XLA for Jax, torch.compile for Torch), which is much harder or impossible to do in a compiled language.

It's largely because Python had excellent numeric libraries, like NumPy and SciPy. You could script Fortran linear-algebra kernels to crunch your numbers fast in a high-level language, and that's a killer app in the ML space.

Worse is better.

Also, python is just the glue, the work happens in libraries.

What else? NodeJS would be sorta viable, but event loop doesn't mix well with this kind of workload, which is a moot point anyway because Python and all the killer libs like Numpy predate NodeJS.

And I think Python's package management and parallelism are both awful in ways that could've been avoided, or fixed during that huge breaking Py2->3 change. And I prefer JS syntax. But it still works.

What would you pick?

Probably one of the math ones like Julia. Or something closer to hardware, like C++ or Rust

Julia doesn't have broad adoption because it's too specific, and C++/Rust are more cumbersome, also Python for data science and ML was popular before Rust or Julia even existed. Python's biggest advantage is that it's good enough for general usage while also being convenient for data science.

Julia has more adoption than many wannabe languages.

https://info.juliahub.com/case-studies

Sure, but not more adoption than Python. All the examples look math-focused; would you write a web backend or a batch script in Julia?

No, that wasn't why it was designed for in first place anyway.

Likewise I wouldn't use Python for anything besides teaching programming, or scripting OS tasks and native libraries.

Well that's the problem. Part of why Python is nice for ML stuff is because it was adopted for several other purposes too. I've never used Julia but have used niche things like Erlang, and there's always more friction that way.

Tcl

Reply on news.ycombinator.com