I remember betting hard on python in 2008. I was all in.
I watched as the ML space embraced it as the lingua franca. It was exciting.
Best decision of my career :)
I watched as the ML space embraced it as the lingua franca. It was exciting.
Best decision of my career :)
i would agree that there is an amazing amount of productivity to be gained by reaching for mastery over a language.
that said, the efficacy of that mastery is determined by how "useful" the language is in a wider context. this is, for better or worse, highly influenced by "popularity" when it comes to programming languages.
Mastering something like brainfuck? ymmv ;)
I feel the same way as you do about python about C#/.NET, and I'm happy to have picked C#. I think for most things it would be as good as Python, e.g. web-dev, app-dev. With Xamarin it's a great choice. For data science I'd have to go running to Python or R (maybe I'd do as much in C# as possible), but luckily I don't do much data science.
I can think of a few specific projects in my early career where I wrote things in Python while other teams in my company were trying to write similar things in C++.
The Python implementations (which used Cython for a few tiny performance critical sections) ended up running much, much faster than the C++ projects (which were written by very veteran C++ engineers at the time). More than that, they were easier to test, easier to modify, and easier to teach to random new programmers joining the team. We benefited from much more user-friendly third party tools and specialized domain tools and could easily wrap any of the in-house & domain tools only accessible in C/C++.
I’m honestly really surprised. Can you provide some more detail as to how Python was beneficial for these?
Cython is a slight superset of Python that compiles to heavily optimized C or C++. So it is extremely easy to write & autogenerate Python bindings to your own C extension modules.
The difference is that it frees you to write Python (much easier to write & maintain) for everything, and only care about compiling down for speed in a few isolated places. Contrasted with writing everything in C/C++, this is a big advantage.
I've been writing Python for over a decade and I find this a bit surprising (not saying you're wrong). Python's strong suit was always speed of productivity over runtime speed IMO.
Python (CPython anyway) is nothing but a special DSL for writing C programs. Any part of Python is absolutely as fast as C if you want it to be.
In the projects I’m referring to, we just used Cython for this. Instead of wasting time writing in C for the whole project, managing memory allocation, building up abstractions over structs with macros, or using these things from inscrutable C++, we just wrote simple Python, used kernprof to measure places where running time actually mattered, and targeted those few small places with a Cython extension module.
Since the small part of performance critical code was much simpler in Python (using pretty much zero abstraction / functions only in Cython), the compiled Cython code was faster than corresponding components of C++ implementations (that took much longer to write and understand).
Most of the rest of e.g. Python standard library calls in the surrounding glue code is already C-level extension functions in Python with no measurable running time difference than C itself, and the rest of the custom Python code was measured in the profiler to be fast enough so as not to matter.
The “slow” parts of Python come from the object data model and a variety of protocols like iteration, properties and MRO. But it isn’t really fair to call it “slow” because it’s a trade-off to get those features. If you need an object that supports being iterated, having its string representation overrided, and inheriting methods from some mix-in, then you are agreeing to pay a running time cost for this. If you don’t need that stuff, you could just write what you need in e.g. Cython. With more heavily automated tools now like numba, this argument for using Python as a first choice for writing very fast code is only getting more mainstream.
Most of the time, you do need a bunch of “batteries included” features of Python and running time is not the resource constraint you’re worried about. And in the occasional exceptional cases when you need raw speed, you can use extension module strategies (e.g. how numpy was made) to be as fast as C.
For instance lists are dynamically allocated, so repeatedly appending to the list may cause it to reallocate.
lst = []
for i in range(10_000_000):
lst.append(i)
However due to the leaky abstraction, you can force allocation all at once which speeds things up, albeit at the loss of clarity.
length = 10_000_000
lst = [None] * length
for i in range(length):
lst[i] = i
However in these trivial cases Python offers more idiomatic approaches, which also perform better.
lst = [i for i in range(10_000_000)]
lst = list(range(10_000_000))
Strings behave similarly when comparing the join method to incremental appending.
string = ''
for _ in range(10_000_000):
string += 'a'
string = ''.join('a' for _ in range(10_000_000))
string = 'a' * 10_000_000
Meanwhile, interesting stuff is being done on Python, Julia and R and Rust , if you also include system programming.
Python didn't even reach the fame Perl had in the 80s to mid 2000s.
Python is for beginner programmers, who absolutely don't want to learn anything new at all.
Python is used by the largest companies as well as the smallest one person company who need to get shit done. You can continue liking Java, without becoming bitter about Python.
It was not the best decision of my career :-/
I was already firmly invested in Java, another good choice.
Another common choice was Ruby, probably not so good.
Oddly enough it started at home. I was a long time Visual Basic programmer, but I decided to try Linux on my home computer, and needed a way to write simple programs. Going a step further, I decided that I wanted my tools to be open source and platform independent. After evaluating a few tools, I settled on Python and started playing with it. I tried out things like Maxima and Octave as well.
A few months later, a big crisis landed in my lap at work, and I decided to use Python to develop the solution. I was amazed by how productive it seemed to be, and I decided to sweep more projects towards the event horizon of Python.
I've also climbed aboard the "reproducible science" movement. Though I don't publish my work, reproducibility is just as much of a problem within a closed setting. Jupyter notebooks have made it much easier for me to dredge up an old project and figure out what I did, months or even years later.