> Why am [I] doing this?
Lack of security updates past 2019 forced our hand. Did you find a way around that?
Lack of security updates past 2019 forced our hand. Did you find a way around that?
Amazon is maintaining Python 2 for at least 4 years, as part of their Amazon Linux long term support release. Google app engine will support Python 2 for an unknown amount of time; they haven't announced an end date. PyPy is Python 2, with (to the best of my limited knowledge) no plans to deprecate support. There are also other LTS releases out there which include Python 2 support.
IOW, the forcing function of the PSF no longer supporting Python is not as big a factor as was hoped.
For example, the python-saml package (for managing SAML-based single sign-on) has separate Python 2 and Python 3 versions, and implements a security-sensitive protocol which means it has (in the fairly recent past) gotten security updates for issues serious enough to rate an assigned CVE. If you're using it, having the current maintainers walk away from the Python 2 version is a serious risk...
Is Amazon planning to support pytest for at least 4 years? It will have its last 2.7-supporting release very soon.
It's particularly uncool that Guido brought up the prospect of lawyers (https://github.com/naftaliharris/tauthon/issues/47#issuecomm...) to force it not to be called Python and opposed to letting people who care about keeping Python 2 alive evolve it as "Python 2". (I know he has the legal right to insist on the name change. Still uncool.)
Besides if the Tauthon people are serious about maintaining their fork long term it needs to become more than a mere fork and a real language ecosystem of its own, in the long run having a different name will probably help with that, assuming that they ever get there.
EDIT: Also reading the rest of the thread I realize that the post that you linked out of context is slightly misleading (but I blame github's aggressive folding more than you here). Guido's answer comes after the following exchange:
stefantalpalaru: "Disregard Guido's objection. The "Python" trademark doesn't extend to "py2" or "py28". Read this for details: https://www.python.org/psf/trademarks/"
Guido: "Isn't the whole point that we're trying to solve this without lawyers?"
stefantalpalaru: "The whole point is that you've been sabotaging Python 2 for years and when someone does what needed to be done from the start, you come up with silly objections."
Guido: "OK, bring in the lawyers."
In that light, and given the other poster's ridiculously inflammatory take, Guido's answer seems rather level headed and appropriate IMO. He stands his ground, so to speak.
Please note that Perl 6 has been renamed to Raku (https://raku.org using the #rakulang tag on social media). So Perl and Raku are now considered to be different languages, albeit from the same inspiration.
Now, if Python 2 people would decide to rename Python 2 to something else, I guess it would be a mirrored parallel :-)
It's not a mirrored parallel, it's the Python folks learning from Perl's mistakes and making sure that this parallel won't come to be.
But I'm very disappointed that the Python Software Foundation isn't explictly supporting people who want to keep Python 2 compiling and running on modern systems. I think that would be well within their remit to "promote, protect, and advance the Python programming language".
This is particularly so because Python is widely used for scientific purposes, and being able to reproduce old results is valuable.
Even before Python 3.0 appeared, I came across scientists saying "I prefer to stick with Fortran because new Python versions break old code too frequently".
This case is different, because it's a project that uses the Python name, but actively adds features to the language. This is the classic example of brand confusion - someone might try to use it, find something to complain about, and PSF's reputation suffers as the result. They also get support overhead from the users of the fork (even if all they do is tell them to go away, that is still triage time that could be spent on other issues).
You can always download an old version and the respective libraries and use them to reproduce any results you want. That doesn't mean that old version should be supported anymore.
I don't see Guido as in the wrong for that. It'd be a smack in the face when you spend years trying to finally push people to switch (for better or for worse) and then a project like this takes the SEO and gets to run freely with it.
Imagine if Stroustrup had done D and insisted that it be called C++ and wanted everyone to stop using the language everyone knew as C++ on Jan 1st 2020.
They aren't stopping people from using Python 2, the language or Python 2, the software.
They are stopping people from using the name “Python” as the name of forked implementations of Python 2 not maintained by the PSF. No implementation not maintained by PSF is allowed to be called unqualified Python; the name is an important indicator of provenance. There are and have been plenty of third-party Python (2 and otherwise) implementations, the implementations just need their own names.
The effort to claim the binary name python for Python 3 is actively hostile to Python 3 and a thing that runs unmodified Python 2 unmodified on the same operating system installation. (It's unclear to me how much this is a PSF push, but at least the PEP isn't telling distros to refrain from this hostile-to-comaptibility action.)
> No implementation not maintained by PSF is allowed to be called unqualified Python
The best situation would be PSF hosting continued Python 2-compatible development by people who want to do the work.
For who? This costs the PSF manpower/overhead that they don't want to expend on a thing they don't want to maintain. It dilutes the language that the PSF are stewards of, and would further cause a schism in the python community. None of those things sounds good for python, its ecosystem, or the PSF. They sound good for, like, a few curmudgeonly companies and individuals that don't want to migrate.
I can't parse your first sentence, so I can't respond to it.
For users of the Python 2 language who have a lot of Python 2 code and for whom migration doesn't make cost/benefit sense on technical merits of Python 3.
There's Tauthon. There's Active State's long-term support for Python 2. There's presumably Red Hat's long-term for Python 2. There are probably others. Also, there's the need to keep the server side of pip up and running for these to work.
It would be great if there was a common venue for collaboration for these by the parties who are interested in keeping Python 2 going. (I'm not suggesting that Python 3 core devs should do the work.) Like a foundation for Python software.
The first sentence meant that claiming the command-line executable name python for Python 3 is hostile to letting an execution environment for Python 3 and an execution environment for Python 2 co-exist going forward without having to modify existing programs that assume that python is for Python 2 and python3 is for Python 3.
Yes, but I don't believe I've seen any (real) suggestions to change PEP 394.
> There's Tauthon.
Which I claim is actively bad for python's ecosystem in the long term. It shouldn't be supported by any organization that wants what is best for Python.
> There are probably others. Also, there's the need to keep the server side of pip up and running for these to work.
That works just fine without any help. pypi continues to support python2 tags and wheels, and I doubt that'll change anytime soon.
> There's Active State's long-term support for Python 2. There's presumably Red Hat's long-term for Python 2.
So the entire reasonable bit here is that the PSF should provide something to help various enterprise companies manage backporting security patches. Which, like, I'm not sure what infrastructure is actually needed for that. They already make security patches public. Unless you're suggesting that LTS enterprise support offerings should co-ordinate additional feature work on python 2, which is both unusual and again I claim actively harmful to the ecosystem.
If you have a large amount of Python 2 code that doesn't make sense to rewrite as Python 3 but does make sense to keep developing as opposed to just keep running as-is, it makes sense to want compatibility-preserving improvements to the language.
That such improvements are considered actively harmful comes from a point of view where there's a top-down imperative to shut down Python 2 in order to make Python 3 succeed. It's not harmful from the point of view of the code people have written in Python 2 being valuable.
The notion that there user community needs to work for Python (by porting to Python 3) and that Python 2 needs to be shut down as opposed to Python development valuing the existing code that had been developed is the core problem with Python 3.
But it really doesn't. If the new features are that valuable, you can convert your code. It's not actually that hard (I have a few 100kloc ported forward now, with millions of lines of dependencies that says so).
Any project that forks changes name:
nagios -> icigna
mysql -> mariadb
NetBSD -> OpenBSD
FreeBSD -> DragonflyBSD
Python -> PyPy, Jython, IronPython
It would be crazy for them to keep the same name and not be compatible. It would cause confusion and also lead to increase of support tickets in wrong bug trackers.
Anyway, the core problem is a top-down effort to try to make a programming language of Python 2.x’s level of usage stop to the extent it’s stoppable under its license, because its creators wanted to do something else, as opposed to facilitating its user community to pool resources to continue its development. Does the PSF have a legal obligation to do such facilitation? No. Is the lack of such facilitation bad for parties who bought into Python when it was Python 2? Yes.
They absolutely are. In fact, python 3.9 is in the works right now, which has many new evolutions beyond 2.7.
You're arguing that the psf should treat python2 and 3 as different languages. In their (any my) opinion, this is harmful. It bifurcates python into two incompatible languages. That's bad long term (Perl).
In other words, what's best for python the language, and what's best for python2 the language are not the same. And for the psf, python is more important.
I meant compatible (in the sense that old programs keep running and you can add new stuff to old programs using the new features) evolutions.
> You're arguing that the psf should treat python2 and 3 as different languages.
For practical purposes, they are different languages and the PSF has been treating them as distinct things.
> In their (any my) opinion, this is harmful. It bifurcates python into two incompatible languages. That's bad long term (Perl).
It indeed is bad. I hope that every other programming language community and designer takes a close look at what happened and makes sure never to do a Python 3 analog of their language.
> In other words, what's best for python the language, and what's best for python2 the language are not the same. And for the psf, python is more important.
That's the core problem from the perspective of Python 2 users. The organization that was the steward of the language that they invested in (in the form of writing code in the language) decided not only that a different programming language is more important for the org but that the old language needed to be shut down in order to benefit their new thing.
It's OK for people to get bored with a project and move onto something else, but with the level of usage that Python 2 had and has, it's very problematic for the language steward organization to turn around and seek to shut the language down instead of continuing to evolve it in a way that's respectful of the language users' investment in the language.
You had like 10 years of warning and it's "disrespectful"? I don't think there's a chance of productivity if you're starting from that baseline level of entitlement. Sure, mandates are annoying. But I just can't fathom that.
Java, JavaScript, C, and C++, for example don't break investment in old code like Python 3 did. They form a reasonable baseline.
There are tradeoffs.
That Python 2 and 3 can't co-exist in an app is pretty bad in comparison.
You're mistaken. I have python3 binaries and python2 binaries that share dependencies.
You're correct that fully automatic transpilation is impossible, but that doesn't mean that there can't be shared source. It does however mean that things like per-file flags or whatnot aren't possible. Python became a better language with text vs. bytes support, but that support couldn't be done in a backwards compatible way. Oh well.
> You can add Rust to your app with with rewriting all C++.
It's not as good as you seem to think. It's a nonstarter for a lot of people otherwise interested in adopting rust into existing codebases. Certainly not better than the py2/3 situation.
Kotlin interop also is troublesome, although granted better than rust/cpp or py2/3.
> That Python 2 and 3 can't co-exist in an app is pretty bad in comparison.
That python didn't get replaced by a different language is an incredible testament to the foresight of the python language stewards.
How does reusing the name facilitate the development? Every time there is a fork of an open-source project the name changes, precisely to avoid confusions. Reusing the Python name in a fork that is not just a redistribution, but a new version with new features and syntax, is just confusing, unusual and does not help anyone.
There's no `--std=python2` flag you can pass to the interpreter, unfortunately.
Indeed, C++ has rarely made any breaking changes. A decade or so ago, GCC did cause some major ecosystem breakage, by cracking down on C++ constructs which had never been valid according to the spec but which GCC had previously allowed. When that happened, there was a flag to (at least partially) revert to the old behavior: -fpermissive.
This literally does not parse. How do you know "nobody" cares about those exceptions?
Dilution of what is commonly accepted to be Python would not be a good thing, and would further add to confusion.
I know that platform upgrades are painful, but we need to move with the times or we'll all be mired in technical debt and old technology.
The whole point of Tauthon is that it is compatible with Python 2 (in the direction that old programs work).
Consider anyone who wants to build something with Python, whether it's a library, application, or service. What's better, having to build for Python 3 and 2, or just Python 3?
Thank God that Guido did this, despite knowing all the blowback he'd get. To me, that's super cool.
For example, https://blog.khinsen.net/posts/2017/11/16/a-plea-for-stabili... describes the "Molecular Modelling Toolkit (MMTK), which might well be the oldest domain-specific library of the SciPy ecosystem, will probably go away after 2020. Porting it to Python 3 is possible, of course, but an enormous effort (some details are in this Twitter thread[1]) for which resources (funding plus competent staff) are very difficult to find."
[1] The thread at https://twitter.com/khinsen/status/930749714567434240 includes "Lots of C modules written for Python 1.4 are waiting for enthusiastic code archeologists ;-)".
I don't think Hinsen is alone in that situation. I can well believe there are some people who, for example, plan to retire in about 5 years and would rather keep with with a Python 2 zombie than spend time to port working code to Python 3.
I'll admit Python 3 is still slower at a lot of things. But that feels like saying your new dog is even worse at math than your old one.
The C extension thing isn't Python's fault. It's the job of library and app authors to update. Do we complain that Vulkan has bad SunOS support? This is totally backwards.
Could Hinsen (and others) not just version their deps? It's not like people are erasing Python 2 off the internet. If his main worry is reproducibility, he should be doing that anyway.
---
I don't want to give the impression I like the whole Python 3 thing. I think it was a pretty big mistake and a huge missed opportunity. I'm very sympathetic to people who had to put in a lot of work for basically no good reason--Python 3 didn't really offer anything significantly better than 2 until... 3.5 (3.4 if you think the first pass at async was useful, I personally don't).
But I also find the ballyhooing about it really insufferable. Yeah it was a mistake; Armin Ronacher (as usual) was right. It was also over 11 years ago. Time to forget all about this and build cool stuff, please please please.
Try "python -vv -c 'pass'" - I'm only showing the first few dozen lines, and I've trimmed some of the paths for conciseness:
% python -vv -c 'pass'
import _frozen_importlib # frozen
import _imp # builtin
import '_thread' # <class '_frozen_importlib.BuiltinImporter'>
import '_warnings' # <class '_frozen_importlib.BuiltinImporter'>
import '_weakref' # <class '_frozen_importlib.BuiltinImporter'>
# installing zipimport hook
import 'zipimport' # <class '_frozen_importlib.BuiltinImporter'>
# installed zipimport hook
import '_frozen_importlib_external' # <class '_frozen_importlib.FrozenImporter'>
import '_io' # <class '_frozen_importlib.BuiltinImporter'>
import 'marshal' # <class '_frozen_importlib.BuiltinImporter'>
import 'posix' # <class '_frozen_importlib.BuiltinImporter'>
import _thread # previously loaded ('_thread')
import '_thread' # <class '_frozen_importlib.BuiltinImporter'>
import _weakref # previously loaded ('_weakref')
import '_weakref' # <class '_frozen_importlib.BuiltinImporter'>
# miniconda3/lib/python3.7/encodings/__pycache__/__init__.cpython-37.pyc matches miniconda3/lib/python3.7/encodings/__init__.py
# code object from 'miniconda3/lib/python3.7/encodings/__pycache__/__init__.cpython-37.pyc'
# trying miniconda3/lib/python3.7/codecs.cpython-37m-darwin.so
# trying miniconda3/lib/python3.7/codecs.abi3.so
# trying miniconda3/lib/python3.7/codecs.so
# trying miniconda3/lib/python3.7/codecs.py
# miniconda3/lib/python3.7/__pycache__/codecs.cpython-37.pyc matches miniconda3/lib/python3.7/codecs.py
# code object from 'miniconda3/lib/python3.7/__pycache__/codecs.cpython-37.pyc'
import '_codecs' # <class '_frozen_importlib.BuiltinImporter'>
import 'codecs' # <_frozen_importlib_external.SourceFileLoader object at 0x10dd44c90>
# trying miniconda3/lib/python3.7/encodings/aliases.cpython-37m-darwin.so
# trying miniconda3/lib/python3.7/encodings/aliases.abi3.so
# trying miniconda3/lib/python3.7/encodings/aliases.so
# trying miniconda3/lib/python3.7/encodings/aliases.py
# miniconda3/lib/python3.7/encodings/__pycache__/aliases.cpython-37.pyc matches miniconda3/lib/python3.7/encodings/aliases.py
# code object from 'miniconda3/lib/python3.7/encodings/__pycache__/aliases.cpython-37.pyc'
import 'encodings.aliases' # <_frozen_importlib_external.SourceFileLoader object at 0x10dd67d10>
import 'encodings' # <_frozen_importlib_external.SourceFileLoader object at 0x10dd440d0>
# trying miniconda3/lib/python3.7/encodings/utf_8.cpython-37m-darwin.so
# trying miniconda3/lib/python3.7/encodings/utf_8.abi3.so
# trying miniconda3/lib/python3.7/encodings/utf_8.so
# trying miniconda3/lib/python3.7/encodings/utf_8.py
# miniconda3/lib/python3.7/encodings/__pycache__/utf_8.cpython-37.pyc matches miniconda3/lib/python3.7/encodings/utf_8.py
# code object from 'miniconda3/lib/python3.7/encodings/__pycache__/utf_8.cpython-37.pyc'
import 'encodings.utf_8' # <_frozen_importlib_external.SourceFileLoader object at 0x10dd44bd0>
import '_signal' # <class '_frozen_importlib.BuiltinImporter'>
# trying miniconda3/lib/python3.7/encodings/latin_1.cpython-37m-darwin.so
# trying miniconda3/lib/python3.7/encodings/latin_1.abi3.so
# trying miniconda3/lib/python3.7/encodings/latin_1.so
# trying miniconda3/lib/python3.7/encodings/latin_1.py
# miniconda3/lib/python3.7/encodings/__pycache__/latin_1.cpython-37.pyc
matches miniconda3/lib/python3.7/encodings/latin_1
... many, many more lines omitted ...
One of the things that bugged me in Python2 was that every startup imported UserDict:
# trying python2.7/UserDict.so
# trying python2.7/UserDictmodule.so
# trying python2.7/UserDict.py
# python2.7/UserDict.pyc matches python2.7/UserDict.py
import UserDict # precompiled from python2.7/UserDict.pyc
% python2.7 -c 'import os; print(os.environ.__class__.__bases__)'
(<class UserDict.IterableUserDict at 0x1029b14c8>,)
% python3 -c 'import os; print(os.environ.__class__.__bases__)'
(<class 'collections.abc.MutableMapping'>,)
# trying python3.6/collections/abc.cpython-36m-darwin.so
# trying python3.6/collections/abc.abi3.so
# trying python3.6/collections/abc.so
# trying python3.6/collections/abc.py
# python3.6/collections/__pycache__/abc.cpython-36.pyc matches python3.6/collections/abc.py
# code object from 'python3.6/collections/__pycache__/abc.cpython-36.pyc'
import 'collections.abc' # <_frozen_importlib_external.SourceFileLoader object at 0x103c2ecf8>
It's just hard to fix.
I'm not sure the os.environ example I gave is low-hanging fruit now. The collections.abc module might be imported anyway.
This is neat! Python 3.7 added the `PYTHONPROFILEIMPORTTIME=1` environment variable to help track down these sorts of import overheads:
% env PYTHONPROFILEIMPORTTIME=1 python -c pass
import time: self [us] | cumulative | imported package
import time: 523 | 523 | zipimport
import time: 722 | 722 | _frozen_importlib_external
import time: 156 | 156 | _codecs
import time: 2254 | 2409 | codecs
import time: 1293 | 1293 | encodings.aliases
import time: 7192 | 10893 | encodings
import time: 1108 | 1108 | encodings.utf_8
import time: 182 | 182 | _signal
import time: 1069 | 1069 | encodings.latin_1
import time: 395 | 395 | _abc
import time: 1486 | 1881 | abc
import time: 1540 | 3420 | io
import time: 100 | 100 | _stat
import time: 975 | 1075 | stat
import time: 1481 | 1481 | genericpath
import time: 1734 | 3214 | posixpath
import time: 2558 | 2558 | _collections_abc
import time: 2234 | 9079 | os
import time: 1407 | 1407 | _sitebuiltins
import time: 3498 | 3498 | sitecustomize
import time: 85 | 85 | usercustomize
import time: 4129 | 18196 | site
For some of my programs, Python startup time is the main overhead. I avoid NumPy and SciPy if at all possible because they have a huge startup overhead.
Some of this is inherent in those packages. NumPy internally imports everything so someone can do "import numpy as np; np.package.subpackage.module.function()" without doing the intermediate imports.
This means NumPy is optimized for programmers (especially novice programmers) using NumPy in long-lived processes where startup cost is a negligible overhead.
Which isn't all use-cases for numeric computing.
15 years ago I supported a CGI-based web app. It was very important to pull out all the stops (delay imports until needed, use zip packages) because it was easier to do that than to re-write everything for another architecture.
The dog does count pretty well after all.
> It's the job of library and app authors to update.
Why? Linus Torvalds doesn't agree with you, for one.
As Hinsen points out,
] Unfortunately, the need for long-term stability is rather specific to scientific users, and not even all of them require it (see e.g. these two tweets by Titus Brown). So while Python 3 is probably a step forward for most Python users, it’s mostly a calamity for computational science.
Some scientific code has been able to run unchanged since the 1970s, through multiple new Fortran language releases.
Now, yes, I know the reasons for the changes to Python. I know the funding and organizational realities.
But why not recognize that for some situations Python 3 is not better?
Hinsen also comments on your proposal:
] The implication is that breaking changes in the infrastructure layers are OK and must be absorbed by the maintainers of layers 3 and 4. In view of what I just said about layer 4, it should be obvious that I don’t agree at all with this point of view. But even concerning layer 3, I find it a bit arrogant. The message to research communities with weaker code development traditions, and thus fewer resources, is that their work doesn’t matter.
> Could Hinsen (and others) not just version their deps?
He addresses that, I think. One of the other commenters gives a more complete reply at https://metarabbit.wordpress.com/2017/11/18/numpy-scipy-back... ending "Freezing the versions solves some problems, but does not solve the whole issue of backwards compatibility.".
> Time to forget all about this and build cool stuff, please please please.
I'll quote Hinsen again "I find it a bit arrogant. The message to research communities with weaker code development traditions, and thus fewer resources, is that their work doesn’t matter."
Your implicit statement is that mmtk (Hinsen's code base) isn't "cool stuff". Why? Simply because it's old, or because you don't know about it or need it? What other cool old stuff will die because it's part of a community without the resources to update?
Instead, accept that that loss is part of the trade-offs, be empathetic to those who suffer, and bear those lessons in mind for future work you do.
Second, I admit to engaging in hyperbole when I said "Python 3 is better in every way"; usually I'm on the other side of these, but I'm just so fed up with people complaining. But you're right, there are still ways Python 3 isn't "better". I'd love to have productive, technical discussions about them, but we can't seem to get beyond the "Python 3 was a super bad idea" stuff, and I'm totally uninterested in that.
But beyond that, you and I are mostly talking about different things. Python 3 isn't NumPy or SciPy. If you're building extensions on top of them, you need to look at their compatibility commitments. If you want them to make more commitments, you have to convince them. This isn't specific to software engineering; this is due diligence for anything you're gonna put years of work into.
Django's page [1] is a great example of this. Python has one too [2]. I don't have any idea about SciPy/NumPy; It looks like SciPy 1.2.0 was an LTS release supported until 1/1/2020, but what do I know.
But importantly, the end result of this "hey, do 100x the work otherwise our science won't be reproducible" stuff will be to force people out of producing free software for scientific computing. And the non-free stuff is expensive, good god. Surely this isn't what you want.
A better tactic here is to work with the developers in establishing more compatibility between releases. You probably aren't gonna get Fortran levels of compatibility--a language and platform that's seen very, very little change over the decades. But then again, the core selling point of scientific Python is that you get to use a modern platform with modern features. Asking for that along with a 50 year compatibility guarantee is a laughably tall order: you can't have it both ways without exponential amounts of work. So just like you're asking other engineers to be empathetic and respect your need for more compatibility with your extensions, you need to be more empathetic and respect their resources. And the best place to do that is probably their contact page [3], not Twitter, HN, or random blogs.
[1]: https://www.djangoproject.com/download/#supported-versions
Perhaps your "fed up"-ness means you overlook conversations which do go beyond that? Or do you put me into that category as well?
> Python 3 isn't NumPy or SciPy. ... this is due diligence for anything you're gonna put years of work into.
Hinsen's essay discussed these issues related to "software layers and the lifecycle of digital scientific knowledge". He put Python in layer 1, and NumPy/Scipy in layer 2.
In his essay he also said "I would like to see the SciPy community define its point of view on these issues openly and clearly. ... It’s OK to say that the community’s priority is developing new features and that this leaves no resources for considering stability. But then please say openly and clearly that SciPy is a community for coding-intensive research and that people who don’t have the resources to adapt to breaking changes should look elsewhere. Say openly and clearly that reproducibility beyond a two-year timescale is not the SciPy community’s business, and that those who have such needs should look elsewhere."
So I'm not convinced that we are talking about different things as you are making points I already referred to, albeit indirectly.
I'm also not sure you understood all of Hinsen's points. I say this because you wrote ""hey, do 100x the work otherwise our science won't be reproducible" stuff"
But Hinsen said "Layer 4 code is the focus of the reproducible research movement" and "the best practices recommended for reproducible research can be summarized as “freeze and publish layer 4 code” -- a solution you mentioned earlier.
It's just that reproducibility isn't the only goal for stability.
Another is to be able to go back to a 15 year old project and keep working on it, without taking the hit of rewriting it to a new, albeit similar, language.
I also have a small amount of umbrage about your comment:
> So just like you're asking other engineers to be empathetic and respect your need for more compatibility with your extensions, you need to be more empathetic and respect their resources.
I earlier wrote "Now, yes, I know the reasons for the changes to Python. I know the funding and organizational realities."
Did you overlook that because of your '"fed up"-ness', or was that not enough for you?
I do put you in that category, because you seem to be focused much more on the negative, rather than being constructive and trying to find solutions to problems.
> I'm also not sure you understood all of Hinsen's points. I say this because you wrote ""hey, do 100x the work otherwise our science won't be reproducible" stuff"
I've read and directly disagreed with his essay. His points are:
- Python 2 going away orphans a lot of software, because there's a lack of resources/willingness to port to Python 3.
- Python 3 didn't provide enough value to the scientific community to justify all the breakage (this is true for almost every community, btw).
- SciPy breaks compatibility roughly every 2-3 years, which is a bad fit for the pace of scientific computing.
- Beyond that, breaking compatibility threatens reproducibility.
- The SciPy community doesn't seem to know or care about compatibility concerns.
- Projects written on top of SciPy libraries ("Layer 3" code) have to keep updating, and they don't always have resources/willingness to do that.
- It would be cool if SciPy laid out a support schedule.
- It isn't cool that SciPy says, "hey use us", and then breaks compat all the time.
- There are some languages/platforms that haven't changed in decades, this isn't an excuse.
Here's what I've said:
- Agree Python 3 didn't provide enough value.
- If you want to build something on SciPy that you expect to last for decades, you should look for a compat guarantee. If you don't, that's on you.
- If you want new features plus decades of compat, that's a ludicrous amount of work.
- If you want to find a way forward, start a dialogue with SciPy devs.
Hinson's examples of Fortran and Java are illuminating. Fortran's a platform that's seen very minimal evolution over its history. That's exactly the reason people want to use SciPy instead of Fortran. Java's a platform with... billions of engineering hours? It's ironic that a guy who doesn't want to spend the resources to update his own software is asserting that someone else can continually deliver a modern scientific computing platform with new features while never breaking compat, they just don't feel like it ("It's all a matter of policy, not technology"). That's wrong, it's a question of resources.
---
My diagnosis here is communication breakdown. Everyone here wants the same thing: use a modern software stack for scientific computing. So again I'll say get on the mailing lists, get on IRC, go to the conferences, and talk to the engineers. Be constructive.
Guido has absolutely every right here.