Code that assumes that something is going to be atomic because of the GIL (or any other implementation detail) is simply broken. If you need something to be atomic you should be explicit about that and use mutex or something.
One study [1] in the US released in 2020 found almost 90% of people admitted to speeding, but I don’t think anyone would say that speeding is now approved by the authorities and consequence-free.
[1] https://www.thezebra.com/resources/research/speeding-car-ins...
Depends on the definition of "correct".
One way to define "correct" is "as defined by the standard". However, standard serves a purpose - to maintain interoperability between components, so the other, deeper, way to define "correct" is "interoperable with the ecosystem".
The official standard can argue that relying on the implementation detail is incorrect because they didn't specify it explicitly, but they would go against the grain of the rest of the ecosystem. Which reminds me of an old joke:
An old woman is watching the news. She sees a news report saying there is a car driving in the wrong direction on the highway. So the old woman calls up her husband.
Old woman: be careful on the highway dear, there is a crazy driver on the highway driving the wrong way!
Husband: It’s not just one car, it’s hundreds of them!The "standard" is a combination of PEPs, the Python docs, and the CPython implementation.
Implementation details are language features, because implementation details are the standard. Python programs that rely on the GIL are not "wrong". Such programs are relying on clearly-documented features of the system that they use.
Is it "wrong" to use GCC-specific features in a C program, if you know that you only intend to target GCC?
The problem here is: Implementation details are not guaranteed to be stable. They can change with every release.
So if I write code that relies on any particular implementation detail, it may be correct today, and may be wrong two weeks from now, even though I didn't change anything.
On the other hand, Linux kernel developers have also been burned by GCC changing what it does regarding undefined behavior (the infamous not-so-corner cases of C/C++). They expect GCC to be sort of like an assembler, which does the most straightforward thing in case of ambiguities. Instead, GCC is an optimizing compiler that is expected to deliver high-performance code for general-purpose programs. It treats undefined behavior as opportunities to enact optimizations. What it actually does is also usually documented, but also here different people have different expectations about what that implies.
The GIL is an implementation choice that was put in place by Python's developers for simplicity's sake. It is important to be aware of it because it has huge performance implications, but it is unwise to rely on it for semantics. Especially since it has not exactly been a secret that various parties would eventually like it to be removed. Anyways, the GIL has very little to do with Python's semantics. User code is racy with or without the GIL, which is actually the point of TA.
Just hard as hell to change
Hyrum's Law says the opposite:
"With a sufficient number of users of an API, it does not matter what you promise in the contract: all observable behaviors of your system will be depended on by somebody."
Named after some guy called Hyrum who worked / works at Google.
I would say jaywalking in NYC is a much better example because breaking the law doesn't kill others except in rare cases. Jaywalking is illegal. It's also not enforced for all intents and purposes. Every attempt to enforce it has led to such loud public outcry that it was stopped.
e.g.; there are C-compilers that usually zero most allocated structures. That's an implementation detail however, not a feature of C. C doesn't guarantee you zeroed memory anywhere, and code that assumes otherwise is just one compilation away from a desaster.
IOW, the python implementation is the standard, and therefore any code that relies on an implementation detail in the reference implementation is, by definition, correct.
That's not quite strictly true, Python does have a documented "Language Reference": https://docs.python.org/3/reference/index.html
If there is a contradiction between the Language Reference and CPython then one, or both, of them needs to be updated and it's treated on a case by case basis.
If an alternative Python implementation follows the Language Reference but chooses different details outside it, that doesn't stop it from being "Python". Of course practically speaking most alternative implementations are incentivized to closely follow CPython.
That's a wild one
from FAQ:
Q: But isn't a language that deletes code crazy?
The library is documented, and the syntax is also documented, but the semantics themselves are not.
The language reference at https://docs.python.org/3/reference/index.html "describes the syntax and “core semantics” of the language.".
There has been a distinction between Python-the-language and CPython-the-implementation ever since JPython back in the 1990s. For example, reference counting is a CPython implementation details. The reference manual says only:
> Objects are never explicitly destroyed; however, when they become unreachable they may be garbage-collected. An implementation is allowed to postpone garbage collection or omit it altogether — it is a matter of implementation quality how garbage collection is implemented, as long as no objects are collected that are still reachable.
(Quoting https://docs.python.org/3/reference/datamodel.html )
A standards document or specification's purpose is to "prescribe" how a system shall work in order to be compliant as opposed to a reference which documents how systems currently happen to work.
For example, the following operations are all atomic (L, L1, L2
are lists, D, D1, D2 are dicts, x, y are objects, i, j are ints):
L.append(x)
L1.extend(L2)
x = L[i]
x = L.pop()
L1[i:j] = L2
L.sort()
x = y
x.field = y
D[x] = y
D1.update(D2)
D.keys()Writing to spec is very rare in my experience. You usually learn the spec once someone tells you your code doesn't work on XYZ.
Assuming sequential consistency is pretty broken. Assuming acquire/release atomicity is much more reasonable. Assuming "at least relaxed" is outright mandatory.
If major libraries and user code are constantly incorrect, but work because of an implementation detail, then removing that implementation detail becomes extraordinarily difficult, verging on impossible.
It would be like retrofitting your language to distinguish valid unicode text from arbitrary byte strings, when you'd previously treated them as equivalent.
For a sufficiently popular project there are often cases where users (other projects) rely on existing observed behavior even if it not guaranteed by documentation/specification.
This doesn't count?
> Consequently, if you were coming from Mars and tried to re-implement Python from this document alone, you might have to guess things and in fact you would probably end up implementing quite a different language. On the other hand, if you are using Python and wonder what the precise rules about a particular area of the language are, you should definitely be able to find them here. If you would like to see a more formal definition of the language, maybe you could volunteer your time — or invent a cloning machine :-).
Depends on the question. If your question is, say, "how should I implement the built-in types", yes, the language reference won't answer that question. But if your question is "what does my implementation have to do to count as an implementation of the Python language", then yes, the language reference does answer that question--since the language reference is what defines the Python language.
To me that is a "specification" of the Python language. It's not a specification of the implementation of the language, but why should that be required for something to be considered a language specification? The whole point is to specify what is required to define the language without specifying every detail of the implementation.
Yes, I know that statement is in the Introduction, but I think it's rather ill-considered. If my implementation is consistent with the language reference, on what grounds would someone claim it was not an implementation of Python but a "different language"?
I would like to see some indication from the people who say that the Python Language Reference I gave a link to does not qualify as "language specification", of what would qualify. Specific examples would be nice.
I don't see why not. The "Introduction" section specifically mentions different implementations and distinguishes implementation details, which can vary by implementation, from the language reference itself, which defines what every implementation has to meet to be considered an implementation of the Python language.
> my point in the GP was that something distinct called “the Python Language Specification” doesn't exist
And that's the point I'm disputing; AFAIK the language reference I linked to is that something distinct, even if it isn't called a "Language Specification" but instead a "Language Reference". Either way it defines what the Python language is.
Not sure why there is an argument here.
No, we don't. I have already said explicitly that I think the Python Language Reference is such a thing. (I am ignoring quibbles about it being called a "reference" instead of a "specification".) If you think it isn't, why? What does count as a "language specification" in your view? Do you have any specific examples that you can contrast with Python?
Yes, there is:
https://docs.python.org/3/reference/index.html
This specification does not mention the GIL anywhere, which means it is, as the GP said, an implementation detail. Other implementations of Python that do not have the GIL are still "Python" implementations because they meet this language specification.
I mean, I can see the point if there are multiple commercial competitors in the market, as there is with C/C++, or if the implementation is proprietary and the users want to avoid vendor lock-in.
But the Minimal BASIC of ANSI X3.60-1978 never did catch on for any of the BASICs I used in the 1990s, and the Full BASIC of ANSI X3.113-1987 was a flop, so clearly it's possible to put a lot of time into a standard only to have it be irrelevant.
Python does not have a specification or a standards document, it has a reference that describes how Python happens to work and the reference is a great resource for people to familiarize themselves with the language but it should simply be clear that its purpose is to reflect the existing state of the language rather than to specify how Python works.
I've already given my answer: they all meet the language reference I linked to (yes, the word "reference" appears in its title, not "specification"; that's just another quibble).
What is your answer?
Python is a lot more like LISP than it is C++. There are many flavors of Python, from MicroPython to GraalPython to Cyston and literally dozens of them. They most certainly do not all meet the language reference you linked to and they all have quirks here and there.
A language does not need a specification in order to exist or to have a name. What matters is that people use it and get work done with it, and ultimately if looks like a Python and quacks like a Python, then it's fine to call it a Python.
Here's another fact: Python doesn't have its own typeface either. Yet that fact is hardly germane to the thread.
A prescriptive document is a formal specification. The Python Language Reference is a less formal specification. It's still a specification.
It is also not a single document, but then again even a formal specification may incorporate other specifications by reference.
As my position is that the CPython implementation is the reference implementation for Python and as the GIL is an integral part of that implementation, then the GIL does form a part of Python's semantics regardless of whether the reference documentation mentions it or not.
The Python reference documentation is not a specification and it's not intended to be one.
You're quibbling. By your definition, no programming language has a "specification". Every language has implementations that do things that aren't explicitly described in any document.
Some languages do not, such as Python and Rust.
The ISO C++ committee even uses quite strong language about the C++ standard and makes it a point to differentiate between the C++ specification and C++ references:
>The standard is not intended to teach how to use C++. Rather, it is an international treaty – a formal, legal, and sometimes mind-numbingly detailed technical document intended primarily for people writing C++ compilers and standard library implementations.
Same question for Java and C#.
If the answer to all of these questions is "no", as I believe it is, on what grounds do you claim that these languages have a specification, while Python and Rust do not?
The grounds that I claim is that you can read them, here they are:
https://docs.oracle.com/javase/specs/
https://isocpp.org/files/papers/N4860.pdf
Note what the actual C++ standard states, and I quote:
>This document specifies requirements for implementations of the C++ programming language. The first such requirement is that they implement the language, so this document also defines C++. Other requirements and relaxations of the first requirement appear at various places within this document
The number of Google hits I get when I search on "C++ implementations that do not meet the language specification" does not seem to support this claim.
> Note what the actual C++ standard states
But your position with respect to Python is that it doesn't matter what the "standard" states because the actual definition of the language is in its reference implementation. Why are you now shifting your ground?
That is an incredibly weak and ill-informed method for vetting technical matters.
>Why are you now shifting your ground?
Python doesn't have a standard.
So is your claim that, because a "standard" says something and claims it's a "specification", that is sufficient to guarantee that every implementation does exactly what the "standard" or "specification" says.
> Python doesn't have a standard
Again, you're quibbling. It has a language reference, which I linked to, and which is what is used to determine what implementations count as Python implementations.
At this point I don't think we have enough common ground for a useful discussion.
It's the other way around, a standard specifies the requirements of a conforming implementation, and then an implementation can claim conformance with respect to that standard. With C++, many well-known implementations do claim conformance to the standard such as MSVC, GCC and Clang. With Python no one makes such a claim, not even the CPython reference implementation, because there is no standard to conform to. On the contrary what you do find are implementations that try to be compatible with CPython reference implementation specifically, such as PyPy and Pyston but no one claims to conform to the Python reference manual. The reference document simply describes how some Python implementations happen to work, but it does not prescribe how a Python implementation is required to work in order to be in compliance.
If tomorrow CPython decides to add a new feature to the language, then the reference will be updated to include this change because the reference is a reflection of the implementation.
If tomorrow the C++ specification changes, then it's GCC or any compiler that claims conformance that will update to include that change because with C++ it's the implementation that is a reflection of the specification.
That's the key difference between a descriptive document and a prescriptive document. This may seem like a quibble as you put it, but quibbles can lead to multimillion dollar lawsuits as Microsoft learned in the 90s when they claimed to have an implementation of the Java specification and then were sued by Sun Microsystems because Microsoft's implementation did not in fact conform to the Java standard.
Microsoft ended up paying 20 million dollars over this so called "quibble":
https://en.wikipedia.org/wiki/Microsoft_Java_Virtual_Machine...
Others object saying that the Python reference document is what specifies its semantics, not the reference implementation.
My position is that both the CPython reference implementation and the reference documentation are valid sources that document Python's semantics and that they both can and should be used.
Implementation details are documented to allow performance optimizations and to give insight into why certain things are how they are. Therefore, users can reasonably expect that the vendor won't cause performance regressions for existing code. However, it is unwise to derive semantics from them, even if they are technically documented in that way.
One of the biggest disadvantages of relying on implementation details is that it makes it way more troublesome for the vendor to maintain and improve the product.
Anyways, the GIL and the presence of possible concurrency bugs are completely orthogonal things as the GIL has always only served to prevent corruption of runtime data structures, not of user code.
And my position is that it is not because there are other implementations of Python that do not have it, and which everybody agrees are implementations of Python.
Not only that, but the very language reference that I linked to explicitly distinguishes CPython implementation details from the language itself. So the Python dev team does not appear to agree with your position.
What would it mean to have a language standard? A publication from ISO or ECMA?
I ask because the Python Language Reference at https://docs.python.org/3/reference/index.html seems to be a (terse) language standard. Among other things, it highlights some of the things which are implementation defined, rather than language defined.
It's an implementation detail, and relying on those for functionality is a great way of getting ones code to break.