Maybe they don't give it enough time in school for people to realize it is the king of practical software development.
For instance `a = foo.bar.baz` in Python involves 3 hash gets (local scope, foo scope, then bar scope), and a single set operation (local scope). This is part of the reason Python programs can be optimized by assigning a deep attribute lookup to the local scope outside of a loop's scope, and it will yield improved performance relative to doing the deep attribute lookup inside the loop's scope.
a = foo.bar.baz
for _ in range(20):
print(a)
for _ in range(20):
print(foo.bar.baz)Can you explain where exactly and why a set operation is performed? Thanks.
Basically `a = 1` is syntactic sugar for `locals()['a'] = 1`
>>> a
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'a' is not defined
>>> locals()['a'] = 113
>>> a
113
>>>
For example:
>>> locals()['foo-bar'] = 1
>>> locals()['foo-bar']
1
>>> foo-bar
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'foo' is not defined
>>>
Global variables use a dictionary, however. The disassembly actually looks similar for both.
def f():
a = 5
locals()['a'] = 6
print(a) # 5
https://github.com/python/cpython/blob/master/Python/ceval.c...
`a = ... ` can be thought of as `locals()["a"] = ...`
(although you can should not actually modify the local variables this way)
1 0 LOAD_NAME 0 (foo)
2 LOAD_ATTR 1 (bar)
4 LOAD_ATTR 2 (baz)
6 STORE_NAME 3 (a)
8 LOAD_CONST 0 (None)
10 RETURN_VALUE
So, the expression 'a = foo.bar.baz' involves at least three hash table lookups and one insert or update.
You can have dynamic binding (assuming that's what you meant, dynamic linking is something else) with way fewer hash lookups.
One example that was brought up elsewhere in the thread is the way python looks up variables in successive scope dictionaries. This is obviously terrible for performance, and that's a big part of why Python is slow.
But how are other languages fast? Doesn't every language have to resolve variable names to memory locations? Well, yes, but fast languages do this mostly at compile time. How? Well, I'm not an expert in this, but at a high level, each variable is assigned a location in memory or registers, and then future references to that variable are rewritten to refer to the memory location by register name or memory address. This takes the whole "looking up a name" issue out of the path of work that has to get done at runtime. And you've switched from looking up things in tables to operating directly on registers and memory locations.
BTW, this has nothing to do with high-versus-low level. It's more about how mutable the process of name resolution is after program compilation. One could theoretically write an assembly language where memory locations are names, not numbers. If reflective features like runtime scope editing are available, this would be a very low-level language that still requires some kind of dictionary lookup at runtime.
A lot of devs are completely unaware of what's happening at the layer of abstraction below them and this is one of the ways that comes out. The number of elements it takes before hash tables are faster than iterating through an array can be surprisingly large and yet it's one of the first "optimizations" that get made.
Some other related problems are not knowing how expensive network calls are, not knowing what the ORM is doing, caching in memory even though the database already is and more. They just don't think about performance issues in code they don't write.
Performance oriented devs should be concerned with bottlenecks, not incredibly minute details. There's almost no situation I can think of where smallish dictionaries are much better or worse than any other data structure when it comes to performance.
Of course, if you're writing a compiler then it can be a serious difference. Most developers don't write compilers though.
If your program is sensitive to that, even a simple GC pause is going to destroy your performance and you need to be out of managed memory languages at that point.
There are a lot of reasons python can be slow, but this is far from one of them.
It won't show up in the hot path because the performance cost so pervasive that profiling tools will ignore it, it's everywhere and you can't escape it without compiler optimizations. This cost will be in the hot and cold paths. This and data locality are the two biggest performance issues in dynamic languages and a lot of effort goes into compiling it out.
Here is a good article on how V8 tries to deal with it: https://www.html5rocks.com/en/tutorials/speed/v8/
For statically compiled languages it can show up but often you'll have to write a dictionary free version to see it. Some profiling I've done with c# at various times over the years shows that it's slower than a list until you have more than 50 to 100 items. The caveat is that I normally keep the dictionary version because it's more semantically correct.
Python does this for local variable names in functions. Because of this, moving a tight loop from global scope to a function can make it a couple percent faster.