Let me introduce: __slots__
chrisbarra.xyz
chrisbarra.xyz
Please, web developers, stop doing this. It's not just the minority of people who browse with JavaScript disabled who are bothered by this, think of all the people on slow Internet connections who have to wait for your megabyte+ JavaScript program to download and execute before they can read the content of the 50 KB HTML page.
I understand that it is not feasible to accommodate no-JS browsers for Single Page Applications because JavaScript is essential to their functionality, but this page is not an application and JS is obviously not essential to it. Use JavaScript to enhance webpages, not degrade them.
- [1] Yes, Reader View in Firefox (and similar) are able to render the page properly. It's a wonderful utility, but I shouldn't have to strip away your broken web design to read your content.
- [2] https://streamable.com/b390d (no JS mirror: https://cgt.name/files/fortheloveofgod.ogv)
When all you have is a hammer and so on and so forth ...
I'm not asking developers to do extra work to accommodate no-JS browsers, I'm asking them to not do extra work to deliberately prevent sites from working without JS. The page would have worked perfectly if the author had not done extra work to add a useless loading screen. I'm asking developers to do less work, not more.
If you are going to show me a pre-loader at least inform me about what is about to be loaded. 1 second more of wait and i would have left that page.
For example, what about the growing population of users in the developing world for whom "slow" internet is the norm?
But when I do, I am painfully aware of just how much crap is included on every page. Oh, Verizon makes me painfully aware. Yes, I could get the "unlimited" dataplan, but I don't use the dataplan to make it cost effective for me (the last time a hurriane knocked the power out for me was back in 2005).
And I have it good! A few of my fellow cow-orkers have even worse cell coverage than I do.
[1] Keeping the refridgerator, freezer, a couple of lamps, a few fans, and iPhone/iPad chargers going.
You do have to not mind routing all your traffic through Opera's servers, though.
This doesn't just apply to the people with JS off.
uBlock Origin: right-click > block element... gone.
For this page, it wasn't. Holy moly, it took a rather generous amount of time until it loaded the loading spinner, followed by enough loading time that I would have closed the tab if not for my curiosity on how long it was going to take in the end. And all that for a brief article with mild syntax highlighting.
It's really only pages that use JS to create a user experience that's worse than not having it at all.
First meaningful paint: 8,820 ms
First Interactive : 11,420 ms
This would actually be a nice side project. Throw each link which was submitted to HN at a headless Chrome/Firefox, do performance/ads/size analysis, save everything to a database and then make it available to the HN user through a browser plugin which adds this info next to each submission. And then you can even display badges of shame for the worst offender or block links and other stuff. Any takers?
* neutralize clickbait * provide an instant endorsement (worth reading) or rebuttal (not) * warn of slow/heavy/ad-ridden user experience
That sounds like something the user could do, client-side.
Why do users have to blindly accept and execute Javascript payloads from every website?
What if the website said, "Hey I have some cool Javascript that can really enhance the content of this website. Do you want to download it to memory and run it on your computer, in your browser?"
The user might respond, "Sure, run it!" or "No thanks, I have some Javascript I wrote myself to enhance website content and would prefer to use that instead. Just give me the content."
I recall in the 1990's browser pop-ups where users were asked if they wanted to (download and) run Java applets. I believe there are still some websites out there that use Java applets. Not sure how browsers deal with this today.
Doubtful that web developers will listen to your plea although it is admirable. What they care about directly is browsers. The way to send them a message as a user is in the choice of what browser you use, what shows up in their logs.
Using a browser that enables all Javascript by default signifies your agreement with what they are doing today. Using a browser that does not read Javascript sends a message that you can do without it.
If the logs started showing a majority of user agents were browsers that do not read Javascript how would web developers respond?
User Agent header is still surprisingly influential.
from typing import NamedTuple
class Foo(NamedTuple):
bar: int = 0
baz: str = 'default'
foo = Foo(1, baz='not default')
The downsides are A, you can't inherit from any other class (it's syntactically valid, but silently ignored (!) though it will be caught by linters), and B, `isinstance(foo, tuple)` returns `True`, which messes with a lot of reflection code (e.g. the `json` module will not serialize them like you expect)
Edit: I stand corrected. There's no easy way (that I'm aware of) to generate an __init__ method in Python without resorting to string manipulation. Perhaps you could do it at the C level, but that defeats the purpose IMO.
The last argument should be the dict that __INIT__ is usually responsible for creating.
[0] https://jeffknupp.com/blog/2013/12/28/improve-your-python-me...
I'm planning on trying again tonight and I will probably blog my results on Friday. I'll read over that post carefully in case there's something I missed about type() and metaclasses.
Last time I approached this was generating classes for Pyramid, an it was hell, because it was incredibly poorly documented, and the community view was that metaprogramming was the devil's work.
Community has started to shift, so there's a few more resources, but things don't behave exactly as expected so you will run into issues here and there.
Hopefully you can add to the knowledge out there!
https://docs.python.org/3/library/typing.html?highlight=name...
Last I checked `collections.namedtuple` had some infamous performance implications (although I vaguely recall some discussion about changing it's implementation):
https://stackoverflow.com/questions/2646157/what-is-the-fast...
I believe that `collections.namedtuple` is beneficial for memory consumption [1], but you pay a cost when accessing members by `__getitem__`.
In general, judging by timings of the StackOverflow above, I think it is better to use `__slots__` in most cases than `namedtuple`.
[1] http://blog.explainmydata.com/2012/07/expensive-lessons-in-p...
Edit:
Also, fun fact, is that namedtuple does a clever bit of metaprogramming where a string class definition is formatted and passed to `exec`:
https://hg.python.org/cpython/file/b14308524cff/Lib/collecti...
Re-running the same tests I get:
In [1]: from typing import NamedTuple
In [2]: class A(NamedTuple):
...: a = 1
...: b = 2
...: c = 3
...:
In [3]: a = A()
In [4]: class B:
...: __slots__ = ('a', 'b', 'c')
...: def __init__(self, a=1, b=2, c=3):
...: self.a = a
...: self.b = b
...: self.c = c
...:
In [5]: b = B()
In [6]: c = dict(a=1, b=2, c=3)
In [7]: d = (1,2,3)
In [8]: e = [1,2,3]
In [9]: key = 2
In [10]: %timeit z = a.c
38.2 ns ± 0.07 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)
In [11]: %timeit z = b.c
48.1 ns ± 0.0461 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)
In [12]: %timeit z = c['c']
38 ns ± 0.062 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)
In [13]: %timeit z = d[2]
38.8 ns ± 0.0425 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)
In [14]: %timeit z = e[2]
39.8 ns ± 0.0641 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)
In [15]: %timeit z = d[key]
48.6 ns ± 0.0713 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)
In [16]: %timeit z = e[key]
49.4 ns ± 0.0369 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)
In [17]: class F:
...: def __init__(self, a=1, b=2, c=3):
...: self.a = a
...: self.b = b
...: self.c = c
...:
In [18]: f = F()
In [19]: %timeit z = f.c
53.7 ns ± 0.265 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)It doesn't appear as if `namedtuple` has changed much:
https://hg.python.org/cpython/file/tip/Lib/collections/__ini...
Running the same tests from the post, I get similar results:
from collections import namedtuple
STest = namedtuple("TEST", "a b c")
a = STest(a=1,b=2,c=3)
class Test(object):
__slots__ = ["a","b","c"]
a=1
b=2
c=3
b = Test()
c = {'a':1, 'b':2, 'c':3}
d = (1,2,3)
e = [1,2,3]
f = (1,2,3)
g = [1,2,3]
key = 2
if __name__ == '__main__':
from timeit import timeit
print("Named tuple with a, b, c:")
print(timeit("z = a.c", "from __main__ import a"))
print("Named tuple, using index:")
print(timeit("z = a[2]", "from __main__ import a"))
print("Class using __slots__, with a, b, c:")
print(timeit("z = b.c", "from __main__ import b"))
print("Dictionary with keys a, b, c:")
print(timeit("z = c['c']", "from __main__ import c"))
print("Tuple with three values, using a constant key:")
print(timeit("z = d[2]", "from __main__ import d"))
print("List with three values, using a constant key:")
print(timeit("z = e[2]", "from __main__ import e"))
print("Tuple with three values, using a local key:")
print(timeit("z = d[key]", "from __main__ import d, key"))
print("List with three values, using a local key:")
print(timeit("z = e[key]", "from __main__ import e, key"))
Named tuple with a, b, c:
0.147349834442
Named tuple, using index:
0.0364408493042
Class using __slots__, with a, b, c:
0.0334758758545
Dictionary with keys a, b, c:
0.0339260101318
Tuple with three values, using a constant key:
0.0425179004669
List with three values, using a constant key:
0.0301761627197
Tuple with three values, using a local key:
0.0436310768127
List with three values, using a local key:
0.0309669971466In Python, I think it's important to do what's directly semantically correct.
However, we can compare apples and oranges on what they have in common, so here's a bit of my own analysis:
>>> from timeit import repeat
>>> from collections import namedtuple
>>> class Slotted: __slots__ = 'a', 'b', 'c', 'd', 'e'
...
>>> NT = namedtuple('NT', 'a b c d e')
>>>
>>> nt = NT(1,2,3,4,5)
>>> s = Slotted()
>>> s.a, s.b, s.c, s.d, s.e = 1,2,3,4,5
>>> min(repeat(lambda: (s.a, s.b, s.c, s.d, s.e)))
0.2850431999977445
>>> min(repeat(lambda: (nt.a, nt.b, nt.c, nt.d, nt.e)))
0.418955763001577
Only if you're too slow, then you look for ways to optimize for bottlenecks.
I also feel compelled to point out that __slots__ changes the way objects are initialized [1]:
> By default, instances of both old and new-style classes have a dictionary for attribute storage. This wastes space for objects having very few instance variables. The space consumption can become acute when creating large numbers of instances. > > The default can be overridden by defining __slots__ in a new-style class definition. The __slots__ declaration takes a sequence of instance variables and reserves just enough space in each instance to hold a value for each variable. Space is saved because __dict__ is not created for each instance.
Which means that, in actuality, simply defining __slots__ changes the nature of the object.
Quick trial demonstrates this:
>>> class mySlottedObj(object):
... __slots__ = ('a', 'b')
... c = 1
...
>>> x = mySlottedObj()
>>> x
<mySlottedObj object at 0x10e6a3510>
>>> x.c
1
>>> x.__dict__
Traceback (most recent call last):
File "<input>", line 1, in <module>
AttributeError: 'mySlottedObj' object has no attribute '__dict__'
>>> x.__weakref__
Traceback (most recent call last):
File "<input>", line 1, in <module>
AttributeError: 'mySlottedObj' object has no attribute '__weakref__'
>>> x.a = 1
>>> x.c = 2
Traceback (most recent call last):
File "<input>", line 1, in <module>
AttributeError: 'mySlottedObj' object attribute 'c' is read-only
_field_template = '''\
{name} = _property(_itemgetter({index:d}), doc='Alias for field number {index:d}')
'''
But at any rate, the benchmark script was not mine and refining the test to be as close to the same code path is welcome.
And I wholeheartedly agree:
> Only if you're too slow, then you look for ways to optimize for bottlenecks.
I was merely responding to the original assertion:
> This provides the same performance benefits, and is very similar in a lot of ways to a Scala case class -- aside from removing the boilerplate of stating each attribute twice in __init__ and again in __slots__,
And I think at this point it is quite clear that namedtuple has a bit more caveats than just "removing the boilerplate" of __slots__.
The references I link I think go into better detail than I can.
If you're not using "setattr" to dynamically add fields to your object, CPython's underlying machinery is overkill. Everything is a dict, and all accesses take a lookup. With __slots__, a Python object is more like a struct in other languages.
Slotted objects should be the default. You should have to inherit from "DynamicObject" or something like it to enable setattr on an object. Actually, dynamic objects stopped being that important once Python let you inherit from "dict". Then, if you wanted "foo[ix]", it was easy to do it.
There are some packages, such as BeautifulSoup, which use dynamic attributes heavily. BeautifulSoup uses it to map HTML tags to attributes. This is more trouble than it's worth, because when an HTML tag name clashes with a Python builtin name, workarounds are necessary.
this would make python a different language completely
The argument is that these objects are a sane default that would prevent a lot of accidental programming mistakes and I agree.
It's also a great way to introduce some static typing. No more setting mistyped class attributes without errors!
I've always found it quite curious that Python made this explicit rather than an implementation detail (I get that there are slight semantic differences here in Python too) because it seems like such a useful optimization. And a similar approach can be used for method lookup too.
I wonder (and I don't have any measurements on this myself) what the relative benefits are to explicitly picking one over the other vs. automating it are. The downside of doing it automatically is that some types of objects might very well be very "sparse" in that it's not a given that all instance variables are used for all objects (this is more applicable in for vtables, especially in languages with single-rooted object hierarchies).
My intuition is that the overhead of a map/dict/hash table to store them is likely to usually outweigh the cost of quite a lot of unused instance variables, and so that inferring instance variables is generally likely to be an improvement.
About Python, it's a little older than Ruby (its design started in 1989) and Guido worked on the ABC language at the beginning of the 80s. IMHO that difference shows in many places, for example in having to explicitly pass self as argument in method definitions. Newer languages do without it and programmers don't get confused at all. All those double underscore methods and the general explicitness of the language smell of 80s and of C. Not necessarily a bad thing but reading some Python code is like getting on a time machine.
What you say makes sense, and I understand that it takes a lot of effort to clean those type of things up without hurting backwards compatibility.
This specific distinction though seems like one where it'd still be possible to get most of the benefit by changing the implementation of __dict__ to make it do something similar to __slot__ under the hood, but fall back to a dict for dynamic properties.
I have also written up __slots__ in great detail on Stack Overflow.[1]
I'm at a meetup right now, but I can try to answer any quick questions here tonight.
[0] https://docs.python.org/3.7/reference/datamodel.html#object....
I had a project where I had to connect to a data source and serialize some data. When I needed to add an attribute all I had to do was add it to two places: the class itself, and the @classmethod constructor. So in this case I got the serialization for free, but that's all I was using. In reality I also got __repr__, __cmp__, etc, etc for free too, I just wasn't using it. Serialization was free because attr.asdict() knows which attributes are attr.ib() attributes.
@attr.s()
class Foo:
bar = attr.ib()
spam = attr.ib()
eggs = attr.ib() # added this line
@classmethod
def from_something_else(cls, x):
return cls(
bar=x.bar.name,
spam=x.spam,
eggs=x.get_eggs().blah(), # and this line
)
with open('blah.json', 'w') as fout:
json.dump(fout, attr.asdict(o)) # got serialization for freeBy the way, I like the approach taken in JavaScript engines such as V8, which determine the "slots" dynamically.
**json.loads(my_json))
with_slots = [get_size(MyUserWithSlots(**json.loads(my_json))) for _ in range(NUM_INSTANCES)]
def foo(a, b, c):
pass
# Arguments passed positionally:
foo(1, 2, 3)
# or passed by keyword:
foo(a=1, b=2, c=3)
a_dict = {'a': 1, 'b': 2, 'c': 3}
foo(**a_dict)
# because of the value of a_dict, expands to:
# foo(a=1, b=2, c=3)
[1]: https://stackoverflow.com/questions/1419046/python-normal-ar...
[2]: https://docs.python.org/2/tutorial/controlflow.html#keyword-...
[3]: https://docs.python.org/2/reference/expressions.html#grammar...