HNHacker News
TopNewBestAskShowJobs

raymondh

2,430 karma · joined February 9, 2010

Highlights:

  * CEO of Mutable Minds, Inc. (a training and consulting company)
  * Python Distinguished Service Award
  * Past Python Software Foundation (PSF) board member
  * Private pilot (single engine land and sailplanes)
  * Certified Public Accountant
  * Certified Internal Auditor
  * Python Core Developer
  * Husband to Rachel
  * Father to Matthew
Python training video series, "Modern Python: Big Ideas, Little Code" available for free to Safari On-line subscribers: https://www.oreilly.com/library/view/modern-python-livelessons/9780134743400/

Videos for conference presentations and keynotes can be found at: https://www.youtube.com/playlist?list=PLRVdut2KPAguz3xcd22i_o_onnmDKj3MA

Contributions to Python:

  * any()
  * all()
  * enumerate()
  * local mapping argument to exec() can be any mapping
  * filter() -- originally itertools.ifilter
  * frozenset()
  * map() -- originally itertools.imap
  * reversed()
  * sum() -- optimized and made more accurate
  * zip() -- originally itertools.izip
  * sorted()
  * set()
  * dict.fromkeys
  * dict.pop
  * float.as_integer_ratio
  * str.format_map
  * str.partition
  * str.rpartition
  * tuple.count
  * tuple.index
  * abc.ABC
  * antigravity.geohash 
  * bisect module -- key function support
  * calendar module -- various fixups
  * collections.ChainMap
  * collections.Counter
  * collections.OrderedDict
  * collections.deque
  * collections.namedtuple
  * contextlib.redirect_stderr
  * contextlib.redirect_stdout
  * contextlib.suppress
  * decimal module
  * difflib.context_diff
  * difflib.unified_diff
  * fractions.limit_denominator -- with Mark Dickinson
  * functools.cache
  * functools.cmp_to_key
  * functools.lru_cache
  * functools.total_ordering
  * heapq.heappushpop
  * heapq.merge
  * heapq.nlargest
  * heapq.nsmallest
  * itertools.accumulate
  * itertools.chain
  * itertools.chain.from_iterable
  * itertools.combinations
  * itertools.combinations_with_replacement
  * itertools.compress
  * itertools.count
  * itertools.cycle
  * itertools.dropwhile
  * itertools.filterfalse
  * itertools.groupby
  * itertools.islice
  * itertools.pairwise
  * itertools.permutations
  * itertools.product
  * itertools.repeat
  * itertools.starmap
  * itertools.takewhile
  * itertools.tee
  * itertools.zip_longest
  * json.AttrDict (later removed)
  * math.dist
  * math.fsum -- with Mark Dickinson
  * math.hypot -- improve algorithm and expanded to n-dimensional case
  * math.log -- added the "base" optional argument
  * math.prod -- implemented by Pablo Galindo
  * operator.attrgetter
  * operator.itemgetter
  * queue.LifoQueue
  * queue.PriorityQueue
  * queue.join
  * queue.task_done
  * random's MersenneTwister
  * random.SystemRandom
  * random._randbelow
  * random.binomialvariate
  * random.choices
  * random.sample
  * random.triangular
  * statistics.correlation -- added spearman rank correlation coefficient
  * statistics.fmean
  * statistics.geometric_mean
  * statistics.harmonic_mean -- added weights parameter
  * statistics.kde
  * statistics.kde_random
  * statistics.linear_regression -- added direct proportion option
  * statistics.mode -- fixed api
  * statistics.multimode
  * statistics.NormalDist
  * statistics.quantiles
  * statistics.variance/pvariance/stdev/pstdev -- correctly ounded
  * tokenize.untokenize
  * userdict.DictMixin -- eventually became ollections.MutableMapping
  * __slots__ can be a dictionary
Designer of:

  * key-functions
  * generator expressions
  * compact and ordered dictionaries
  * the peephole optimizer
  * __length_hint__ optimization
  * the thousands separator in string formatting
  * original proponent of exception chaining
  * original proponent of sending data and exceptions into enerators
  * eval-loop opcode prediction
Maintainer of:

  * the argparse module (successor to Steven Bethard)
  * the bisect module
  * the collections module
  * the collections.abc module
  * the decimal module
  * the heapq module
  * the itertools module
  * the queue module
  * the random module
  * the statistics module (with Steven D'Aprano) 
Author of:

  * Python How-to: Descriptor Guide
  * Python How-to: Sorting Techniques
  * Python tutorial: Looping Techniques
  * Python tutorial: Tour of the standard library
  * Python tutorial: Floating Point Arithmetic
  * Blog post: Super Considered Super
  * Whatsnew 3.1
  * Whatsnew 3.2
  * Whatsnew 3.8
Social media:

  * https://twitter.com/raymondh
  * https://rhettinger.wordpress.com/
  * https://www.facebook.com/raymondh
  * https://www.linkedin.com/in/raymondhettinger/
  * https://stackoverflow.com/users/1001643/raymond-hettinger
submissionscomments
raymondh··on Cleve Moler has died
In the 1990s, I briefly met Cleve while taking a two-week Matlab course in Natick. During that course that a classmate introduced me to Python, a language where I later became a core developer.

Cleve's papers were an inspiration. I soon published my own matrix package called matfunc. That work was heavily influenced by Cleve Moler and by algorithms in Golub and Van Loan. Even my more recent Python contributions, like the super accurate math.fsum(), math.hypot(), and math.sumprod() functions, have their roots in that fertile time in the Matlab ecosystem. In particular, it newsgroups and lists of papers taught me Cleve's never ending quest to create clean front-ends for numerically sophisticated code.

Thank you Cleve. Your legacy will live forever.

raymondh··on Tim Peters – Dispelling Information Asymmetry
For those who don't know the name, Tim Peters is the author of "The Zen of Python". He is the one who uniquely captured was Python is all about with this inspirational little poem:

    The Zen of Python, by Tim Peters

    Beautiful is better than ugly.
    Explicit is better than implicit.
    Simple is better than complex.
    Complex is better than complicated.
    Flat is better than nested.
    Sparse is better than dense.
    Readability counts.
    Special cases aren't special enough to break the rules.
    Although practicality beats purity.
    Errors should never pass silently.
    Unless explicitly silenced.
    In the face of ambiguity, refuse the temptation to guess.
    There should be one-- and preferably only one --obvious way to do it.
    Although that way may not be obvious at first unless you're Dutch.
    Now is better than never.
    Although never is often better than *right* now.
    If the implementation is hard to explain, it's a bad idea.
    If the implementation is easy to explain, it may be a good idea.
    Namespaces are one honking great idea -- let's do more of those!
Also, he is the author of the famous TimSort algorithm: https://en.wikipedia.org/wiki/Timsort
raymondh··on Lesser known parts of Python standard library
The reason Guido didn't want 3.6 to guarantee dict ordering was to protect 3.5 projects from mysteriously failing when using code that implicitly relied on 3.6 behaviors (for example, cutting and pasting a snippet from StackOverflow).

He thought that one cycle of "no ordering assumptions" would give a smoother transition. All 3.6 implementations would have dict ordering, but it was safer to not have people rely on it right away.

raymondh··on AnandTech Farewell
Thank you AnandTech. Happy ride in to the sunset.
raymondh··on Seven basic rules for causal inference
Is there a simple R example for Rule 4?
raymondh··on Python extensions should be lazy
We should probably get rid of that. It is old (predating numpy) and has limited functionality. In almost every case I can think of, you would be better off with numpy.
raymondh··on Python extensions should be lazy
This is an impressive post showing some nice investigative work that isolates a pain point and produces a performant work-around.

However, the conclusion is debatable. Not everyone has this problem. Not everyone would benefit from the same solution.

Sure, if your data can be loaded, manipulated, and summarized outside of Python land, then lazy object creation is a good way to go. But then you're giving up all of the Python tooling that likely drove you to Python in the first place.

Most of the Python ecosystem from sets and dicts to the standard library is focused on manipulating native Python objects. While the syntax supports method calls to data encapsulated elsewhere, it can be costly to constantly "box and unbox" data to move back and forth between the two worlds.

raymondh··on Millions of Taxpayers Call the IRS for Help. Two-Thirds Don't Reach Anyone
We've had to contact the IRS a number of times. It was always difficult to get through. However once we did get through, the representatives were polite, professional, and really seemed to care about achieving case resolution.
raymondh··on Ask HN: What brought back the joy of programming for you?
Programming Pearls, Thinking Forth, The Little Schemer, and SICP
raymondh··on The Numeric Tower Fiasco
As described by Alan Kay, what OOP means is encapsulation, messaging, and late-binding.

IIRC the inspiration came from multicellular organisms. A cell "encapsulates" complexity within a cell wall. The organism as a whole works by have the cells work together via "messaging". Any shared interfaces (e.g. oxygen transpiration and nutrient absorption) can be viewed as "polymorphism".

If you buy into the definition and biological analogy, then any language that implements "encapsulation, messaging or polymorphism" actually is OOP and your question is a tautology.

raymondh··on More Itertools
Your nice work on the JS itertools port has a todo for a "better tee". This was my fault because the old "rough equivalent" code in the Python docs was too obscure and didn't provide a good emulation.

Here is an update that should be much easier to convert to JS:

        def tee(iterable, n=2):
            iterator = iter(iterable)
            shared_link = [None, None]
            return tuple(_tee(iterator, shared_link) for _ in range(n))

        def _tee(iterator, link):
            try:
                while True:
                    if link[1] is None:
                        link[0] = next(iterator)
                        link[1] = [None, None]
                    value, link = link
                    yield value
            except StopIteration:
                return
raymondh··on Google AI: People should eat at least one small rock a day
Perhaps Google's AI included TheOnion in its training data:

https://www.theonion.com/geologists-recommend-eating-at-leas...

raymondh··on The Numeric Tower Fiasco
Numeric towers are indeed a sore point.

* There are more 64-bit integers than C floats/doubles. But in math, reals are a superset of the integers.

* Because floats are rounded, `x < x + 1`, is not an invariant.

* The possibility of a `NaN` value means that the assignment `y = x` does not guarantee that x and y are equal.

* Floats are implemented as binary fractions, so they are actually rationals.

I disagree with the OP that OOP is entirely flawed. The core ideas of encapsulation and messaging are a really useful organizing principle. And polymorphism beats maintaining giant case-statements.

Only when inheritance is added to the mix does it get dicey. As a tool for code reuse, it is not a bad idea. I think the central problem is that people want more from inheritance that it has to give (especially if you expect that children are always substitutable for their parents).

raymondh··on Song lyrics getting simpler, more repetitive, angry and self-obsessed – study
A lot of classic popular songs also had low information content:

https://happyhollyproject.com/2014/04/27/flowcharts-and-song...

raymondh··on Flowers for Algernon (1965) [pdf]
The Terminal Man by Michael Crichton
raymondh··on Python listed as memory-safe language in latest CISA recommendations
This is a red-herring. In the end, everything is run in machine language which is intrinsically unsafe. But that doesn't matter. High level languages can enforce constraints that eliminate or mitigate common programmer errors for memory access.

For example, it is a simple matter for a language to provide a list or array API that will perform bounds checking for indexed access. This simple measure would have prevented something like the HeartBleed security bug.

raymondh··on Artificial intelligence systems found to excel at imitation, but not innovation
What counts as an innovation is in the eye of the beholder. Mark Rober demonstrated Gemini's creative ability by having it suggest a video, details of contents, and how to produce it. The results were impressive but far from world changing: https://www.youtube.com/watch?v=mHZSrtl4zX0
raymondh··on Higher quality random floats
FWIW, there is a recipe for this in the Python docs[0]. It was developed in collaboration with Alan Downey and Tim Peters:

    from random import Random
    from math import ldexp

    class FullRandom(Random):

        def random(self):
            mantissa = 0x10_0000_0000_0000 | self.getrandbits(52)
            exponent = -53
            x = 0
            while not x:
                x = self.getrandbits(32)
                exponent += x.bit_length() - 32
            return ldexp(mantissa, exponent)
[0] https://docs.python.org/3/library/random.html#recipes
raymondh··on Why Does Python Code Run Faster in a Function?
Note that this effect varies across builds and versions. For most of Python's history, we could give clear and invariant optimization advice (locals and nonlocals are fastest, global variable access was at least twice as slow, and builtin variable access was even slower). That ordering will likely remain true but absolute speeds have improved dramatically and the ratios have shifted).

Here is a run of Tools/scripts/var_access_benchmark.py for Python 3.12rc1 stock build for an Apple M1 Max (your mileage may vary):

    Variable and attribute read access:
       1.9 ns read_local
       2.4 ns read_nonlocal
       2.8 ns read_global
       4.1 ns read_builtin
       5.0 ns read_classvar_from_class
      12.1 ns read_classvar_from_instance
       4.8 ns read_instancevar
       4.7 ns read_instancevar_slots
      12.2 ns read_namedtuple
      29.0 ns read_boundmethod

    Variable and attribute write access:
       2.4 ns write_local
       2.5 ns write_nonlocal
      10.5 ns write_global
      26.8 ns write_classvar
       4.3 ns write_instancevar
       4.2 ns write_instancevar_slots

    Data structure read access:
       5.7 ns read_list
      11.1 ns read_deque
      10.1 ns read_dict
      10.5 ns read_strdict

    Data structure write access:
       6.2 ns write_list
      11.3 ns write_deque
      11.2 ns write_dict
      12.0 ns write_strdict

    Stack (or queue) operations:
      18.4 ns list_append_pop
      17.8 ns deque_append_pop
      18.1 ns deque_append_popleft
raymondh··on In the Battle Between Bots and Comedians, A.I. Is Killing
If you repeat the same word, it doesn't count as a rhyme ;-)
raymondh··on Evening Club Bridge Is Dying (2015)
""" Ceausescu and his wife, Elena, outlawed bridge, popular with the intelligentsia, and shut down clubs where the card game was played. “They were worried that when too many intelligent people got together, it could easily develop into something else,” mathematician Mircea Mihu said. """

https://www.latimes.com/archives/la-xpm-1990-05-28-mn-69-sto...

raymondh··on Mars Keeps Spinning Faster, and Scientists Don't Know Why
From the abstract, "We also find evidence of a slow acceleration in the Martian rotation rate, which could be the result of a long-term trend either in the internal dynamics of Mars or in its atmosphere and ice caps."
raymondh··on How fast should you accelerate your kid in math?
In our case, the main push to accelerate is to get more knowledgeable teachers.

While experimenting with number sequences in Excel, our child said, "I found e, 2.718281828". When he went to show off his discovery, his two math teachers said that they had never heard of it before.

raymondh··on Python Dict – An Explorable Explanation
For those who learn better from videos, here's a 37 minute Pycon 2017 talk that covers all of the major aspects of Python dictionaries including how they became ordered and compacted, how object dictionaries share keys, why full hashes are stored, how the probing sequence works, when resizing occurs, etc.

https://www.youtube.com/watch?v=npw4s1QTmPg

raymondh··on Supreme Court Narrows the Scope of the Clean Water Act
> Increasing rollbacks of various right and protections - labor, environmental, consumer - seem to be happening lately.

I take issue with the "rights and protections" phrasing being used in the exact opposite sense of what they mean in this case. The underlying issue is that the vague wording in the CWA was has come to be interpreted expansively and that the consequences of even unintentional violations can be crushing. Here, ordinary people were about to build a house not obviously in "navigable waters", and they were threatened with $40,000 per day in sanctions.

Also, calling it a "rollback" is deceptive phrasing that it implies going back to the time before the CWA was enacted. It would be more accurate to note that the law has been interpreted more and more broadly every year since its creation and that some outer limit is being set on what constitutes a "waterway" or "pollutant" (neither are being interpreted in a way that most people would expect).

I get that some HackerNews commenters can easily fall into the environment-good/conservatives-bad mentality, but this case can also be viewed as falling into the same category as Aaron Swartz being crushed by an expansive and overly punitive interpretation of the CFAA.

Relevant excerpts:

> Within a few years, the agencies had “interpreted their jurisdiction over ‘the waters of the United States’ to cover 270-to-300 million acres” of wetlands and “virtually any parcel of land containing a channel or conduit . . . through which rainwater or drainage may occasionally or intermittently flow.

> By the EPA’s own admission, nearly all waters and wetlands are potentially susceptible to regulation under this test, putting a staggering array of landowners at risk of criminal prosecution for such mundane activities as moving dirt.

> The CWA is a potent weapon. It imposes what have been described as “crushing” consequences “even for inadvertent violations.” Army Corps of Engineers v. Hawkes Co., 578 U. S. 590, 602 (2016) (Kennedy, J., concurring). Property owners who negligently discharge “pollutants” into covered waters may face severe criminal penalties including imprisonment.

> The Act prohibits “the discharge of any pollutant” into “navigable waters.” 33 U. S. C. §§1311(a), 1362(12)(A). It broadly defines the term “‘pollutant’” to include not only contaminants like “chemical wastes,” but also more mundane materials like “rock, sand,” and “cellar dirt.”

> And due to the Act’s 5-year statute of limitations, 28 U. S. C. §2462, and expansive interpretations of the term “violation,” these civil pen- alties can be nearly as crushing as their criminal counter- parts, see, e.g., Borden Ranch Partnership v. United States Army Corps of Engineers, 261 F. 3d 810, 813, 818 (CA9 2001) (upholding Agency decision to count each of 348 passes of a plow by a farmer through “jurisdictional” soil on his farm as a separate violation)

When vague laws with powerful punishments are interpreted expansively and aggressively by enforcement agencies, we should all hope that the Supreme Court should set limits.

raymondh··on Show HN: Live coaching app for remote SWE interviews, uses Whisper and GPT-4
From one point of view, this isn't a problem at all. If a candidate can use tools to generate, understand, demonstrate, and explain a solution in real time, that is likely what you want them to do on the job as well.

Also, it is likely that in the near future that most programmers will be "pair programming" with Copilot, ChatGPT, or some other tooling to augment their capabilities.

raymondh··on New York Times says it won't pay for Twitter verified check mark
How is charging for a blue checkmark any different from games that sell virtual merchandise for real world money?

This is HackerNews. I would expect that more people would be interested on how a company like Twitter can monetize its services. As as long term share holder, it seemed to me like the company was being run as if it were a not-for-profit.

raymondh··on Florida parents upset by Michelangelo’s ‘David’ force out principal
IIRC, there was an episode of The Simpsons that addressed exactly this scenario.
raymondh··on Covid drug drives viral mutations – and now some want to halt its use
Same here. I went from delirium to symptom free overnight.
raymondh··on The Unreasonable Effectiveness of AWK
The authors themselves were astonished to see what their language could do in the hands of other programmers.
Page 1 of 11Next →