Existence does not imply correlation

In programming, in my experience, the nastiest bugs to fix are actually two or three separate bugs interacting in weird ways. If you find a bug like he did, and it's easy to fix and unlikely to break something else, but you can't reason how it could be causing the issue you're seeing, FIX IT ANYWAY. It's quite possible it's interacting in some subtle way with another bug, and fixing it may make the other issue start behaving more consistantly, and easier to fix.

It may feel wrong, because you feel like you should set that theoretically unrelated bugfix aside until you can work out the bug you're trying to focus on. In my experience, that's often not the right approach.

"A couple of weeks ago I pointed out Sam Altman making bogus arguments about sexism - arguing by juxtuposition that the existence of sexism reduces the number of women in technology."

I think the fact that sexism reduces the desire of women to participate in something is established. It doesn't make sense to dismiss this as an 'argument by juxtaposition' since scientific thought begins with drawing new guesses from established patterns, particularly when the data are found wanting. So now, reading between the lines, we have the dual inference that

1) the author doesn't understand his own argument, and

2) the author has a reactionary axe to grind regarding gender issues in technology.

Whether or not these inferences are true (I don't think they are), a large audience is now lost since most people are socially aware enough to understand that 'sexism doesn't repel women' is a poor null hypothesis, even if they can't articulate these feelings in the language of science.

* The Redis example is forced. Here's why: most engineers faced with that problem would, sooner rather than later, collect stats about the nodes and immediately observe that the root cause was an overloaded server. As the post admits, the cause was immediately obvious. No statistical reasoning was required, and, in fact, this reader doubts that statistics played too much of a role in Stucchio's own diagnosis. (Whether it actually did or didn't isn't material to my point.)

* The insight being provided about correlation is extremely simplistic. Essentially, it communicates in two graphs the definition of correlation. That might make sense if the post was written in a fashion that tried to communicate the fundamental idea of correlation to someone with literally no acquaintance to the term. But it's not; despite having subheds about what "is" and "isn't" correlation, the writing style more clearly signifies that the author is trying to debunk a mistaken idea about what the term is, implying that its readers are already somewhat acquainted with the idea. Or, put more simply: the fundamental idea the post tries to communicate could have been conveyed in two simple sentences. (Glass houses about my own writing style: duly noted).

* The transition from technical discussion to Sam Altman is abrupt. More importantly, the Altman subject has much more valence than the point about diagnosing Redis failures. Thought experiment: chop everything in this article after Redis out, and imagine it was posted not by Chris but by some random account. Would anyone pay attention to this post?

Unfortunately, the post doesn't have much insight to offer about Altman's post. It's framed Altman in a manner incompatible with Altman's post --- suggesting, contrary to reality, that Altman was trying to present a complete empirical argument about gender disparities in technology. It then tries to beat Altman over the head with that framing. Altman emerges unscathed, because the author is swinging at his shadow, not him.

It sounds true. It kind of feels true to me, even. But there's too many feels being passed for fact lately.

    (Co-)existence is necessary for correlation
    Correlation is necessary for causality

    1. It's valuable and common to note E and Co
       suggestively of Co and Ca respectively. If
       I *notice* E then I begin to hypothesize about
       Co because I know have evidence for its 
       possibility if not yet evidence for Co directly

    2. Being lazy/imprecise in speech or reasoning
       might cause someone to "skip a step" and claim
       something more powerful than what they actually
       have.

So how can we make sure that (2) occurs casually without precluding (1)?

I think that articles like this one and all around the whole "correlation is not causation" tagline are great. They help to ensure people remember the size of the space between each step in E > Co > Ca.

I think another powerful technique would be to demonstrate more arguments that make the jumps and highlight the properties which allow that to happen. Chris did a good job here demonstrating correlation—what it looks like when it exists or fails to exist—but much, much more can be said about the E > Co jump.

The Co > Ca jump is far more complex. Worse, it's often obscured through opaque words like "randomized, controlled study" or "scientific method" which are actually quite far away from the mechanisms which allow that jump to be made—they're more like implementation details obscuring a great API. Demonstrating clear arguments here (and not "rain + wet grass" oversimplified ones) could be a great boon to public reasoning.

What else can be done to reduce (2) without precluding (1)?

(*): Really (linear) correlation is just one kind of relationship-of-interest between things. This is often pointed out when people talk about correlation in terms like "circular relationships have 0 linear correlation, oh no!"

What I'd like to write instead might be E > M > C where M becomes "model building". Choosing to highlight linear correlation means that you're choosing a linear model. That might be perfect, or it might be wrong, but you still must choose it before you can structure your observations into evidence of some kind of causality. Generally, at this point, you would also want to begin developing covariates all in preparation for the C step.

"sexism in technology" is not an entity, it is a description of an aggregate. saying that the "sample size" is 1 is absurd.