datastoat

I think the article is confusing three things. In Bayesian terminology, suppose we've found a posterior distribution for effect size. Then there are three things we might consider: (a) how tightly concentrated the posterior distribution is, (b) how much probability mass there is in a given part of the space, Prob(effect size >= thresh), (c) how we're choosing to incorporate other data.

To illustrate: suppose we've found a posterior distribution for effect size, and we've found that 95% of the posterior probability mass is >= 0. It might be a broad flat distribution (huge MAP effect size, massive uncertainty about effect size), or it might be a tight distribution (small MAP effect size, small uncertainty about effect size). There's nothing intrinsically wrong or suspicious about either of these two cases.

The headline of the linked article has the words "higher than is plausible". In other words, the premise of the article is that there is some other evidence which tells us what effect size is plausible. We could incorporate that evidence by adjusting our prior, which will obviously push the posterior distribution closer to what prior belief says it should be. Alternatively we could incorporate that evidence as a subsequent Bayesian conditioning step, i.e. treat the other evidence as further observations. The magic of Bayes's rule means that both of these approaches give the same answer. (The frequentist approach is pretty much like the latter -- it says "here's the result of my experiment, and I'll leave the reader to incorporate it with their own prior beliefs, in a meta-analysis." There's nothing intrinsically wrong with this.)

The article says "I find it frustrating when researchers don't think about their effect sizes." That's the wrong conclusion. The proper conclusion is (1) always report confidence intervals for your effect sizes, (2) don't pay any attention to a statement about effect sizes unless it comes with confidence / credible intervals -- and so the article's "painful email exchange" is missing the point, (3) when you report your results, make sure there's enough information for the reader to incorporate it with their own priors -- and the whole point of confidence intervals is to let us do this.

Ideally, there's some field in our data that we can use -- e.g. train on data from cities in several countries, test on data from cities in a different country; train on one corpus, test on another; train on climate data at one level of CO2, test on data at another. If there's no such field, I do a simple 1d dimension reduction, and pick off 20% of the data at the extremes.

If we know exactly the domain where our model will be used, and it matches exactly the disposition of the training data, then shuffling is fine. But if we want our model to work well in new domains, then we're like scientists looking for generalizable laws, and the best way to test this is by testing in unseen domains. It's often hard to get hold of data with this domain-diversity, which is why a lot of ML models are fragile.

Big enough data that you can afford not to use some of it for training! Different disciplines hit this threshold at different times -- language and speech much earlier, as you say; clinical trials not there yet.

Maybe we could talk about two cardinalities of "big" data. The first is when you can afford not to use all of your data for training. The second is when you can usefully fit highly overparameterized models.

Validation on holdout sets.

When I was a student in the 1990s, I was taught about hypothesis testing (and all the hassle of p-fishing etc.), and about Bayesian inference (which is lovely, until you have to invent priors over the model space -- e.g. a prior over neural network architectures). These are both systems that tie themselves in epistemological knots when trying to answer the simple question "What model shall I use?"

Holdout set validation is such a clean simple idea, and so easy to use (as long as you have big data), and it does away with all the frequentist and Bayesian tangle, which is why it's so widespread in ML nowadays.

It also aligns statistical inference with Popper's idea of scientific falsifiability -- scientists test their models against a new experimental data, data scientists can test their model against qualitatively different holdout sets. (Just make sure you don't get your holdout set by shuffling, since that's not what Popper would call a "genuine risky validation".)

The article mentions Breiman's "alternative view of the foundations of statistics based on prediction rather than modeling". Breiman does make a big deal of evaluation on holdout sets; but his "prediction" idea isn't general enough, since it doesn't accommodate generative modelling (e.g. GPT, GANs). I think it's better to frame ML in terms of "evaluating model fit on a holdout set", since that accommodates both predictive and generative modelling.

When I scrape, I attach a header that says roughly “By responding to this request, the provider allows me to use the response for my own personal use; and accepts that this overrides any terms stated on the web page.”

Cut and dried? A lawyer friend told me, informally, that if it went to court a younger judge would throw out my “contract” because it’s silly, but a more senior judge might well take the view that both contracts, my version and that of the web site, are equally silly, and both fall short of the “meeting of minds” standard.

Relatedly, a lot of recipe books just have a dry statement of what the ingredients are and how to combine them. They should be more like CS and science papers, and explain the narrative of why the recipe is exciting, and where it comes from, and what it means for the cook.

(Posted from a parallel universe.)

> It is the dataset used for training that is often at fault, not the source code of the model

This remark is absolutely spot on. This is why I don't like the language around 'algorithmic bias' and 'algorithmic accountability' -- it puts too much attention on The Algorithm, which lets big tech companies deflect us from scrutiny of their training data.

Your comment about the conflict with privacy is also spot on. Here's a paper (shameless plug!) about it: Show Us the Data: Privacy, Explainability, and Why the Law Can't Have Both [1].

What's interesting about the GDPR is that it's not purely about what regulators do: it's legislation that grants rights to people, rights that can be litigated in court. If I as a GDPR data subject demand an explanation of an ML decision about me, as I'm entitled to do, my lawyer could argue "the source code isn't an adequate explanation, we want to see the training data too". Then it'd be up to the court to make a ruling about my lawyer's request, not up to the regulator. The paper [1], co-authored by a data scientist (myself) and a lawyer, thinks through the privacy / explainability conflict as it might play out in court.

[1] https://www.gwlr.org/show-us-the-data/

I've not heard of nonlinear causal reasoning. It sounds interesting. Could you suggest some reading? (A quick search throws up uninteresting hits about nonlinear models.)

The article seemed to me to go back and forth between profound and tendentious. To pin down the ideas precisely, or perhaps to trivialize them, here's a simple technical + mathematical example: TCP, the congestion control protocol for transmitting data packets on the Internet. Here are some ways to think about TCP:

(1) TCP's congestion control is an algorithm. It maintains a variable "congestion window", and it has rules for increasing or decreasing this variable based on the acknowledgement packets that come back.

(2) TCP's congestion control is a mechanism that results in the outcome "throughput = sqrt(2) / (RTT * sqrt(p))", where p is the packet drop probability.

(3) TCP's congestion control is a system that results in the available capacity of a network being shared between all the active flows. Specifically, it acts so as to maximize a "total utility" function, subject to network capacity constraints [1].

These three levels of description are all true, all at the same time. When you ask "why did TCP give me this much bandwidth?", the 'why' can be answered at any of the three levels, and different levels will be useful for different purposes. Level 3 is close to 'purpose' and level 1 is close to 'cause and effect', but I think those labels get confusing especially when we talk about the intention of the designer. (In the case of TCP, the level 3 purpose-style understanding of TCP only came about in the late 1990s, long after TCP was invented and deployed.)

IANA philosopher ... but these levels of explanation go back to Aristotle. I think level 1 is the "efficient cause" (it happened because of an action), level 2 is the "formal cause" (it happened so as to achieve a certain form), level 3 is the "teleological cause" (it happened to achieve a purpose).

When I read the article in the light of this three-level picture of TCP, it seems to me that the "purpose" view of a system, which I take to be level 3, is just one of several equally valid views, not more nor less important than the others.

[1] https://www.princeton.edu/~chiangm/stochastic_num.pdf

What are the facts that changed, in this lab leak story?

As far as I'm aware, our knowledge of the real-world facts haven't changed -- we're still in a position of great uncertainty. All that's changed is how various commentators are dealing with that uncertainty: they had been downplaying it, now they're acknowledging it.

How about making the tax increase each month the unit is unoccupied, but also making it proportional to the asking rent? No one has any use for it, high asking rent => high tax. No one has any use for it, very low asking rent => low tax.

Here's an example: https://youtu.be/eHwy-neG_W8?t=791

It's from an introductory course on Algorithms; the dataset is the students' programming assignments, rated for similarity by an off-the-shelf similarity scorer.

This heatmap example is presented in the class on Kruskal's algorithm. We normally think of Kruskal's algorithm as a method for finding a minimum spanning tree, but it can also be thought of as building a classification tree -- which means we can use it for seriation. It's not the best method for seriation by any means, but it's nice just to see it used in this unconventional way.

It's always fun showing this in class, on the coursework that the students submitted just a few weeks ago!

> none of which seem more intuitive than writing "mu"

Write for the convenience of the reader, not of the writer!

For me: alt+enter (switches to Greek keyboard layout), m, alt+enter (back to usual layout). I use Greek symbols in Python and in Javascript. I do it almost without thinking now, as lots of my coding is tightly linked to maths.

That forum post advises "new drugs aimed at preventing Alzheimer’s should probably target surrogate markers rather than trying to fix the end-stage clinical problems".

Here's One Weird Trick for targeting Alzheimer's, to try at home. Grey hair is correlated with Alzheimer's. Hair dye is a treatment that targets that particular biomarker, with remarkable efficacy. How about we all use hair dye, in the hope it prevents Alzheimer's! Way cheaper than $56k/year too.

I completely agree about the steep learning curve and the feeling of dark magic -- how many times have I had to relearn what deparse(substitute(x)) means -- but oh the satisfaction of broadening my programming horizons. For me it didn't feel like C macros, it felt like "This must be what it feels like to have the power of Lisp"!

That's the weird thing about R. All this dark magic is hiding under the hood, but the core R team hid it so deftly that to the casual statistician it's a straightforward data modelling language that "just works". I'm not sure that it's possible to get rid of the dark magic and retain that data-modeller friendliness.

R certainly expanded my programming views! Haskell did too, but the lessons of Haskell didn't stick the way that R's lessons did. Here are some of the things I learnt from R (though they can be found in other languages of course).

* Multiple dispatch. Before learning R, I knew about polymorphism in Java and C++, and multiple dispatch in R broadened my mind and turns out to be very handy.

* The idea of "frames". In R, when you invoke `lm(height~sex*age, data=mydataframe)`, the first argument (the formula) doesn't get evaluated until the lm command asks it to be evaluated, and lm can set up the "frame" for that evaluation, i.e. the place where variables are looked up, however it likes. In fact, lm sets it up to include variables from both the scope in which you invoked lm, and also from mydataframe. This is what makes R so wonderfully concise for modelling in data science, compared to e.g. Python + pandas. I knew about frames from interactive debuggers, but until R it never occurred to me that the programming language could manipulate them.

* "Held" arguments. In R, when you invoke `plot(x, y1+y2)`, it doesn't just evaluate the arguments and then call the plot function -- it leaves the arguments unevaluated, and invokes plot. Plot then (1) decides when to evaluate them, (2) gets access to the language expression `y1+y2`, which means that it can print "y1+y2" on the plot label, (3) it can even define extra variables to include in the scope when y1+y2 gets evaluated. (I knew about held arguments earlier, from Mathematica, but they only clicked when I read the R documentation.)

I've read that R is a descendent of Scheme, and that that's where it gets all its "manipulate language expressions" from. I don't know any Scheme, nor Lisp, and I should definitely learn them -- but in the meantime, my experience has been that R's ability to manipulate language expressions is what makes it such a wonderful sweet spot as a data modelling language. I mostly use Python + pandas nowadays, but it feels such a slog in comparison.

The article explains why it's simplistic to think of legal argument as pure maths-style logical deduction. This goes back at least to Oliver Wendell Holmes (US supreme court justice from 1902), who made fun of those who treated a dissenting judgement "as if it meant simply that one side or the other were not doing their sums right, and if they would take more trouble, agreement inevitably would come."

Holmes said that law should be seen as induction, not deduction. Induction involves pattern matching, analogizing, and so on. I think that at least the first 4 of the 5 types of argument described in the article can be thought of as types of induction.

Holmes also said that the law consists of nothing more than prediction of what courts will do. And he described the development of the law in language that would make any Machine Learning person think of nearest neighbour classification: "The growth of the law is very apt to take place in this way: two widely different cases suggest a general distinction, which is a clear one when stated broadly. But as new cases cluster around the opposite poles, and begin to approach each other [...] at last a mathematical line is arrived at by the contact of contrary decisions."

It seems to me that the legal theory of Holmes, and the modern practice of machine learning, are remarkably similar. They're both about prediction based on experience / datasets. Shameless plug: I've written a (free!) book about it, "On the path to AI: Law’s prophecies and the conceptual foundations of the machine learning age".

https://link.springer.com/book/10.1007/978-3-030-43582-0

The probability that at least one tracker is compromised actually grows sub-linearly in the number of trackers.

Let n be the number of trackers, let p be the probability that an individual tracker is compromised, and assume compromises are independent. Then: Prob(at least one comp.) = 1 - (1-p)^n ~~ np. In other words, linear in n. This approximation is fine for small n*p. For large n*p, the probability has to be <= 1 of course, so the real probability is sublinear.

> But not Greek plurals, so it's octopuses, not octopodes.

Though Australia is in the antipodes (to the UK), not the antipuses.

To be precise, it's impossible to measure one-way latency between a pair of nodes without external information. But if you have a mesh of nodes, the story is different [1]. If you have N nodes and hence N unknown clock offsets, and if you have ping data from N(N-1) pairs, you can do better.

[1] https://www.usenix.org/conference/nsdi18/presentation/geng

As the article explains, latency and clock sync go hand in hand. Here's a blog post [1] that goes further into clock sync, contrasting NTP and hardware-based systems. The company behind the blog post says that their solution is available as a managed service on Azure and GCP, but I've never looked out for it.

[1] https://www.ticktocknetworks.com/tick-tock-the-clock-runs-wi...

While we're waiting for the paper on the Platonic link ... here's a book about how machine learning is all just a rerun of Oliver Wendell Holmes's theory of epistemology in the law, from the 1890s.

https://link.springer.com/book/10.1007/978-3-030-43582-0

The article starts with "if it's not reproducible, it's not science, right?" And then it goes on to talk about a limited idea of reproducibility, i.e. plain duplication and recipe-style rerunning an analysis.

I think there's a more important idea of reproducibility: your ideas have to be reproducible in different settings. We expect scientific rules to apply across a range of settings. If they don't, it's a recipe, it's not science. So the real burden of reproducibility is communicating your ideas so that other people can see how to apply them to other settings, and moreover so that they want to.

All the fetishizing of "give me a repo where I can just run 'make'" is, I think, distracting people from this. I wonder if making such a big deal of reproducibility-of-analyses is actually harmful: maybe it'd be a better test of reproducibility-of-science if other researchers had to reproduce your findings based on how you communicated your ideas, i.e. based on your writeup.

Probability notation used in ML and engineering has this problem, of overloading p(). Probability notation as used by probabilists in maths departments is completely different: it’s more explicit, and sometimes more clunky.

There’s a hybrid notation that I prefer, for example “Pr_X(x)” for the density function of random variable X at point x; you drop X if the random variable is clear from the context, and you drop x if you’re referring to the entire distribution. Or Pr_X(x|Y=y) for a conditional density. But this notation still has problems when you’re working with hairier conditional distributions, or with distributions that are neither discrete nor continuous.

(Source: used to be a mathematical probabilistic, now working in ML.)

He bricked the marble!

The outputs of the model _were_ being treated as predictions.

The Ferguson paper from 16 March used the language of prediction: "In the (unlikely) absence of any control measures [...] given an estimated R0 of 2.4, we predict 81% of the GB and US populations would be infected over the course of the epidemic." [1]. The news coverage also used that language: "Imperial researchers model likely impact of public health measures" [2]. And look at the rest of the comments in this discussion, and count how many types "predict" appears!

> If we'd waited until we had enough data to make predictions with it

This is like the drunk looking for their keys under a streetlight. "Did you lose the keys here?" "No, but the light is much better here." -- "How confident are you in your model's predictions?" "I have no idea, but it's the model I have."

Also -- the Ferguson model made predictions, based on the parameters they picked. You don't need to wait for data to make predictions; you only need data to validate your predictions.

> How would you ethically collect training data for the interventions?

You don't. You (as a scientist who influences public policy) should publish validated confidence intervals for your predictions. You (as a government) should understand that there is a huge margin of uncertainty in the predictions, and accept that sometimes you just have to make decisions in the absence of knowledge. You (both the scientist and the government) do not go around spouting "Our decisions are led by science".

[1] https://spiral.imperial.ac.uk:8443/bitstream/10044/1/77482/1...

[2] https://www.imperial.ac.uk/news/196234/covid19-imperial-rese...

I would pick a value of R that shows itself to have good predictive accuracy.

The way to test predictive models is always to look for their predictive accuracy on holdout data. Machine learning has this ingrained. Classic statistics does this too -- AIC is used to compare models, and it's (asymptotically) leave-one-out cross validation [1].

There's nothing intrinsically wrong with models that have millions of parameters; they might overfit in which case they will have poor predictive accuracy on holdout data, or they might predict well.

I agree with the original article that software engineer scrutiny isn't appropriate for this sort of code -- but I would argue instead that it needs a general-purpose statistician or data scientist or ML expert to evaluate its predictive accuracy. You can't possibly figure this out from a simulator codebase.

At the time the model was published, and acted on by the UK government, there was very little data on which to test predictive accuracy. That's fine -- all it means is that the predictions should have been presented with gigantic confidence intervals.

[1] http://www.stats.ox.ac.uk/~ripley/Nelder80.pdf

It's good you put "contained" in quotes. Japan, Singapore, and Hong Kong are all (according to this plot [0]) showing exponential growth in the number of cases. It's a slower exponential than elsewhere, but it's still an exponential.

[0] https://www.ft.com/__origami/service/image/v2/images/raw/htt...

If Facebook lawyers understood the implications of LeCun's argument, they wouldn't be happy!

There are two types of explanations here: (1) why did the data come to be as it is, (2) why did my ML make the prediction it did.

Science looks for the answer to (1), and causal models are a great way to think about it. Science and engineering, when they go hand in hand, build a machine by saying "Here is data, let me do science to understand nature's underlying laws, and my machine shall be based on those laws". The machine is inherently explainable because it's based on scientific laws.

In the ML world, we can bypass the "learn scientific laws" part, and jump straight to "build a machine based on data". So the best answer to (2) has got to be "my ML made the prediction it did because of its training data". As Pearl said, ML is just curve fitting, so the only way to "explain" a ML prediction is to say "here are the points that the curve was fitted to". Prediction is just reading a value off the curve. Think the machine is biased? Look for bias in the training dataset! Think the machine is inaccurate? Look for sparsity or conflict in the training dataset!

So the consequence of LeCun's distinction is that, when the GDPR calls for explainability of ML decision making, it is really calling for sharing of the training data. Facebook, watch out!