Say you have a coin that might be unfair, and you want to estimate its bias. You flip it a bunch of times, and it mostly lands heads.
Predictions of the coin's bias based on this observed data are usually going to be that it's biased some % towards heads. (unless maybe you have a strong prior, but that's a different topic)
But there is also a chance that really the coin is fair, or even biased towards tails and you just got unlucky in your flips.
There's a mismatch between the true model (the coins actual bias) and the observed data (the result of your flips) because of this chance of `unluckiness'
Semantics is the essence of communication.
Suppose I take the function y = log(x) and add random white noise. The function log() and the parameters on the random white noise process are the data generating process. We could then fit a model y = \beta X + \epsilon, and then compare the "true" (first) model to our second model. When the natural world generates our data, the idea behind all this is the same: there is a process which generates the data, and the data reveals information about that process to an approximate degree.
Some reads:
[1] http://www.rimini.unibo.it/fanelli/econometric_models2_2012....
Edit: I don't know if people don't like the grammar, or what?
How about this:
Can you give a non-synthetic, ie. natural, example of a data generating process?
The notation is pretty sloppy for someone handing out statistical advice.
The main point of his argument is that R² compares to observed mean and not into true model behind the data.
Terms "true model", "true distribution" and "true population" are well defined concepts in statistics – unless Jorma Rissanen is nearby.
y = log(x) + u
Where u is an error/noise term, and in this case takes a normal distribution. The error term is meant to Hoover up all the stuff you can't account for because it's impossible to measure everything. As ever, what you're trying to estimate is the parameters for everything but the error term.He's maybe being a little sloppy with the notation, but TBH it's a pedantic distinction that I haven't seen anyone make a big deal of since I last took intro stats. In R, for example, you'd specify the model as "y ~ log(x)" and leave it at that.