More data is better. The fact that humans have better algorithms that need less data does absolutely nothing to negate this.
More data is better. The fact that humans have better algorithms that need less data does absolutely nothing to negate this.
Street level mapping data isn't relevant or meaningful? Basically every company working on this problem seems to pretty strongly disagree with you.
> But to your point, the assumption that this is even a problem of “not enough data” is questionable at this point. How to turn that data into results is something no one has come close to figuring out yet.
This is trivially false. Given infinite data, all possible situations would be represented in the data, and the solutions applied in those situations could be copied exactly, something that existing algorithms are completely capable of doing.
In principle. In practice, you'd need infinite time and infinite storage.
Btw, do you have to add stuff like "This is trivially false" to your comments? It doesn't make your comments sound more right, only less well considered.
That is irrelevant.
> Btw, do you have to add stuff like "This is trivially false" to your comments? It doesn't make your comments sound more right, only less well considered.
Trivial in the mathematical sense. As in, there is a trivial counter-example to your point. Citing infinity is a 'trivial' case. I'm using 'trivial' to describe my counter-example, not his error.
There are machine learning techniques that generalise well from few data, but they are not very well known in the industry.
Also, though more speculatively, I think the idea of "lots of data" is attractive to marketing departments. There's something about algorithms that need huge amounts of data and compute, that only a select few companies can use. I guess it gives bragging rights, of a sort: "we got the biggest data around. Buy our stuff!".
But like I say, that's speculation on my part.
Precisely.
> There are machine learning techniques that generalise well from few data, but they are not very well known in the industry.
Sure, and we'd all love to be using those. But even if you generalize well from small datasets, you still generalize better from larger ones.
> Also, though more speculatively, I think the idea of "lots of data" is attractive to marketing departments. There's something about algorithms that need huge amounts of data and compute, that only a select few companies can use. I guess it gives bragging rights, of a sort: "we got the biggest data around. Buy our stuff!".
It may be attractive to marketing departments, but it is also essential to data science projects like this.
Not to my knowledge. What techniques did you have in mind that work like that?
ME: There are machine learning techniques that generalise well from few data, but they are not very well known in the industry.
YOU: Sure, and we'd all love to be using those. But even if you generalize well from small datasets, you still generalize better from larger ones.
That is not how those techniques work to my knowledge, so I was asking which ones you had in mind.
Like I say, this is not the case. There are learning algorithms that generalise so well from few data that their performance can improve only marginally with increasing amounts of data, or not at all.
I appreciate that you probably have no idea what I'm talking about. I certainly don't mean linear regression.
Erm, no. Not unless they are solving the problem perfectly.
> I appreciate that you probably have no idea what I'm talking about. I certainly don't mean linear regression.
I work in the field. I'm quite certain i'm familiar with whatever it is that you think you're talking about.
The category of algorithms that attempt to learn things from few examples is called 'One shot learning'. It's usually in the context of image classification, but it applies equally well elsewhere. These algorithms still learns better from more data.
Do feel free to share an example of an algorithm that generalizes better from less data. I'll wait.
Well, yes, that's what I mean.
I gave an example here a while ago, of how a Meta-Interpretive Learning algorithm, Metagol, can learn the aⁿbⁿ grammar perfectly from 4 positive examples:
https://news.ycombinator.com/item?id=17837055
That's typical of Metagol, as well as other algorithms in Inductive Logic Programming, the broader sub-field of machine learning that MIL belongs to.
>> Do feel free to share an example of an algorithm that generalizes better from less data. I'll wait.
To clarify, my claim is that there are algorithms that learn adquately from few data and therefore don't "need" more data. Not that less data is better.
That said, there are theoretical results that suggest that a larger hypothesis space increases the chance of the learner overfitting to noise. So what is really needed in order to improve generalisation is not more data, but more relevant data. Then again, that is the subject of my current PhD so I might just be interpreting everything through the lens of my research (as is typical for PhD students).
You work in the field? What do you do?
It would likely be more informative to instrument a few cars with some advanced sensor package and let well ranked drivers drive them around than to try to gather data from smartphones in existing cars, but I suppose it depends on what the end use is.