Hacker News Reader

I mean all optimization algorithms do is game a benchmark. That’s the whole point.

The hard part is making the benchmark meaningful in the first place.

Yeah, and if anything, RL has a rep of being too good at this job, because of all the cases where it gamed a benchmark by picking up on some environmental factor the supervisors hadn't thought of (numerical instabilities, rounding, bugs, etc.).

My favourite is this one:

https://news.ycombinator.com/item?id=43113941

The ML version of Professor Farnsworth[1]:

It came to me in a dream, and I forgot it in another dream.

[1]: https://www.imdb.com/title/tt0584424/quotes/?item=qt0439248

No, that is patently false. Many optimization algorithms which computer scientists, mathematicians or software developers devise do not involve benchmakrs at all, and apply to all possible inputs/instances of their respective computational problems.

Example?

Those times when people write code with loads of theoretical micro optimisations that they never actually test against a benchmark because otherwise they wouldn't do it.

Perhaps “example” means different things in different cultures.

I was being somewhat facetious.

Plot twist: the loss function for training is basically a benchmark

Is it a plot twist if it’s the whole plot?

Reply on news.ycombinator.com