HNHacker News
TopNewBestAskShowJobs

Mike_12345

173 karma · joined March 3, 2023

submissionscomments
Mike_12345··on Geoffrey Hinton leaves Google and warns of danger ahead
> this removes any need/justification to carpet bomb anything,

You totally missed the point. The mass bombing in WW2 was intentional. That was specifically the goal, to kill as many civilians as possible.

Mike_12345··on Are emergent abilities of large language models a mirage?
Yes shifting the goal posts and finding edge cases not well suited to LLMs, and also ignoring the chain of thought prompting.

It can solve math word puzzles that are not in its training set.

Yes you can find these edge cases. We know about these edge cases and that's just missing the point.

There are countless examples of emergent properties in these LLMs which by definition are solving tasks outside of its training set.

Its reasoning has been demonstrated on examples outside of its training set.

Mike_12345··on Are emergent abilities of large language models a mirage?
All you have shown here is that its arithmetic reasoning is flawed, not that it cannot reason at all.

"Reasoning" is a much broader term than you think it is.

Mike_12345··on Are emergent abilities of large language models a mirage?
> Give them a math problem that doesn't exist in their training set and they cannot solve it.

They routinely solve math problems (and other reasoning tasks) that don't exist in their training set. Examples were in that paper I linked to. This is one of the incredible emergent properties of LLMs / deep neural networks.

Try it out today on GPT-4. Make up your own math problems and go for it.

Mike_12345··on Are emergent abilities of large language models a mirage?
You have a narrow definition of reasoning. Formally and technically it is solving a symbolic reasoning task through a sequence of steps. Yes we know it's not conscious and not human reasoning.

> At no point does the LLM know that 5+6 = 11

Does it need to "know" that (by your narrow definition of "know") in order to reason about a word math problem?

> if asked to solve a problem in which 5+6 was an implicit component of the solution but not explicitly present in the text, it would be completely lost

Can you provide an example? What makes you believe it can't be trained to solve those too? That's just a higher abstraction over the language. Add more layers, more training, etc. Many humans cannot solve basic math word puzzles that this artificial neural network can already solve.

Mike_12345··on Geoffrey Hinton leaves Google and warns of danger ahead
LOL. Hinton won the f**ing Turing Award for his research in deep learning / neural networks, and you're telling us his knowledge is irrelevant.
Mike_12345··on Geoffrey Hinton leaves Google and warns of danger ahead
> Calculators are smarter then humans in calculating, what does he mean by that?

He means AGI.

Mike_12345··on Geoffrey Hinton leaves Google and warns of danger ahead
In the good old days we intentionally bombed entire cities with the goal of killing as many innocent civilians as possible. It wasn't a technological issue. The goals have changed since then.

https://en.wikipedia.org/wiki/Bombing_of_Dresden_in_World_Wa...

https://en.wikipedia.org/wiki/Atomic_bombings_of_Hiroshima_a...

Mike_12345··on Are emergent abilities of large language models a mirage?
> When I say understanding I mean the meaning and context around it.

Yes it is actually modelling the meaning and context around it. ChatGPT is a ~100 layer deep neural network that was trained specifically to solve "natural language understanding tasks". The term "understanding" is an important term in NLP research along with the concept of "semantic similarity".

I believe it is more powerful and ultimately (eventually) more dangerous than you fear it is.

Mike_12345··on Are emergent abilities of large language models a mirage?
"Language Models Perform Reasoning via Chain of Thought"

Posted by Jason Wei and Denny Zhou, Research Scientists, Google Research, Brain team

https://ai.googleblog.com/2022/05/language-models-perform-re...

Mike_12345··on Are emergent abilities of large language models a mirage?
> Somehow, people are throwing up their hands and giving up at understanding the thing

Go tell that to the researchers who are working hard on studying emergent properties.

Mike_12345··on Are emergent abilities of large language models a mirage?
> I had someone much knowledgable on this topic than myself claim ChatGPT and the like "understand" stuff. My standing argument is that they wouldn't hallucinate incorrect responses if they understood anything, the hallucination is when their approximation of what a real response would be falls short.

ChatGPT models semantic relationships in the data. That's what your smart buddy means by "understanding". That is a high dimensional model of the data set which infers abstract semantic relationships / concepts. But he would not claim that those semantic relationships are exactly the same as any human interpretation of the data (which you refer to as the "real" response).

Language models also have limited reasoning abilities. They are capable of misunderstanding as much as understanding.

Mike_12345··on The State of Serverless GPUs
> I truly don't understand the bizarre and conspiracy-theory laden opposition to terms like this.

I truly don't care and not opposed to the term. Just trying to explain why someone might get confused.

Mike_12345··on The State of Serverless GPUs
> It's a very standard construction in English.

It's a new buzzword coined by the marketing team at Amazon in 2014. Somewhat confusing here as you are renting time on a GPU server described as "serverless".

Mike_12345··on Half of vinyl buyers in the U.S. don’t have a record player: study
Audio is audio? 4 bit mono sounds the same as 16 bit stereo?

Vinyl sounds blatantly different due to analog distortion in the vinyl mastering and playback mechanisms. The distortion, frequency response, dynamic range, are all significantly different. You must be practically deaf to not hear the difference between vinyl and CD quality.

Mike_12345··on Why I'm no longer writing stories with AI
I don't understand your point. It doesn't seem relevant to the article.
Mike_12345··on Why I'm no longer writing stories with AI
> So they were fine with monetization in a scenario when it benefited them, and against when the scenario, AI assisted writing, wasn’t as favourable for them.

Yes. So what? Not every business is identical. Some businesses are more greedy and ruthless than others. Some provide more societal benefits than others.

Mike_12345··on Europe to ChatGPT: disclose your sources
You are confused. The EU is proposing that the developers should disclose the sources of training data. That is explained in the first sentence of the article. They're not requiring the language model itself to disclose the sources.
Mike_12345··on Commercial lunar lander presumed lost after moon landing attempt
This was not a test flight.
Mike_12345··on Maintain consistent styles for developers working across various editors
Meh. That is so obvious to anyone who would use that function. Just put it on one line.

If your function has too many parameters then the code probably needs refactoring.

Yes agreed about DSL in Scala. Scala is complicated. I was a Scala dev for many years. We tried to avoid the more academic and overly abstract designs with that language.

Mike_12345··on Maintain consistent styles for developers working across various editors
Yes layout is semantic and the automatic layout configuration should not be random nonsense. Configure a style that makes sense and apply it.

I've never seen that code that needed to be weirdly formatted with its own rules separate from other files in the codebase in order to make sense. That just sounds like a developer being a prima donna with their code style, not something that genuinely improves understanding and collaboration.

Mike_12345··on Maintain consistent styles for developers working across various editors
I would say everything in the codebase should be auto formatted to a consistent style. This reduces whitespace noise in pull requests. No room for egotistical developers to take ownership of files in a shared codebase and waste time on petty formatting arguments. It keeps everything professional and tidy.

When someone commits changes to an existing file, they auto format before committing. If existing parts of the file already contain weird non standard formatting this creates distracting whitespace noise in the pull request that could have been prevented. It's a waste of time to not use standard formatting.

Mike_12345··on Scaling Databases at Activision [pdf]
> CQL to me is just another SQL dialect

SQL is for relational databases. That's a different paradigm to CQL. The similarity is superficial. CQL is not a dialect of SQL. Not even close.

Mike_12345··on GitHub Copilot emits GPL code
> Any model would do

This does not apply to humans or machines.

By your logic, just pick any random bum off the street, give him the right training set, then he will become a 180 IQ genius and discover the unified theory of gravity and quantum mechanics.

Some models are just inherently better at modelling.

Mike_12345··on Agents are the next AI killer app after ChatGPT
Lets focus on the Boston Dynamics turbo ninja robot cats jumping through windows and bouncing off the walls with machine guns, lasers, and fully autonomous AI.
Mike_12345··on Valve Releases Proton 8.0-1c
It's not a snipe. You're bragging about Linux knowledge and your A2000. Just saying that definitely doesn't work on Linux without some effort and you would buy it knowing there was some effort involved. If you want a no hassle easy install for gaming you get a consumer AMD GPU, not a professional Nvidia GPU. The context of this thread was gaming on Proton, not whatever you are doing.

> Expecting people to buy based on an HCL is not reasonable

It's totally reasonable for an OS with ~1% market share.

Mike_12345··on Valve Releases Proton 8.0-1c
> RTX A2000

If you know that much about Linux, you should expect issues with Nvidia. AMD just works.

Mike_12345··on Valve Releases Proton 8.0-1c
No. Overwatch is not on Steam. So you would need to mess around with Wine or Proton separately. Maybe it works, but it's not a single click installation.
Mike_12345··on Valve Releases Proton 8.0-1c
For me it's the opposite experience. Easier to get Linux installed, no driver issues, no messing around, and my games Just Work on Steam/Proton without any tricks. Windows has more hassle with downloading/installing drivers and disabling crapware and forced ads in the OS before I can begin to use it.
Mike_12345··on Valve Releases Proton 8.0-1c
> Until all games start releasing native ports

More realistically, game devs can target Proton as a platform, making sure it works on that.

← PreviousPage 2 of 5Next →