2,745 karma · joined December 30, 2012
This seems to be less true on YouTube and Reels unfortunately.
“Anti-science” Americans are not less intelligent on average than “pro-science” Americans. It’s simply a difference of who they trust to give them the facts. One group trusts and believes the public health organizations; the other group treats those same organizations with extreme skepticism. No solution to this problem will be effective until trust in the system is rebuilt.
The core problem that no amount of trying to “get through” to people will solve is that America’s public health and science institutions actually are deeply problematic organizations that are politicized and do not always act honestly and in the best interest of the people. The media, meanwhile, has lost all trust with a significant segment of the population who rightly recognize it as a propaganda machine for the ruling elites.
Why should my parents, who are quite intelligent but never received any significant science education, believe what they are told about science by experts that they can clearly recognize as untrustworthy?
You have to recognize and address this before any progress can be made.
Is that true, though? Training runs for frontier models don’t happen without enormous resources and support. You don’t run one in your garage. It doesn’t happen unless people make it happen.
Is this really a harder coordination problem than, say, stopping climate change, which Gates does believe is worth trying?
The rise of amateur sleuthing is less clear. I can’t actually think of a single case where the true crime community actually solved a case that the police couldn’t. But you regularly see a mob of weirdos going after innocent people and trying to ruin their lives.
Good technical writing is concise and tells you what you need to know as directly as possible. ChatGPT written documentation always reads like bad marketing copy.
Once you have that working, you can generate synthetic examples to train the next version of the LLM, or for open source coding models, a small LoRA file that anyone can download to add support to their model of choice.
> “On a dataset of human-written essays, we find that gpt-3.5-turbo can accurately infer demographic information about the authors from just the essay text, and suspect it's inferring much more.
> Every time we sit down in front of an LLM like GPT-4, it starts with a blank slate. It knows nothing about who we are, other than what it knows about users in general. But with every word we type, we reveal more about ourselves -- our beliefs, our personality, our education level, even our gender. Just how clearly does the model see us by the end of the conversation, and why should that worry us?“
https://www.lesswrong.com/posts/dLg7CyeTE4pqbbcnp/language-m...
Easy: By not using that optional feature.
If you’re like 99% of developers you really don’t need stuff like that until your app reaches a certain stage of growth, if ever - and React doesn’t force it on you.
I found it to be really insightful all the way through and well worth the time to watch for anyone with an interest in Feynman.
I think you’re overthinking this. System 1 thinking as the term is being used by AI researchers means making a fast decision based on reasoning processes that are wired into your brain by evolution. For any task that humans have faced for millions of years this works well. It can also work well for experts in a domain who have practiced a task so many times that their brains have adapted to perform it unconsciously.
System 2 thinking is consciously using explicit reasoning techniques to think through a problem, slowly and rigorously, often in ways that feel unnatural due to our cognitive biases but can solve problems that System 1 is unable to.
The analogy to LLMs is straightforward: LLMs learn to solve many kinds of complex problems during training and encode processes for those specific problems. They can then perform these tasks in a single forward pass through their weights. This is System 1 for LLMs and again, works well for any task that they were exposed to repeatedly during training.
However they don’t generalize to tasks that were not well represented in the training data. Training them to use explicit reasoning strategies instead (System 2) is shown to improve performance and let them solve a broader range of problems.
OpenAI trained o1 to pick better steps in its chain of thought. (The moat is the dataset they used to do that.)
Didn't these "few tokenizer related tests" prove the API was using Claude's tokenizer instead of Llama's, based on how words were being divided into tokens?
That's a hard one to explain (it doesn't appear they're even trying to).