169 karma · joined July 23, 2024
https://www.klarna.com/international/regulatory-news/klarna-...
> I do so philanthropically without any profit motive (typically losing money)
the cost (and hassle) of consulting with a lawyer is potentially a lot in relative terms.
That said, I thought that the rule in the UK was generally that the loser pays the winners costs, so I'd think that limit the costs of defending truly frivolous suits. The downside risks are possibly still high though.
He has an incredible mind.
Yeah, I'm a little confused at what you're offering over existing options (e.g. Wealth front, Betterment, M1). If the answer is just "it costs less" won't you raise prices when it comes time to make money? I suppose "the thing that already exists, but slightly better" is not necessarily bad.
I don't really see this bias existing to the extent you seem to be implying. The first Trump shooter didn't have many apparent political motives, and the second was completely crazy. The media definitely stops talking about things soon after they happen, but I don't have the impression that they're turning the a blind eye to right wing extremism. I mean, just look up "right wing extremism US" on Google news.
I expect that in this case, the shooter's motivations will defy the standard left/right split as more information is released, much like his apparent influence Ted Kaczynski. You mention Rogan, but Rogan supported Bernie Sanders in 2016. I would say liking Rogan shows an anti-authority streak more than anything.
As for why right wing politically motivated attacks seem to be more common than left wing ones, I'd suggest a few causes.
- People who have access to guns, grew up around guns, are willing to buy guns are more likely to be right wing, and are more likely to be identified as right wing for obvious reasons.
- People with low agreeableness and high neuroticism are more likely to be right wing, and those traits perhaps make politically motivated attacks more likely.
- Extremist right wing thought has perhaps been more effective in recruiting followers than extremist left wing thought since the fall of the Soviet Union.
2. The point of the test is to determine whether the model has the capability to act against its users and hide those actions, not whether it will under normal circumstances. Some models aren't powerful enough to do so.
3. The behavior occurred even when the "at all costs" was not included in the prompts, though very infrequently.
I've found they struggle with obscure stuff so I'm not doubting you just trying to understand the current limitations.
The point is whether the system will come up with plans to work against its creators goals, and attempt to carry them out. I think you are arguing that outputting text isn't running a command. But in the test, the AI model is used by a program which takes the model's output and runs it it as a shell command.
Of course, you can deploy the AI system in a limited environment where its outputs don't trigger arbitrary shell commands, or sandbox/monitor it so that it can't access it's oversight mechanism. But people do already deploy their AI agents with access to the shell. in unsandboxed environments that don't have any monitoring or oversight. Plus, these evaluations are partially to show how much OpenAI has to worry about sandboxing, monitoring, etc.
Moreover, OpenAI has already talked about using its models to automate portions of AI development. In that setup, presumably the AI model would be called by a program deployed on a server with access to the shell, ability to run commands, access to GPUs, etc. It seems pretty important to ensure it robustly does what it's creators intend in that case.
- Lead paint
- Water supply
- Lead in cookware
- Lead in spices, especially tumeric, where leaded dye is used to make it more yellow
- Industrial airborne emissions
Leaded gasoline used to be on that list but not anymore. Not sure about the specifics in Bhutan.
If you'd like to help prevent lead exposure in countries like Bhutan, the two charities I know about are Pure Earth and LEEP. Not sure if they have Bhutan programs.
https://www.unicef.org/bhutan/press-releases/national-blood-...
Now I don't think 10M is the correct denominator. It's unlikely all those workers are in construction where a lot of the deaths are presumably concentrated. According to this[2], there were around 4.4M foreign construction workers in 2016. 3k/5M (assuming numbers have increased) is 60/100k, so around as dangerous as the most dangerous professions in the US.
But this also doesn't account for the alleged missing.
[1] https://www.bls.gov/charts/census-of-fatal-occupational-inju...
[2] https://iopscience.iop.org/article/10.1088/1755-1315/140/1/0...
Yoshua Bengio has written in detail about his views on AI safety recently[2][3][4]. He seems to put less weight on human level AI being very soon, but says superhuman intelligence is plausible in 5-20 years and says:
> Faced with that uncertainty, the magnitude of the risk of catastrophes or worse, extinction, and the fact that we did not anticipate the rapid progress in AI capabilities of recent years, agnostic prudence seems to me to be a much wiser path.
Hinton also has a detailed lecture he's been giving recently about the loss of control risk.
In general, proponents see this as narrowly tailored bill to somewhat address the worst case worries about loss of control and misuse.
[1] https://www.theregister.com/2023/07/28/ai_senate_bioweapon/
[2] https://yoshuabengio.org/2023/05/22/how-rogue-ais-may-arise/
[3] https://yoshuabengio.org/2023/06/24/faq-on-catastrophic-ai-r...
[4] https://yoshuabengio.org/2024/07/09/reasoning-through-argume...
I think you're right that I misread your first two paragraphs though. Sorry.
The point is that just as the car replaced the horse, programs on computers are set to replace the human mind. Right now, we see only the beginnings of the trend; inventions which look interesting but limited. But eventually, and perhaps sooner than many believe, the replacement of human cognition will radically transform the world in unprecedented ways.
The table with a comparison to alpha fold gives a less than one percentage point improvement.
- nuclear fission, which provides an abundant source of environmentally friendly energy, but allows people to make bombs capable of wiping out whole cities at once (and potentially causing nuclear winter)
But even in that case, I believe that it's a good thing that we have access to nuclear power, and I certainly want us to use more nuclear power. At the same time, I'm very glad that a bomb is hard enough to make that ISIS couldn't do it, let alone any number of lone wolf terrorists. So I think I would apply the same logic to biotechnology; speeding up medical progress seems extremely valuable and I'm excited about how AF and other AI systems can help with this, but we should mitigate the ability for bad actors to use the same tools for evil.
An aspect that's unique about biotechnology that's different in comparison to the examples you gave is that most of those technologies help good and bad people approximately equally, and since there's many more reasonable than crazy people they're not super dangerous.
There's a concern that technologies that make bioengineering easier could make it easier to produce and proliferated novel pathogens, much more so than they make it easier to prevent pandemics; in other words, it favors "offense" more than "defense". The only one example you listed that has a similar dynamic in my mind is the bosch-haber process, but that has large positive downstream effects separate from its use for ammunition. Again, this is not to say we should stop medical progress, but that we should act to mitigate the dangers, and keep this concept in mind as the technology progresses.
That said, I'm not certain how much the current tools are dangerous in this way. My understanding is that there is lower hanging fruit in mitigating these issues right now; for example, better controls at labs studying viruses, and better vetting of people who order pathogens online.
[1] https://en.wikipedia.org/wiki/Tokyo_subway_sarin_attack
[2] https://en.wikipedia.org/wiki/Masami_Tsuchiya_(terrorist)
Also the paper says they basically copied the methods used for AlphaFold, but then included the ability to input language embeddings, and input some other side constraints that I don't have the biology knowledge to understand. They don't show any data that indicate how much these changes improve performance. They show a very modest improvement over AF3 (small enough that I would think it could be achieve through randomness/small variations in the training parameters). So I don't think this is very revolutionary, but I suppose it replicates AF3.
I don't know how much releasing this model is a delta on safety, but we certainly need to do a better job of vetting who can order viruses; my understanding is there's very little restrictions right now. This will become more important as models get more capable.
I remember I looked into a lockup I was experiencing (this was a few years a ago so I forget the mode). Turns out emacs was writing to disk every so often and this was noticeably causing stutters in the main thread.
There's also a strange bug (at least I think it's a bug and not a feature that I don't know how to use) where the cursor will still exists and clicking will move it around, but the window doesn't respond to keyboard inputs.
> This is one reason several closed providers consistently lobby governments against open source.
Is this substantially true? I've noticed a tendency of those who support the general arguments in this post to conflate the beliefs of people concerned about AI existential risk, some of whom work at the leading AI labs, with the position of the labs themselves. In most cases I've seen, the AI labs (especially OpenAI) have lobbied against any additional regulation on AI, including with SB1047[1] and the EU AI Act[2]. Can anyone provide an example of this in the context of actual legislation?
> On this front, open source should be significantly safer since the systems are more transparent and can be widely scrutinized. Historically, open source software has been more secure for this reason.
This may be true if we could actually understand what was happening in neural networks, or train them to consistently avoid unwanted behaviors. As things are, the public weights are simply inscrutable black boxes, and the existence of jailbreaks and other strange LLM behaviors show that we don't understand how our training processes create models' emergent behaviors. The capabilities of these models and their influence are growing faster than our understand of them, and our ability to steer them to behave precisely how we want, and that will only get harder as the models get more powerful.
> At this point, the balance of power will be critical to AI safety. I think it will be better to live in a world where AI is widely deployed so that larger actors can check the power of smaller bad actors.
This paragraph ignores the concept of offense/defense balance. It's much easier to cause a pandemic than to stop one, and cyberattacks, while not as bad as pandemics, seem to also favor the attacker (this one is contingent on how much AI tools can improve our ability to write secure code). At the extreme, it would clearly be bad if everyone had access to a anti-matter weapon large enough to destroy the Earth; at some level of capability, we have to limit the commands an advanced AI will follow from an arbitrary person.
That said, I'm unsure if limiting public weights at this time would be good regulation. They do seem to have some benefits in increasing research around alignment/interpretability, and I don't know if I buy the argument that public weights are significantly more dangerous from a "misaligned ASI" perspective than many competing closed companies. I also don't buy the view of some in the leading labs that we'll likely have "human level" systems by the end of the decade; it seems possible but unlikely. But I worry that Zuckerberg's vision of the future does not adequately guard against downside risks, and is not compatible with the way the technology will actually develop.
[1] https://thebulletin.org/2024/06/california-ai-bill-becomes-a...