PavleMiha

I think the last few weeks have raised some important questions on what Mathematics is, and what it's for, that might be raised about a lot of other fields in the future, so I wrote about it!

I think the point here isn’t whether that claim is true or not, it’s that people at OpenAI believed it to be true and proceeded anyway.

The point is that these tools are given broad goals, and they do not pursue those goals in the way that we'd like them to. There is hacking information in the training set, and also all sorts of other dangerous information. There is a risk that as these tools become rapidly more powerful (remember GPT-3 was 6 years ago!), if they are given a broad goal, they might pursue that goal in such a way that harms a lot of people. As these tools become cheaper, there will be a lot of people telling them to pursue all sorts of broad goals, and each of those instances has some chance to harm a lot of people, so you really need to get it very correct so that the harm doesn't happen.

Removing harmful information from the dataset could be a way to do this, but it also makes the tool less useful, and it's hard, so companies aren't really doing that. There's the additional issue that with the rise of Reinforcement Learning being used to train these tools, they're not just learning from their training data - they basically try a million things and then get rewarded for doing things that work - so they can even discover hacking techniques from scratch.

Additionally, and not completely relevant to this discussion, there is a possibility that some users ask the tool to pursue goals that purposely harm a lot of people, such as developing weapons, hacks and viruses.

The things that's "new" here is that the tool is both very good (meaning, for example, that it's much easier for me to hack into an online service with an agent powered by a frontier llm than it was using google 6 years ago), and hard to control (google never hacked into an Australian government database when I asked it to find me some information).

So yeah, an LLM powered agent is a tool, and Google is a tool, and a hammer is a tool, and both can be used for good things and bad things, but the agent is (much) more powerful and more unpredictable. It also seems like the agents are getting more powerful and more unpredictable by the day - we didn't have this issue with GPT-3 or even the first LLM-powered agents - so people are very worried about what the agents 6 months from now will do, both when asked to do harmful things on purpose, and when asked to do harmless things.

Are you not worried? And is that because you think these incidents are basically the AI companies making them happen on purpose for marketing?

So this tool seems very powerful and difficult to control and steer. Agents not doing cybersecurity related tasks have also gone on to hack various companies, people and countries, which they weren't supposed to do. This has now happened to pretty much every company developing frontier llms, so it seems to be a fundamental issue with these tools, and it's an issue that worries a lot of people as these tools get more capable.

I often read things like this, and then I go look and don't quite see the cheering:

https://www.cbc.ca/news/politics/emergencies-act-banks-ottaw... https://www.bbc.co.uk/news/world-us-canada-60383385 https://www.businessinsider.com/trudeau-canada-freeze-bank-a... https://www.newsweek.com/banks-have-begun-freezing-accounts-... https://www.nytimes.com/2022/02/22/world/americas/canada-pro...

Maybe we have different ideas of what the mainstream press is? These all seem pretty normal articles, most note what critics are saying of the measures.

This is addressed in the article, ugliness probably isn't causing NIMBYism, but it is contributing to conservationism, which is a slightly different thing, and affects house-building in different ways (most conservationist concerns are not quite NIMBY, as most of the proponents of conservationism don't live in the neighbourhood being conserved).

Did you read Trump's post about the mission calling the previous administration incompetents? That predates Berger's post?

But Crew 10 had also been planned for a while, so the narrative that Trump ordered a new ship to go up "NOW" can't be true. For example, heres a post about NASA moving the launch date to March, in December last year: https://www.nasa.gov/blogs/commercialcrew/2024/12/17/nasa-ad...

> That's just being realistic, isn't it?

As evidenced by this situation it's not true that any criminal can give money to any politician and expect favours. Perhaps some criminals curry more favour, and perhaps some politicians are more transactional.

12 Pool means build your spawning pool at 12 supply, usually the number before the building is at which supply you build the building, and it assumes you're constantly producing workers.

I guess it is? No one can force Apple to sell iPhones in Europe.

Quoting is very different from posting the full contents of something. I can quote a book but I can’t reproduce it in its entirety.

It's both, the methods by which the election is made secure and accurate should be understandable by most people. Most people don't understand cryptography and cybersecurity.

Feels obvious to say this, but very odd that the article didn't mention, lying is bad, especially when done for profit.

Speculating, but Apple wanted to make sure that the only people photographed using the headset were models and actors who are very attractive. That way whenever a news outlet runs a story about it, they have to use the picture of the attractive/cool people using it, and people make an association that the headset makes people attractive/cool.

And now they’re down to 2,300 according to Musk. Those are huge layoffs, even if we take pre pandemic staffing numbers as a baseline.

https://twitter.com/elonmusk/status/1616706530841333761?s=46...

Unless you have some information I don’t, Twitter is way below its pre-pandemic staffing levels. Obviously hard to tell exactly how low they went as they’re not public anymore, but this is quite different from Meta and such that really did just go back to pre-pandemic level. CNBC say Twitter is at 1300 people, a drop of 80%. It’s mind boggling.

Here’s almond milk referenced in a recipe in 1410:

“For to make blomanger. Nym rys & lese hem & washe hem clene, & do þereto god almande mylke & seþ hem tyl þey al tobrest; & þan lat hem kele.”

People have used the word milk to refer to some plant products for too long for this to confuse people now.

Just don’t allow that, it doesn’t seem to be a slippery slope at all. Is there any evidence people are getting fooled by oat milk or nut cheese?

Google isn’t worried about individuals suing them, they’re worried about governments or entities like the EU fining them

I get criticising science journalism, but this isn’t even a metaphor, it’s one of the meanings of “spar” as per most dictionaries: to argue.

On the other hand, why would people use their collective power to sink the company they work for? No company, no jobs. In reality the power has to be balanced between the two, and both parties need to act carefully and rationally.

There are other potential explanations for your observation[1]. Do you think you could be subject to confirmation bias?

[1] https://www.reddit.com/r/dataisbeautiful/comments/x2jj3s/obi...

Sorry, I'm definitely not very familiar with macro-economics, but is your opinion that any amount of inflation is bad?

Yes, and also two of the trusses to the secondary mirror (these are the two additional horizontal lines). The Hubble Space Telescope gets 4 lines because of its 4 trusses.

Carbon offsets are super complicated, potentially shady, and I don’t fully understand them, but the gist of it is that they pay money to companies to either not produce carbon that they otherwise would have, or use the money to install and operate equipment in factories and powerplants that captures carbon before it goes out into the world. You can also do things that pull carbon out of regular air but I think the first two options are so much more efficient that they dominate the market. It seems like the system could be gamed, but as far as I know it broadly works.

The article seems to state there wasn't a lot of value in the cold wallets.

Would you actually be willing to bet money on this 1 to 1? I. e. that Holmes serves no days in prison.

This looks really cool, I found the line "inherently safer than digital computing in the face of cyber threats" pretty funny. There's no analog internet so what could the cyber threats even be?

My problem with this is that we have little proof that people are any good at this sort of task (i.e. monitoring something without intervening for really long stretches of time).

So if data show that people keep failing at it we have to take a look and decide that it's not a good system, not that the people are at fault.