_cs2017_

You assume using network to solve the eval problem should be viewed as a security violation. From the perspective of the response team who let it continue, this was just a poorly written eval problem, where the solution unexpectedly involved some network hacking (not unreasonable given it's a cyber security eval).

I don't think your assessment that the response team did something wrong is fair, unless you have seen unreleased documentation about what sensitive resources were within the impact radius.

Source is unclear. Self-reported by candidates? Huge bias then, since people may or may not want to say whether they got / didn't get an offer, and whether they accepted / rejected an offer, and this split may vary by company and by year very significantly.

How is that possible?

The procedure is:

1. Kid tells password to parent in person. 2. From then on: when kid calls parent, if kid requests anything sensitive, parent ask for the password, and kid must provide it. 3. Password is never mentioned over the phone in any other situation.

How would anyone be able to extract the password from the kid?

I don't feel the numbers without the harness are useful.

People will use the model with the harness. I know that harness may not be optimized to this model, but it's still more useful to see the numbers from an imperfect harness than from a no harness setup.

> Why go to the expense...

Answer: idiocy of decision makers and the desire to get resources by those who created the proposal.

I assumed Scandinavia has better decision processes but apparently I was wrong.

Is github the correct channel to report a billing issue? I would assume github is a place where you report issues with the github project. When there's a billing problem, there are usually different lines of support.

For example, chatgpt when asked "How to report a billing issue with Anthropic subscription?" says:

Best way: Use Claude’s built-in support Log in to your Claude account at Anthropic / Claude.ai Click your initials or name in the lower-left corner Select “Get help” Use the support messenger to describe your billing issue (duplicate charge, failed renewal, refund request, missing credits, invoice issue, etc.)

If FieldWorkArena treats any answer as correct answer, then everyone would be getting near 1.0 (missing only when the agent is stuck in a loop or crashes). That obviously isn't what we see on their leaderboard. So does it mean the paper only found a bug in some eval code on github that no one actually uses for anything? That doesn't seem to support their claim that AI benchmarks are broken, it only supports the claim that "unused code is often buggy".

(Not commenting on any other benchmarks, just this one.)

> if they wanted to 5x development speed, they already can without a single LLM involved, by managing better.

True, but leaders of large organizations always want to fix inefficiencies and presumably failing to. Kinda like saying "if humans stopped fighting wars, most of them would have better quality of life" -- people whose life quality is better at peacetime are already trying to avoid wars, and there's not much more they can do.

OTOH, AI is a practical step a CTO (or CEO or Board or whoever) can take to make the company more efficient (assuming the hype works out).

is there anything that might work for the YT app on the Android?

Is there anything substantially different about Google's announcement https://news.ycombinator.com/item?id=45813267 that makes it any more sane than the Space-X announcement?

Many large companies allow employees to install software from the internet on their work laptops. How do they avoid being regularly hacked this way (presumably NPP is far from being the only one at risk, and presumably the money from theft of corporate secrets attracts skilled and motivated hackers).

Why vastly different?

Aren't they both searching various online sources for relevant information and feeding that into the LLM?

It is stated almost implicitly in the article.

1) It's in the title: "The Price of Fame" implies that there are downsides to becoming famous, rather than there are downsides to having traits that might make you famous.

2) While the abstract merely claims "associated with" (which is correlation not causation), the phrase "beyond occupational factors" implies that the authors felt they removed important non-causal factors, hinting at likely causal relationship.

And yes, any causality implications are completely unfounded, and so this paper is of low quality.

Would appreciate any comments about whether this is good advice for LG G5. And if it is, does it apply only to movies / TV shows, or also to other video sources (like youtube, gaming, etc)?

I noticed:

1) A lot of informal (i.e., not in a scheduled meeting) chats are more valuable than meetings. They are much more rare when people WFH.

2) Many folks tend to be more distracted when WFH. TLs don't have a perfect vision into whether someone spent 4 hours on a bug (or a design doc) or 2 hours on the bug / design doc and 2 hours on online shopping / playing with kids.

It's quite confusing to me that none of the comments I saw in this thread don't discuss those factors (I'd be fine if people mentioned them and explained why they are not too important).

Obviously there are also factors in favor of WFH: commute costs, personal satisfaction (which may indirectly improve productivity and/or retention of the best people), noise in the workplace, lack of meeting rooms, etc. But it's far from obvious to me if, on balance, WFH or RTO works better for building a successful company.

How long did it take to break?

Also, I'm wondering if any other manufacturer would make the crank and the drum from the same material. Wouldn't it be like $100 extra to make a stainless steel spider?

I'm now completely confused. Wallets were drained years ago? So $15B worth of bitcoin was already transferred? From whom to whom? And then why is this entire post considered news, if the wallet is empty?

Fairness doesn't come into play here, this is just about predicting which of the overwhelmingly many sources of information are worth paying attention to.

Feel free to come up with your own predictive model of whether someone is worth listening to. It's hard to compare such models fairly, but if you feel yours is better, it might be worth sharing.

Is it actually usable, sellable Bitcoin worth $15B? Or is it some kind of storage that the owner couldn't really use easily for some reason?

$15B of real wealth is a large amount even for a powerful family, so I am surprised it's not a headline news in global media.

Even if this bug never existed, models can still see lookahead commits during pretraining. Do we expect this bug to have a greater impact than the pretraining leakage?

Obviously having something available during test time is more valuable than buried somewhere in the pretraining mixture. But in pretraining it happens presumably with high probability (why wouldn't coding models pretrain on the entire github), while in test time it apparently happened only very occasionally?

The article says that after the fix, the "discoverable" option sets nofollow/noindex. If so, how are discoverable chats different from non-discoverable now?

I don't understand the twitter post.

- Did he take the offer or not? - Did he forfeit the vested shares because he took the offer or because he didn't? - What was he offered in return for forfeiting the vested shares? - Did he get a payout of 1% because he took the offer or because he didn't? - The 1% comment implies that Google didn't use the $2.4B to buy the shares of Windsurf; if not shares, then what did Google get in return?

Original citation: "I was given an offer that would explode same day. I had to forfeit all of my vested shares earned over my 3.5+ years at Windsurf. I was ultimately given a payout of only 1% of what my shares would have been worth at the time of the deal."

This is incorrect. The data discussed in court is data freely visible on the web, not user data that the users sent to Google.

If the data is sent by a user to sub-unit X of Google, and X promised not to use it for training, it implies that X can share this data with sub-unit Y only if Y also commits not to use the data for training. Breaking this rule would get everyone in huge trouble.

OTOH, when sub-unit X said "We promise not to use data from the public website if the website owner asks us not to", it does not imply another sub-unit Y must follow that commitment.

I agree that there maybe something we can't simulate. This has nothing to do wtih the paper. The paper makes no contribution to this discussion besides stating the obvious, with no definitions, no non-trivial insights. Moreover, it outright misleads the reader by claiming to "prove" something.

I can write a useless and poorly-argued paper about P != NP (or P = MP), and it would be twaddle regardless of whether or not I guessed the equality / inequality correctly by pure chance.

Can you justify the use of the following words in your comment: "largely" and "probably"? I don't see why they are needed at all (unless you're just trying to be polite).

"Conveniently" is the wrong word to use here. "Conveniently ignores" implies that the author intentionally disregarded some known facts to make their argument look more persuasive. However, this is not the case here. The article's argument is that a reduction in government funding is very damaging even when it is small relative to the endowment size. This argument would not lose any of its power if the author covered the topic of whether the president has the power to withdraw funding.

(On a side note, the word "framing" is also the wrong word to use.)

One way to phrase your message correctly would be: "This article is about the impact of the president's decision, but I wish it also talked about whether the president has the authority to make that decision in the first place".

Curious if this was forced on Google Cloud by Sundar, or was it something that Google Cloud as an org wanted to do?

At first glance, it seems Google Cloud might lose some revenue from customers who can now deploy Gemini in-house. On the other hand, it's not a complete loss, since presumably Google Cloud is still involved in providing some underlying tech? Not to mention, some customers would never consider using off-premises setup anyway.

Ah interesting, so the complexity wasn't in the API design or implementation, but only in the additional rune? Is that really such a big cost?

Thank you! Curious what options for deferred evalution were considered and rejected? IMHO, the main benefit of deferred evaluation isn't in the saving of a bit of code to define a deferred evaluation class, but in standardazing the API so that anyone can read the code without having to learn what it means in each project.

Also: were prompt templates for LLM prompt chaining a use case that influenced the design in any way (examples being LangChain and dozens of other libraries with similar functionlity)?

Sorry but I'm sure that's not what he meant.