CSMastermind

I mean yes that is what you'd say if you were writing a blog post about your new benchmark.

DeepSWE is the benchmark you want to actually look out for. Only one that aligns with actual user reported results from trying the models.

There was a point in time when this would have been untrue of Google. They didn't manifest into this market position, they rose there over the years after starting with a relatively modest product offering. Nothing saying someone else can't do the same.

I couldn't get trading working but maybe I'm doing something wrong.

I'd argue the value of college has been steadily declining ever since we tried to push more and more people through it. When it's a filter it's a good signal, when it's a participation trophy far less so.

A blanket cap makes no sense to me. There's a power distribution of AI use in my company and I'd imagine it's the same at a much greater scale at Uber.

I'd guess there should be a few people Uber is bascially allocating unlimited AI spending to and a large swath they're giving basically nothing.

One of the most attractive things a company can offer its engineers right now is a large token/compute budget.

I realize this is supposed to be a post about how scary the security vulnerabilities these agents will find are.

But personally I love when agents do things like this and appreciate the help. Last thing in the world I want is for them to nerf the models.

What's the business model? Their core functionality, while useful, seems like something that will just be an open-source package. I assume there will be some Saas layer on top of it?

Was this written by AI?

MCP is essentially just JSON RPC with a few special fields that must be included. I have reservations about JSON RPC, but there needs to be some 'service discovery' layer for LLMs to interface with.

It needs to be available in places like websites, desktop applications, backend services, etc. The CLI is only one place that these systems interface with.

Whatever you replace MCP with will be in a similar shape even if you specify a different communication protocol or different fields for tool discovery.

AI customer service bots are awful. Their only redeeming feature is how bad most customer service processes already are.

Wow cool to see.

Will more copilot usage fix this? We should try more copilot.

This happens all the time, it's a classic phishing tactic.

> The player’s choice is not restricted to pieces that have been captured previously

I grew up playing chess but my grandfather always insisted that pawns could only be promoted to captured pieces so when I played him we had to play a that variant.

I suspect this came from players not having extra pieces with their chess sets.

If anything I think Microsoft is IBM

I got downvoted for saying this last time the topic came up but constraints focus a project. It’s best to start work with as few variables as possible, and only add new ones when absolutely necessary.

I'm working on a similar problem in computer vision and we're quickly approaching the point where our pure vision work is better than our Lidar supported track because we've had to deal with the constraints instead of having a crutch to lean on.

More people should try Grok. I don't use it for coding but it's replaced a lot of my ChatGPT usage. Definitely more perferred model for quick questions or easy answers.

I'm blown away by the things people think to create.

No, Erdos problems were accepted as sort of a benchmark. There's a bunch of reasons they're favorable for this task:

1. They have a wide range of difficulties. 2. They were curated (Erdos didn't know at first glance how to solve them). 3. Humans already took the time to organize, formally state, add metadata to them. 4. There's a lot of them.

If you go around looking for a mathematics benchmark it's hard to do better than that.

> It’s crazy to me that two decades after the iTunes Store the trade and resale of digital goods isn’t protected by law.

You aren't buying a digital good, you're buying a limited license to use that digital good.

That hasn't been my experience, I use LLMs in place of search. Two examples from tonight that I asked Grok for:

> Can you find the girl who did a bunch of posts critisizing David Graeber's Debt? I thought it was really well done

> I saw a comment on hackernews a while ago about the optimal amount of credit card fraud being higher than zero because of game theory dynamics, can you find it.

In both cases it turned up the exact posts I was looking for in like 30 seconds which would have taken me much longer using traditional search. I've had similar success looking for technical documentation. It's downright magical how they're able to turn my vague idea of what I'm looking for into a pointer to the exact thing.

This was the promise of Bing that never materialized.

Aren't these kinds of watermarks easy to remove or distort? Seems like they're only helpful as long as people are relying on them sparingly so it's not worth the effort to circumvent.

If social media platforms started banning images with these watermarks seems like they'd be stripped out overnight.

Sure I speak to ChatGPT all the time and I've used the feature you've linked but it can't do the things I described. It won't be like, "hey let me go look into that" and then come back in 3 minutes with an answer. It's essentially a dictation feature.

Not as far as I know. Can you walk me through the process?

I have my phone, I'm going for a walk, what app am I opening?

Still not useful enough for me and I really want this feature!

The problem I encounter is the inability of the LLM to look stuff up and respond to me. "What's that name of that database table?" "What are all the services that call this endpoint?" "Are there any open PRs for this repo right now?"

Once information can flow in both directions not just one it will be a gamechanger for me.

This war game has been discussed many times.

One thing to consider is that Van Riper summoned assets unrealistically he used small boats to avoid detection but then attributed load outs that they couldn't realistically carry.

He also moved information unrealistically assuming that his units could communicate as efficently with paper moved by hand as they could with radios.

There are real, valid criticisms of the lessons we should have learned from the exercise, but it's not as simple as most analyses make it out to be.

This brings me back to the glory days of StumbleUpon. Highly recommend.

Elon argued that even though the events in question took place sometime between 2017 and 2020 OpenAI intentionally hid the information from him until 2022-2023 which is why he wasn't able to file the lawsuit until 2024.

That's what the jury found against - they said he was reasonably informed enough to have brought the suit earlier and thus the 3 year clock should start ticking in 2020 not 2023.