timabdulla

You can unpack and view the code of any extension after you've installed it. There's even a rule against obfuscation, though I'm not sure how enforced that is.

A Chrome extension is basically a zip archive with a bunch of JavaScript inside. There's no safeguarding of the code within.

Many companies (like Vercel, Supabase, and so on) have built big businesses "wrapping" AWS. They literally compete with AWS and use AWS to deliver their offering.

This is a big market. There are room for lots of approaches.

I'm sure OpenAI, Anthropic, and Google will make a big business of this, but I don't see how you can rule out anyone else having good ideas and relying on big infrastructure providers to make them a reality.

> If it is dangerous in your anecdotes, why don't the reported numbers show more accidents when FSD is on?

Even if it is true that the data show that with FSD (not Autopilot) enabled, drivers are in fewer crashes, I would be worried about other confounding factors.

For instance, I would assume that drivers are more likely to engage FSD in situations of lower complexity (less traffic, little construction or other impediments, overall lesser traffic flow control complexity, etc.) I also believe that at least initially, Tesla only released FSD to drivers with high safety scores relative to their total driver base, another obvious confounding factor.

Happy to be proven wrong though if you have a link to a recent study that goes through all of this.

He mentions the entropy in the article...

I find it interesting that the folks at Qodo are not clear on why more groups try to push the needle on SWE-Bench rather than on Codeforces.

I think most software developers would find that performance on SWE-Bench (wherein an AI system has to resolve a real-world Github issue) is much more relevant to their day-to-day than the raw algorithmic problem-solving capabilities of an AI system.

Competitive programming questions are challenging, obviously, but they are tightly defined with prescribed inputs and outputs and the solutions are usually compact, though hard to arrive at.

In contrast, real-world programming is much fuzzier and more ambiguous. The solutions that one must implement are usually much larger in scope and require much greater project-specific context to build. And of course real-world tasks are generally not things that demand lots of formal data structures and algorithms knowledge, like one would need to do well at Codeforces.

What's interesting is that you can have systems like o1 or AlphaCodium that are much, much better than the median software developer at solving tricky algorithmic puzzles, but that also can't do very well at SWE-Bench, which largely comprises Github issues that have been estimated to take <1h for a human developer to do. Even though o1 would absolutely dominate Claude 3.5 Sonnet at competitive programming questions, it seems to basically be a wash on real-world tasks.

So I suppose my question to Qodo would be: Why the emphasis on Codeforces as a benchmark? It's quite clear -- and has been for some time, since e.g. AlphaCode in 2022 -- that AI systems can be really powerful when it comes to solving Codeforces-style questions. What seems much harder (for now) is making significant progress on real-world development tasks.

Neat. Do you have the Bananalyzer eval results for Tarsier published somewhere?

Highly doubt it. You can't insure the inevitable.