I don't think we're in a bubble yet for AI. We've only really reached the software engineering market, and even then it's not permeated throughout every software engineering firm. There are so many other engineering disciplines to accelerate!
I wonder if the US will actually be fine here. We're starting to restrict immigration (at all levels, legal, illegal, skilled, unskilled) and boomers are retiring en-masse. Will the productivity gains negate the loss of labor. Labor isn't quite fungible but we might find some equilibrium between these 2 functions.
I think we'll end up seeing legislation against the license version of all of this content. Instead, you'll get to own it and move it around as necessary. This hasn't been necessary yet, but if that becomes the only option, then ownership becomes a more important question.
Pretty sweet. I wonder if they can continue to distill this down so it runs smaller. I think we're at a point where more intelligence isn't really needed for a lot of use cases.
Also honestly, the models don't have to get much smarter to handle the consumer side of things. They could innovate a ton on the harness to farm out requests and manage the state of the agent. They have an incentive to do this to reduce the infra usage so they can go wider with the agent.
Has anyone actually eval'd the other open source options against Jev on real world tasks rather than looking at benchmarks?
I see a lot of people parroting the quick open source alternatives as being better on the benchmarks, but it's such a new category that I'm not convinced we have solid benchmarks.
I'm hoping a company releases an internal eval benchmark for these options. I'm sure some of the open source ones are solid in some cases, but would love to see more reliable data.
This is pretty cool! I wonder if datacenters can get permits to colocate geothermal power + heat transfer. That might make permitting the actual datacenter significantly easier if there isn't a huge additional grid tie in.
These look great! I still have a Kindle Voyage from 2015 that's going strong, though I do wish it had the warmer LED lights. It's nice to have some devices like this and my Fujifilm XT-30 that don't make me feel the constant need for upgrading.
Congrats to Google on this! I wonder when the labs will start requiring commits in spend. It must be gnarly to do capacity planning if users swap between models every few weeks.
I mean think about the magnitude of services and conflicting information there probably is from the us government. Just getting clarifying information on the trusted traveler program is a pain today, and that's just a single service. Imagine trying to figure out medicaid eligibility, social security disability, and any number of other things that would be helpful.
Painfully this will fall to the court decide and create the precedence here. If this is an official government app, then the onus probably is on the government to provide correct information? Who knows, it could go either way and the courts could say "you're on the hook".
I don't think we have a good answer for liability here, like for Tesla FSD, Waymo, or even the current frontier models.
OK, there's a lot of negative comments here, but this is a great idea at a high level. It really is hard to figure out where to do a thing and it's also very easy to get phished. If they can figure out how to help people get all the services they're eligible for that'll be an amazing improvement for the people.
Yeah, what things can you write to statically eliminate the things that are not acceptable. What would you have to change in your prompting process to get that outcome? What bad patterns is it copying from the code base that maybe you should spend tokens fixing?
I do agree that they're not great at program design by default and that's where we as engineers should spend our time. Data structures and data flow are king. But once you suss that out, they're pretty good at writing the resulting code.
This is also where I disagree with dhh about just using lower level languages. Good abstractions make for excellent program understanding and we should continue to build extremely good building blocks that make program design naturally solid.
True, but if you can think about tests as "how do we validate that the outcome I want has been solved", then you can orient your tests around that. I do wonder if BDD style integration tests will end up being the path we go down.
Frontier models today don't really write incorrect code at the micro level. They do miss edge cases at the high level though, and that's what we want to test, is the scenarios.
The cat's out of the bag already. We can't undo the idea of LLMs or coding agents. If training progress stopped today, we have years and years of harness improvements to extract more performance out of today's models.
We also have open weight models too, and ways to host those at home.
Most people don't look at the assembler output of their C++ code (I used to write win32 programs in asm!). Most people don't look at the opcode instructions or JIT output of their ruby / python code. We're starting to work at a higher level of abstraction using LLMs. It's ok to be sad about it, but just being angry about it isn't going to change that there's a new world out there with a new skill set that's needed for honing.
There's different layers of understanding the system. I generally care about high level data flow, concurrency and performance (batching, holding transactions too long, back pressure etc.) rather than the mechanics of how the code actually does a thing. I still look to see what the final output looks like and ask my agent questions on how it fits in the larger system and evolve things if necessary, but agents are pretty good at writing code if the rest of the code base looks pretty decent.