Your Agent Will Pick the Hackable One
geoffstearns.com
geoffstearns.com
> This changes how people buy hardware. Ask your agent which TV, router, or synth to get, and it will lean toward the one it can change, because that’s the one it can fix later. The buyer never has to think about openness for it to matter.
It's just sci-fi and has nothing to do how the current LLMs work.
Humans can’t explain how humans think. We can tell you what we thought. And how it felt to think it. But we can’t explain the mechanics.
LLMs will be able to explain how they think or come up with their conclusions. Once that information is pulled into their data model. And it recognizes that the how LLMs think info usurps the decades of computers can’t think reality of the past.
My experience tells me that an agentic system can absolutely do this.
Of course, that ain't happening... Just because something became "easy" doesn't mean it'll get popular, and an average person doesn't know what the hell "firmware" even is. The author is very much leaping over the very first step of the chain there.
Imagine everyone in your life suddenly has a very capable hacker friend with lots of free time and loves helping out.
With something like a TV, okay, I could see that happening -- in a vacuum -- those bloody things need adblockers, right? So hey, agent, put one on there for me... but think about how many people don't even install one of those on their goddamn computer or phone. Ones that already exist. People will simply not consider using a device outside of its default configuration.
And specifically in the realm of hardware purchasing decisions, the popularity of the iPhone should also point you towards this fact. People don't want customization, they will often actively resist it for something that "just works". All the complaints about the device's limitations, all the discussions of "sideloading" and browser/JIT limits and whatnot they impose, all the talk about app store content restrictions and moderation problems, all of that? It has done nothing to harm them being a popular device, nor has it prevented LLMs from recommending them to the general public.
I do not see this changing. People, quite simply, do not care.
For it to be a default, though, for it to affect product rankings outside of very specific requests? That is what I don't see happening.
Imagine now a future where more people do this, more search results as evidence to help future agents find similar solutions.
Today’s proactive frontier agents are very capable of this kind of thing.
> I asked Claude to find me a way to customize my Vizio tv
Quite sure you can tell the difference and why the difference matters.
from https://www.youtube.com/watch?v=aQvSG8paVjM&t=6419s
> "Rather than trying to produce the actual meaning step by step from the input text so that we build up the model of the world and then can answer the question asked at the end, it does the inverse of that and says: 'I am going to build the world in reverse, tracking only the pieces of information that will affect the answer.' In a way, it's ray tracing it backwards... We're going to start from the thing we wanted to know and we're just going to follow back the bounces that are necessary to get that piece of information. We're not going to care about everything else that was happening in the world at this time until somebody asks about it."
I have not found this to be true. I like the rest of the article and the principle behind it but the title is not backed up at all
The mwave fm1 already targets a particularly nerdy, tinkering crowd in a very niche segment. No way Samsung and Sony relinquish their grip on TV, fridge or washing machine firmware :-(. I suspect instead the arms race between manufacturers trying to lock up and control their system, and hackers trying to breach it, will simply move up one level until it resembles Neuromancer AIs fighting that much more :).
And for a really cynical take - given LLMS will recommend the way they've been trained to recommend, how much longer until organic LLM agent recommendations are sponsored at the training and system prompt and harness levels?
Edit / update : not to mention, I think there are already more than 200 firwmares available for the fm1. How long until one or more of them is malicious or just bricks your device, giving most people reason to not dabble, and manufacturers the obvious scapegoat double down on "security".
I don't think this is necessarily a bad thing. Closed source proprietary software systems are anti-consumer as hell (why do I need a proprietary code scanner from GE to figure out what error code my dishwasher has?) and personally I'm hoping that intrepid hacking becomes far more normalized and popular.
But yeah, I doubt LLMs tend to lean so with a blank context. Not in my experience either. Not that I'm saying they don't (maybe some biases had emerged, it's not impossible), just that I doubt it.
As this practice becomes more common, the ai will pick up on that and start to recommend this path for those that would benefit from it.
> Then M-Vave did something uncommon for a hardware company. In October it published its own firmware as open source under GPLv3
They did it after the alternative firmwares were out of their control. What I take from that is that there is a big need to reverse engineer firmwares, because by defaults companies really don't want to open source them.
Ironic how the company released it as GPLv3. Like "we really wanted to keep it proprietary, but now that we feel like we should open source it, we really wouldn't want for a competitor to change it and keep their changes proprietary".
A less cynical view could be that they didn't realise that people _wanted_ to hack their budget synthesizers in this way, but reacted positively when they realised there was a spot in the market for an open(ish) synthesizer. And I mean it's working for them, their device is selling like hotcakes, largely off all of the available community firmware
When in doubt, the business people always choose to keep everything as closed as possible. I have had a boss ask me if I could encrypt some metadata "so that competitors cannot read it", for something trivial that I did in an hour.
> This is a pretty cynical view.
Well, they could have gone "okay, there are many open source firmwares for our device, we may as well use a permissive licence". But they went "actually now we care about the freedom of our users" with GPLv3.
You could say "they don't know about open source and they just picked a random licence", but I wouldn't believe it. EVERY company in the world knows that they really don't want to get close to GPL.
You could say "whatever they did, someone would have complained, and it is better than nothing". But it's not better than nothing: the firmware was already de facto open source, nobody needed theirs. Still I don't complain, I just think it's ironic.
I would actually respect it if their next products had an open source firmware (GPLv3), which would suggest that maybe they learned something.
Apple has understood this since forever.
Its practically the Apple II of synthesizers at this point - the device which upended a lot of monolithic, expensive, belligerents.
Citation needed.
AI agents don't have any goals of their own. They don't care about anything at all.
So when you ask an LLM to recommend a product, it's taking the things your prompt says are important to you, and combining them with professional reviews and user reviews it finds.
If you don't say that hackability is important to you, and professional reviews and user reviews aren't bringing that up much either, then the LLM is going to totally ignore hackability.
So this post seems to be more about the author's personal wish, rather than based on how actual LLM's work.
And all your conversations with it and all it knows about you. The current crop of proactive agents know or can infer so much about your life and home.
The thesis here is simple: you ask your agent to solve a problem for you, it will try to solve it with what I can do, and more and more those capabilities will include building or heavily customizing a device.
So do you really think it won’t choose that path if it’s available?
But 99% of the time, nobody is asking their agent to solve some larger problem. They're just asking which vacuum cleaner or humidifier or security cam is best.
"Building or heavily customizing a device" is not something that any agent is going to consider for 99+% of people.
So yes, if your agent has identified you as someone who has identified hackability as important, then it will include that. But that's the same as someone who just includes it in their prompt, like I said.
However, the idea that LLM's are broadly going to be preferring hackable devices as a general rule just doesn't make any sense, because the vast, vast majority of people don't consider that relevant.
The "agents" don't care which side they are used on and will be just as happy to make security robust for the devices whose creators don't want people changing them as they are attempting to hack into such devices.
No, it'll suggest which ever manufacturer has paid OpenAI or Anthropic for advertising
What is that hackable TV that your agent found?
> Geoff Stearns is a Product Manager at Google building AI agents and skills.
Ah, well, nevertheless.
> Geoff Stearns is a Product Manager at Google
Ironic.
The first recommendation was actually something like “onn 4k streamer” that Walmart sells but it was sold out.
"This is a (SOC device model) without any firmware, a (device model) USB speaker is attached to it internally. Write some firmware for it to get it working."
And you just have your AI write whatever firmware it wants.
Or would you want the company to put a little more effort into it before shipping it? (For example, actually testing it to see if it works at all.)