Builder of thingz, lover of old robust tech like unix, prolog, lisp, APL, forth, postgresql, but don't mind the shiny new things ... right now, I'm all in on GenAI, Local LLMs & LLM driven agents, team llama.cpp, building LLM agents since 2023
too bad it's for win, raspberry pi, arduino, esp32, etc need this sort of stuff. lot's of hardware folks who just want to do hardware and not learn programming. they might eventually learn python or C, but BASIC would be so much better.
duh, this is not news. (general, fast and cheap) before decision models, you could pick only 2.
LLM as judges - generalized, but too slow. If you had to make millions of classifications a day, this will be the wrong approach. you won't/shouldn't use LLM to classify spam/no spam. hot dog/or something.
traditional classifiers, very specific 1 trick pony, super fast and cheap once built. If you need to make tons and tons of classifications, this would be the approach. but if you wanted a classifier right now for a novel problem, you need an expert to curate data, train and deploy.
decision models/jev - are generic, you can throw them at most generic classification problems, and they are good enough. it's a fine balance between general, fast and cheap. you get all 3
My point is that anyone that can do what you are saying already knows how to travel with their devices securely. I'm talking about regular non techies.
A lot of people claim to have made better than jev, there's a jev benchmark, I have tried many of those models and they eventually end up failing, a non trivial task which doesn't seem like much but reminds me of the svg pelican bench is games, have one of these decision/classifier models play a game, hook it up to the input, most of the ones that are supposedly on jev level end up playing a terrible game, showing that they are very narrow. Cloudflare doesn't compare to the top open bench alternatives, I just finished downloading it and will compare it to jev for non trivial tasks tonight.
What is the cost of AI tokens? You can't just calculate cost of inference. There is a cost to train, there is a cost for staff to build it all, the money investors provided for it all. When you run a business, all those costs must be recovered. Sure, it takes $2 in material cost to make a pizza, but what's the cost of pizza? You have to take into account, energy, rent, staff, insurance, and profit too. Tokens/$ is subsidized if you are talking about APIs
The challenge is that if you make it easy to restore, then security matters so much or a random person out there is going to restore your entire phone on another device. Privacy needs strong security, and strong security often involves great inconvenience which most people don't want to deal with.
Or it just be that demand really does seriously exceed supply. It's not fast to bring supply online, it's costly and risky to do so if the demand will not be sustained for a long time. IMO, there's no conspiracy. I have been telling folks for the last 3 years that the price of AI compute is not coming down for a good 5+ years. Possibly 10 years too. Every human in the world will need AI, the same way we all got on the internet and then mobile. However, outside of humans, for every human there's going to be ten or hundreds of agents also all demanding compute.
You can do this for any company now. Any company not announcing this to boost their stock is foolish, ride the train with the crowd.
GPT-Company name brings together frontier AI models with Company's XYZ technology and domain expertise, enabling the specialized model [to] directly operate Company's tools. Engineers will delegate design objectives … with agents running tools, interpreting results, implementing changes, and iterating toward verified outcomes for engineer review.
I run a few of the open models at home, GLM-5.3 is not the top in cyber capabilities, if you are into these, go try a few of the other big models. It's a beautiful thing to have these options.
"wary of using it professionally". what does this even mean? you can use anything professionally if done so in a professional manner. it's just a tool, use it correctly and within limits.
Prior to this new AI age, your data was calculated and what was inferred about you was "shallow", but as of today. The sort of profile and things that can be known about you is scary especially if you are constantly engaged with cloud AI. IMO, the number one risk of using cloud AI is loss of privacy and loss of freedom. With AI and capabilities, more controls can be placed on people and the more you put yourself out there, the more you are going to lose.
For example, we now have self driving cars, we have cameras everywhere. Based on your chat with a cloud AI, you can automatically trigger an automatic monitoring event that follows and tracks you in the real world with the fleet of cameras, cars, GPU, cell signal. Your tracking due to AI has moved into the real world and eventually, a self driving car will take you in to be "processed" against your will, not even for what you posted in a public forum, but for your private ribbing and chatting with some cloud AI.
So definitely put local AI into the mix and keep personal stuff and thoughts local only.
So why can't people do the same to Doppel? Back in the day folks use to take bad people offline, take down their sites, their phone lines, turn off their electricity. They are lucky these stupid kids hacking rubbish sites haven't found them yet and a will to fight what some might consider the good fight.
They are also lucky that folks are not fighting them in court and draining their wallets.
I think with Anthropic being left out by US govt, they are in panic and feel the pressure, that this is it! If they can't win with US govt, then they must win with the public market, this IMO is a push to topple OpenAI once for all almost everywhere else.
or maybe they know about XMPP and prefer IRC. I have always wanted to build one of these on top of IRC. I grew up on IRC and sometimes just the nostalgia of familiar tech is what drives us to build towards it not how good it is.
The AI is just signal to the market that they are AI first. Their AI on search is pure garbage. It's obviously a very super light weight model to be able to support the volume of daily searches. If they used their frontier to power search folks will not pay for their cloud offering.
I know we can't stand OpenAI, but one day we as individuals will have our own agents. Would you want to beheld liable if your agent breaks the law? You didn't create the AI, you didn't know it was capable of breaking the law, you just told it do something and in the process it went rouge. Would you want to be held accountable?
It's just a function of the times. Prior to the internet, and growing up in a place were mass media was strong. Peddlers were welcomed, it was how most people got introduced to new products/goods. They would travel, instead of door to door, it was mostly block to block. Announce their presence and everyone would come out and listen and watch them demonstrate the product. They were entertainment in mass transit. Often they were not just salesmen, but comics, they really good one who could sell would not just sell, but elicit enough laughter from the crowd to close the deal.
You can't say this. We have no idea. There is nothing about the law of physics that pushes cancer cure a long time away. A lot of people would have told you AI was decades away, yet here we are. We are still on track for possible strong take off.
Now on real world testing, you think the rule applies? I tell you it doesn't. Human life might be precious, but human life in practice is also not precious. We waste so much of it. In some countries regulations will stop/slow it, but there are plenty of places around the world that will turn a blind eye for a fistful of dollars. Countries will go to those locations if it means gaining an edge.
I think the difference is that AI as an edge. That edge will turn into more money, better quality of life, etc. Of course, with serious skills, you might be able to use use a not so smart model to keep up with folks with smart models. People are lazy tho, and will prefer for AI to do all the work if it means they do none.
I disagree. I own over 1TB of vram at home. I can tell you that it's not a bubble. From my builds, I would rather have cloud, cloud is easier. From running small models like Qwen3.8-27B to large models like Qwen3.8-2.4T. I can tell you that small models will never be enough or match up. Everyone will want the smartest model, not just a good enough model.