The technology is needed to avoid short video / social media addiction but a working map app is a must.
3,052 karma · joined June 3, 2020
The technology is needed to avoid short video / social media addiction but a working map app is a must.
The effect on the team of developers with various level of experience started showing up as well:
The application architecture turn into a horrible mess, it's worse than junior engineers. The application started exhibiting tons of hard to debug issues, because the generated code was too low level and not covering corner cases.
Every attempt of the AI engineers to fix the issue generated one more class wrapping the existing codebase - with a fix which never worked (eg. ConnectionManagerWithTimeouts).
Eventually we basically had to rewrite the application, throwing away most of the code twice. One to just get something working with the existing architecture without crashing every hour and then another to use a framework and eliminate the last bugs occurring every once and then.
LLM needs to be in incredibly capable hands in order to be used safely and engineers will have to fight their instinct and not get swayed by the LLM telling them they're right.
They still need a baby sitter.
It's uncanny how good / fast it is
It enables dictation that actually works and it's as fast as you can think. I also have a set of scripts which just wait for voice commands and do things. I can pipe the results to an LLM, run commands, synthesize a voice with F5-TTS back and it's like having a local Jarvis.
The main limitation is being english only.
If you keep your standards high when hiring contractors you'll get the same level of quality you have with employees. Contractor agencies are also pretty happy to have long lasting clients (I have been with my current clients respectively for: 4 years, 3 years, 1 years and 1 month).
I work as a contractors with all my clients (who know of each others) and they all pay significantly more per hour compared to an employee. As an employee I could expect to make 1/4 of what I actually make.
The only exception in this arrangement was when I worked with an US company, they wanted to hire me as an employee and paid 1k per month to some company in my country just to hire me. An insane waste of money, not to mention taxes on my side.
Both approaches are inferior to yours because of the load on the server (one way or another).
Personally, I feel like we need to go one step further and just build locally, merge all layers, ship a tar of the entire (micro) distro + app and run it with lxc. Get rid of docker entirely.
The size of my images are tiny, the extra complexity is unwarranted.
Then of course I'm not a 1000 people company with 1GB docker images.
But it's not "greed": it's the end of zero interest rate policies.
For higher quality at a higher cost of VRAM, you can also use Flux.1 Fill to do inpainting.
Lastly, Flux.1 Kontext [dev] is going to be released soon and it promises to replace the entire flow (and with better prompt understanding). HN thread here: https://news.ycombinator.com/item?id=44128322
Your usecase can be useful for "local web applications" where you ship a binary and you get a local webui - but at that point, why not just build a Tauri app? (not electron as you would be shipping an extra browser instead of reusing the system one)
Candle is a great choice overall (and there are plenty of examples) but performance is slightly worse compared to tch.
Personally, if I can get it done with candle that's what I do. It's also pretty neat for serverless.
If I can't, I check if I can convert it to onnx without extra work (or if there is an onnx available).
As a last resort, I think about shipping torchlib via tch.
I generate plenty of solid.js without any issues.
I'm also sure in a couple of iterations coding tools will have better RAG (/ finetuning?) support on existing documentation and types.
I'd take a freelancer / mercenary who wants to get stuff done, invoice me and afford their own house - over someone from /r/anti-work
But even if they get something cooler working it would still take 20 years before we see them emerge out of testing (if ever).
I'm still waiting to get supervision with https://ocumetics.com/ and they're stuck doing and redoing tests
They do get acquired every once and then so I suspect they are being stalled with wads of cash so that they don't destroy the eyewear sector
In that scenario running code on the data with minimum evaluation of the data (eg. a schema with explanation) is a much better approach and it will scale up to use cases of a certain complexity.
Even this system is not perfect: once your data definition and orchestration grow to big you'll face the same problems.
This should allow you to scale to pretty complex problems though, while the naive approach of just embedding API responses in the chat fails soon (I run into this issue frequently, maintaining a relatively simple systems with a few tool calls).
The only proper solution is reproducing the level of granularity of human decisions in code and call this "decisional system" from an LLM (which would be then reduced to a mere language interface between human language and the internal system). Easier said than done, though.
Is the press responsible for the bad performance? I don't think so. (source: I worked in startups in EU, UK and US, I started two companies in EU and UK, one failed).
I think the underlying problem is the anti-work attitude of people. Most people don't want to work hard, they don't have any ambition. And this attitude is normal in Soviet Union 2.0, a society where you get beaten down by taxes if you try to climb up the ladder and regulations make sure your company gets stopped before it's too large. The US is objectively better in this regard, but it's getting worse every year. Luckily there is a lot more capital from the past keeping the startup scene afloat. Will it survive another 20-40 years or will the next Silicon Valley be in China?
Anyway the general lack of ambition in EU means that the composition in an EU startup is completely different between US and UE. While you can find real gems in US startups, in EU you find your typical average corporate employee - but less skilled or more junior.
People in EU startups are just people who couldn't grab a job anywhere else. They're not hoping to make it big because there is no capital to have life changing exits and the government will get most of the profits anyway.
Leaders and managers in US are also considerably better than in the EU. Many countries in EU never had a decent entrepreneurial class. Building those skills will take decades.
Just because the tech scene became this lefty hell circle, we should not consider controversial a thought that is so widespread in today's culture that it puts a president in the oval office twice.
Several upgrades wrecked havoc on my system, a lot of things didn't work as expected, I had to package a lot of stuff; eventually I just bit the bullet and started using homebrew
I ran nixos for a few years but I stopped when I started experimenting with AI. Dependencies were too out of date and running random small projects is riddled with extra pain, compared to other systems.
I also had problems during upgrades, caused by incompatible features and configurations. Instead of relying on the project documentation I had to rely on nix documentation. I have massive respect for nix maintainers (and I have even contributed) but maintaining any software for everyone is a monumental task. Distros maintainer have it hard enough.
Arch with AUR ended up being a simpler experience.
The worst is when I ask something complex, the model generates 300 lines of good code and then timeouts or crashes. If I ask to continue it will mess up the code for good, eg. starts generating duplicated code or functions which don't match the rest of the code.
Reddit often gets used by my users to make decisions (and it also has strict moderation, so you need to find a balance between being useful and spamming).
For some ideas facebook could be great.
I wish we had completely unregulated capitalism, so that middle class people would have more weapons to climb up society instead of paying the bill for everyone while drowning in debt and soulcrushing careers.
It's nothing to be proud of.
What you can be proud of is in REDUCING dictatorship, by removing power in centralised entities and giving it back to the individuals.
The government is the only one with a legal monopoly on violence; it redistributes resources in the society and it's not run by incorruptible angels but by fallible human beings - human beings who were put there thanks to investments of millions of dollars.
It's a recipe for disaster.
And you get decent quality models without the hassle of maintaining the infra.
I build something almost identical last week (closed source, not my IP) and I recommend: NeMo Parakeet (even faster than insanely_fast_whisper), F5-TTS (fast + very good quality voice cloning), Qwen3-4B for LLM (amazing quality).
Hono really needs something like axum's debug_handler