roadside_picnic

> resetting their trustworthiness score

But this is not how actual FICO scores work, and more interestingly, also not how historically Google has assigned domain authority for PageRank.

In both cases a history of transactions is required. Even if you could just spin up a new ID for a FICO score, having zero or little credit history typically means you'll have a very hard time securing any meaningful loans. Likewise, in the days when PageRank mattered a new domain would typically take a year to build up the authority required to get good results ranking well. Of course there were work around, but there are plenty of examples where easy to create a new identity does not equate starting with a fresh reputation (in fact even HN has the same principle! New accounts cannot, for example, downvote)

It's also harder to explain to your customers why you're the only product they use that has a security problem when an exploit does eventually happen.

You see this already with uptime. If Github or AWS has a major outage, it's in the news and even people that don't understand software realize it's just "something happening today, everything is down".

And, unfortunately, in practice customers don't care about this inverse, when "everyone else is down but not that one product".

> If everything is more bespoke the attack surface is smaller

This is exactly the problem that LLM assisted attacks solve. For a human attacker, the extra work to figure out a bespoke solution is too time consuming to be worth the effort. LLMs can search for vulnerabilities and spit out bespoke attacks like it's nothing. On top of this the bespoke solutions weakness leans heavily into the AI attackers advantage: response time. Even if they can just get your company's single Anthropic key, if they do it at 2am while your one infra person is on vacation, they have plenty of time to extract enough value for their effort.

I agree with the move to hyperscalers. The future (well, actually present) is going to be constantly seeing new attacks happening. The only real solution to this going to be having teams of people and agents constantly watching for these and able to deploy fixes and respond to risks immediately.

For me it's been liberating because I no longer have to worry about what languages I choose to spend my free time on at the risk of missing out on something more practical. For example, I love Common Lisp, but taking time relearn it always felt to come at the expense of something else that might also confer practical benefit. Now, in a sense, all programming languages are equally useless.

I recently did a major move and had to restructure my bookshelf pretty dramatically. The only programming books I kept out of storage where the Common Lisp and Prolog ones. Spending time learning a language that doesn't directly impact how you think about computation itself seems to be more of misuse of time (of course really, when programming truly for fun it shouldn't matter one way or the other).

> for very fat salaries

For a fairly sizable portion of the population (especially the tech worker population), salary/TC is seen as the measure of someone's value as a human being. I've had acquaintances who really, sincerely struggled to understand the concept than an engineer making 1/N what an engineer at Meta/Google/etc does could be overall a better engineer. The typical reply has been something along the lines of "but if they're better, why don't they just make more money?"

They really couldn't understand when I tried to explain that, in my experience, nearly all the really brilliant and interesting technical people I knew tended to be making less, because they would rather work on interesting problems with a high degree of control than play leveling games at a large corporation.

> I have created.

Are you working on a team? I've found that for solo projects this works but, even when you own the design of your piece, integrating with the work of several people doing this ultimately leads to even the part you initially designed no longer working exactly as you remember it.

A good chunk of my time now is spend working with an LLM to understand how other parts of the code base work and how they relate to and interact with the product features I manage. But I'm still on a small enough team that this is possible. I'm not surprised at all that an engineer at an org the size of Meta would not even be able to perform this level of work in understanding what's happening.

> Layer on top that some offices are owned by private equity!

I used to be very regular with dental visits, and have had an enormous amount of work done over the years. Then I went to my first PE owned dentist (by accident) and it was such a horrific experience I've basically avoided dentists since.

There used to be some reasonable balance between the dentist wanting to make money and wanting to build a secure custommer base/happy clients. PE dentists are just money extraction machines with little to no concern for how much pain is caused getting money out of your wallet.

I've never understood why "Hacker" News so frequently gets "But why though?" comments at the top.

The entire history of innovation is filled with people doing something just to see they can get it to work, even if badly, and then people continue to iterate on that until it works better, then works well, and then is so obvious people would never even question it. But it all starts with someone doing it to scratch an itch.

Neural networks, the foundation of our current AI revolution, used to fit well into the "neat, but practically useless" category.

Sure there are countless "but why though?" experiments that don't pan out, but that's just the cost of exploration. There can be no step-function innovation in a world where people only do things that make immediate practical sense.

> and I would not call it cheap

It is compared to the cost of making an authentic ramen broth at home at the scale of feeding 2-4 people. Pho has a similar property. I've made homemade pho broth once and it was very hard work, fairly costly, and it resulted in something that was pretty good, but not comparable in quality or price to something I could have bought in a restaurant down the street. (admittedly typically also cheaper that ramen)

> We do this and then no one reads it.

This to me is an anti-pattern that I see everywhere because we're still in the early stages of understanding how to adopt these things and create real agentic workflows.

Why should humans read all that documentation if they're ultimately not going to be the ones implementing it?

I see similar patterns like this everywhere, people asking questions about a PR that an engineer then copy pastes to an agents and the copy pastes back to the PR. It's obvious that humans are still in the loop in awkward places.

I've pointed this out before, but web development went through similar growing pains where we did all sorts of weird stuff because we hadn't quite figured out the patterns that work.

Right now nobody really knows what works and the tools we're using are still changing very rapidly. 6 months ago almost nobody was using agent harnesses and thinking about agents writing all of your code was preposterous. Today even the hardcore, AI-skeptic people on my team still have agents writing the vast majority of their code.

It's very clear to me that we're still stuffing agentic behavior into workflows designed for humans. It shouldn't be at all surprising that these patterns feel weird and sometimes broken. It has nothing to do with "winning over the average Joe" but rather these patterns evolving so they actually make sense.

MoEs don't route like most people imagine. They aren't learning topic based experts despite the name

The original Mixtral paper [0] (in the "Routing analysis" section) found:

"surprisingly, we do not observe obvious patterns in the assignment of experts based on the topic"

A quick skim of more recent analysis on MoE shows that this hasn't changed. MoE models do appear to work, but don't appear to do what the name implies, if anything they're routing based on the structure of the text and not the semantic content (and we're still not entirely sure what they're doing).

0. https://arxiv.org/pdf/2401.04088

> from 40 to 50 tok/s generation

That's actually much better than I would have thought!

Thanks for the answer and it does make this approach make more sense as a budget solution to running larger models locally.

What's the tokens/sec you're getting on that setup (genuinely curious because it's a setup I haven't actually run myself)?

> no other widely accessible path for young people to build a future

This is exactly right. Widespread gambling has a fundamental nihilism baked into it. If you believe hard work and frugality are the best path to success, then gambling doesn't feel like a good use of your income and energy.

But if you feel like your dreams are falling out of reach despite your hard work, if it starts to look like even big investors are really just playing a game (often with loaded dice) rather than acting on fundamentals, if grift seems like a more reliable path to success then hard work and ingenuity, then all of a sudden gambling seems a lot more sensible. Even if the odds aren't in your favor, you can at least get lucky when you gamble. When the game feels rigged its easy to believe that hardwork is destined for failure, whereas even loaded dice have some probability of landing in your favor.

If you're using a MoE model, then why do you care about the larger RAM offered by these devices? That's the main problem with low bandwidth devices: they limit the effective ram you can make use.

I do (and have historically done) quite a work with both local LLMs and local diffusion models. I have an M3 Max MBP at 400 GB/s and also a desktop with a RTX 4090 with 1,008 GB/s

While the M3 Max MBP can serve up MoE reasonably fast (~60 token/sec)the RTX 4090 is an entirely different experience (~170 token/sec). I also do a fair bit of experimentation and am currently running a custom decoder that requires expensive look-ahead, but I'm still able to get a usable 25 token/s on the RTX.

The raison d'etre for the DGX spark is not practical home inference, but rather offering the same fundamental architecture as data center cards for a affordable CUDA prototyping. If you want to build software to run on H100s, you probably can't justify buying (and running) a single card. The DGX spark solves this by having the same fundamental setup as what those cards have.

That makes these non-NVIDIA DGX-like devices confusing to me. The entire benefit of the DGX series is the NVIDIA architecture itself.

Anyone interested in home LLMs should decide whether a Mac or a dedicated GPU is the more sensible path based on their budget and other computer use. Each has their own benefits.

> Yet those 2-3M channels get the lions share of the views.

...because they're directly tied to YouTube own revenue.

As someone who creates a fair bit of content, and am fortunate enough not to have to worry about the revenue from that, I deliberately try to minimize monetization because it ultimately becomes the main driving force behind your content creation.

Monetization as the only model for content creation leads to an incredibly safe and boring world while simultaneously meaning every piece of content you interact with is trying to extract money from you the viewer.

In general if you're setting up a local LLM you should assume it's going to be primarily working as a server and talking to various clients. I use my MBP, but that's because I don't travel much anymore so it can happily work as a server at all times. With the right agent setup you can probably manage most things from your phone even if you don't have a seperate machine to use as a client.

I have an older laptop I run a hermes agent on backed by an API based open (non-local) model and Macbook Pro M4 for running another model locally (also using hermes). The agents have a Mattermost (open source version of slack) server they run and I run Mattermost on my phone so I can talk to them and task them with things. In fact, it was through the hermes WhatsApp endpoint that I got the first agent (non-local) to setup the Mattermost server and unboard the second agent (local mbp).

Then I can just chat with them through Mattermost when I need work done. Whenever I need something done I just hope on the Mattermost server and chat with them. I've had them build me multiple research reports (the fully local agent did awesome at this), learn how to use Stable Diffusion on my desktop to generate images, install and perform maintenance on various local services I run (including Open WebUI).

It depends on your use case. There's a lot of hype around machines like the DGX spark (I'm assuming this is the type of device you're referring to) because they look awesome, and are priced reasonably well. However all of these have notoriously low memory bandwidth despite the high ram.

These devices, especially the DGX line, are fantastic if you are interested in low-level CUDA programming. The DGX spark can be used to prototype CUDA code/libraries for GPUs that most of us couldn't think about affording. If you want to learn how to program for datacenter level GPUs then these are the best way to get that at home. Sure your code will run very slow compared to the real thing, but you can take that code and, theoretically, run it on the real thing. For anything else though, I feel there are better options.

If you're interested in pure inference I'm pretty partial to Apple devices. The M4 Max gets you 546 GB/s, the M5 MAX 614 GB/s, and the M3 ultra (you'd have to buy used at this point) 819 GB/s. Plus you have a very useful computer even if you realize you don't want a full time home inference server. Additionally these devices require very low power (if you're running high end consumer GPUs you do have to think about what your energy costs are per hour and how warm you like your room).

If you're interested inference and training, or already have a pretty beefy desktop PC, or simply demand the most token/s you can get, then GPUs are the way to go. The downside is they're still pretty memory restricted (but honestly the options for what you can run on any RTX N090 are pretty good). You'll get blazing inference and prefill speeds on these devices. The only down side is, if you are using them heavily, you will see it on your energy bill and feel it in your room.

The "should I wait" question is also potentially applicable. The world of consumer hardware is looking increasingly bleak (and expensive) but if Apple does release a new "Ultra" model we could be looking at inference speeds very close to GPUs (there's still limitations to these devices that makes training preferable on GPU)

My experience working in the open model space pretty deeply (both LLMs and diffusion models) for years now is that it is not quite as simple as that.

In the open model space an insane amount of effort goes into getting more powerful models to run with the same or less RAM. For example in the diffusion world many things that could not be run on easily under 24GB of VRAM actually run much better today with much less VRAM than they did a few years ago. You can do many things today with 8-16GB of VRAM that would not have been possible. At the same time the most advanced open models, like LTX 2.3 for video gen, still seem to respect 24GB of VRAM as the upper bound.

Similarly the standard "big" but localish open model for LLMs back in the day was Llama 3 70B, this was both a much worse and much larger model than Qwen 3.6 27B

So in two different spaces I've witnessed the "RAM required to run the best" decreasing or at least remaining stable, while the performance being achieved in both areas is astounding (LTX 2.3 is faster, better and more capable than the Wan 2.2 model that held popularity before it).

The biggest thing to watch out for is not just RAM/VRAM but memory bandwidth. You can try to "future proof" yourself with lots of RAM, but if it's 400 GB/S you're still constrained to smaller models.

In addition to models getting better, the quantization methods have also got much better. If you already have an RTX 3080 it's absolutely worth the time to just mess around and see how it does, experiment with different quants that fit in your VRAM. If you're purchasing I would recommend coughing up the extra cash for the 3090.

If you are experimenting it's worth mentioning that the harness/tooling is very important to getting a solid experience. Herme's agent is great for running helpful agents and OpenWeb UI can get really make the experience feel on par with paid chat interfaced.

A reasonable halfway step is to pay for an open model through the provider or open router. You'll get many of the benefits (especially around pricing) without needing to shell out on hardware before deciding if you like the way these models work.

M3-Max laptop: ~55 token/sec

RTX 4090: ~190 token/sec

I don't have the number around but there is a notable latency for pre-fill on the M3, but once it's running the delay is negligible.

The RTX, unsurprisingly, is all around superior performance wise, but: I use that computer for gaming and image gen work so I can't dedicate it as a server, and, especially when it's warmer, the heat generated under heavy loads is noticable.

There are a couple of things, but basically it boils down to the same reason people prefer Linux to Windows/MacOs: customization, control and privacy (arguably all of these are really subsets of 'control').

Having full control over how your data is retained, what the system prompt is, which version of the model you're running, etc leads to much a more consistent experience. For example, for chat sessions, I can't stand the new "let me push back" version of Claude. For my home models I never have to worry about that.

There's never a mystery as to whether the model secretly degraded performance, I always know exactly which model I'm using and how well it's utilizing resources etc. Open models also give you full visibility into the reasoning steps, so you never have to guess what the model is thinking.

Then when you start getting into things like uncensored/abliterated models we're talking about something you can't even pay for. In case you're unfamiliar, even open local models have guardrails built in. But people in the community have found ways to remove these. One of the things I've found most concerning about AI, which is under discussed, is the combination of people having personal chats with an agent that both monitors the conversation and refuses to discuss certain topics. This leads to a very deep level of self-censoring I find dystopian.

I also have multiple hermes agents setup, some with local backends other with open but non-local backends (e.g. Kimi through the API). For some tasks, I've just started to find the local agent tends to work better for the type of tasks I want (maybe it just over thinks less?). I don't use it for coding so much as research tasks and sysadmin stuff, but I've been really happy with the results.

Oh, and let's not forget, especially running on a Mac, these local models are basically free to run.

Just running it through `llama-cli` so that there's absolutely no persistent state related to the chat (and least I believe this to be the case).

See my comment to parent. I've been using local LLMs for practical, personal tasks for a few months now very successfuly.

You can run fantastic local models if you have either:

- M-series Apple device with ideally >= 24GB of VRAM

- RTX [345]090 GPU

I'm fortunate enough to have both and use an M-series laptop as basically a persistent server (I don't use it much and when traveling typically just use my work laptop). My desktop doesn't act as a persitent server but I fire up llama.cpp on it all time for quick chat sessions.

If you have one of the above devices and can dedicate it as server there are additional layers of tooling you can use that dramatically improve the experience. In particular Open WebUI allows you to add tons of useful tools (image gen, web search, code eval, etc), and agent harnesses like Hermes can make the current gen small models very capable. I have an agent in chat on my phone that basically handles all the sys-admin for the server it runs on.

I have a home server that runs Qwen3.6-35B-A3B through llama.cpp with Open WebUI for the user facing interface.

My teen isn't super interested in AI, but whenever they do feel curious they have their own account they can use on our home network. As far as chatting goes local models are more than capable for handling standard chat questions, doing research, helping troubleshoot problems etc. In fact it was an agent powered by the same model that setup the open webui server and took care of all the account management features through my phone (using Hermes agent).

If you're building AI powered features and using sophisticated agent setups for coding for work, then it make sense to use SoTA from these providers. But I've been using local models increasingly for personal use and am starting to find them preferable (I run an uncensored, ephemeral model for my own use and it's an entirely different experience than anything you can pay for).

Still haven't cancelled my personal Anthropic subscription, but considering it soon.

> It's like the infinite monkeys on typewrighters that will type whatever you are looking for, given infinite time.

In the monkey example the infinite time is doing a lot of work there. The fact that LLMs can search through semantic space and find reasonably correct paths in a reasonable time is directly tied to the reason why they are valuable.

Saying "these two things are similar except one can be useful and one can't" is not a great comparison.

For me the real lesson learned isn't how "smart" LLMs are, but rather how much human work is basically reducible to repeating past work with minor variation. Human's believe they are "reasoning" but so much code writen is just the human brain doing the same autocomplete style work that LLMs can do now.

I used to think comparisons of AI with the web where ridiculous, but increasingly it looks like they're not that dissimilar as far as how they change how we work. But as someone who graduated college during the bust there was a loooong gap between peak hype and things like online banking and e-commerce becoming standard.

Even if AI is an absolutely bubble, and SpaceX, Anthropic and OpenAI all cease to exist in a year... there's simply no way that AI has not fundamentally changed work. Even if I was forever pinned to the local models I'm running and the agent harnesses they use, I would never write code for work the same way.

But I lived through the rise of the web. I remember serving dynamic websites through cgi (which meant a new instance of an interpreter was spawned per user session). I vividly remember great JavaScript books saying things like "never use JavaScript for core functionality". I recall Java engineers saying Ruby on Rails was a toy and would never take off, that Python offered nothing over Perl and that "rich web applications" where never going to replace app native interfaces. I remember when the MVC pattern from the early Smalltalk days being dusted off and repurposed for web applications, completely changing how we designed software for the web.

And all of that is just software. It wasn't until the pandemic that ebooks replaced print books in share of academic library circulation (reversing a decades long trend of reduced circulation).

In my daily use of agents for coding and other forms of problem solving, while it is a wild accelerant, it's also clear we have not even started scratching the surface of how to think about building things with these tools.

I suspect we'll adapt to AI faster, but having lived through one major tech revolution, transforming work still takes some time. I'm not surprised we don't see an immediate jobs crisis.

Not to mention the completely separate topic that huge classes of employees were not and are not all that productive, so boosts in productivity don't imply lost jobs. That would require a boost in productivity combined with pressure to create concrete value with less, looking at the SpaceX IPO we're still a ways away from working about how efficiently we create concrete value.

It's more insidious than that. These IPOs aren't being rushed, they were waiting for all the pieces to be in place to force 401ks and other retirement plans to buy these IPOs.

The most recent change was the NASDAQ adopting the "fast change rule" which allows newly IPO'd companies to be listed in the index after only 15 days of trading. This rule was decided March 30, 2026 and only came into effect May 1, 2026.

The plan is to rapidly drive these prices up in the first 15 days, get the companies listed in the NASDAQ so funds are forced to purchase them at higher prices, then leave retirement accounts holding the bag.

As you likely know, rules have recently been changed that basically force many 401k funds to invest in these IPOs while simultaneously having a relatively small number of the initial IPO to be sold to the public forcing the funds to by at inflated prices.

The bubble won't pop until these retirement accounts of have been raided.

Have you personally used any of the latest batch of even smaller local models? They certainly don't beat SotA models at coding... but with a good harness they are able to achieve things with SotA that I couldn't last year.

I've repeatedly given local models non-trivial projects that involve research and coding which they've successfully completed with minimal intervention from me (almost exclusively in the domain of reviewing the results). Again, nothing comparable with current SotA, but definitely tasks I could not have given SotA models last year (without agent harness).

Now that pure progress from these models seems to have slowed down, we're seeing a ton of options for both making models more efficient and other tools that help improve them (everything from agent harnesses to RLVR).

That's just looking at "what can small do today", when you look at what's possible with larger open models that are still much smaller than SotA from the major providers, their performance is extremely close to SotA, enough that for personal projects I'll just use Kimi instead of any anthropic offerings.

So it's not terribly hard to image a solution in the middle happening within a few years. We still have tons to learn about optimal sizes of these models and how to build them with maximal efficiency (and we've already seen a lot of recent improvements in this space).