weitendorf

You’re absolutely right

note that this also applies to ironic LLM-assisted shitposts

I think there should maybe be a little more leeway/amnesty for accounts that aren’t new or don’t have a high rate of LLM tells. IMO if you could self-declare the AI usage and wrap them in <details> tags to handle the verbosity and attribution you could integrate it reaosnably

it's precisely because you can tune them to do that off a strong base

and because they start off generalized, you can train them against multiple of your workloads (they'll start with some generalized performance/world knowledge about them)

Which allows you to embed proprietary processes/data (world knowledge only you may have) across multiple domains into a model cheaply

General classifiers are actually incredible for two things:

1. inference on arbitrary context / world knowledge with fast, structured output

2. I almost don't want to say it because I don't know if people realize this yet... but that makes traces from any kind of non-AI process or system "learnable", you can easily model it by by just asking it to predict the next outcome during learning and compare against what happened.

which also means it could also learn new strategies through rollouts in a way that generalizes across semantic space which means...

so these models are actually quite promisingly useful for existing companies because they do have general traces of what happened in their company and want to discover ways they can tweak their business processes, and then there is durable demand for models you can embed within a product or UX flow without exposing your user to slop + then train or customize and build real IP or custom capabilities

general classification itself is quite interesting IMO for applications that need to make judgment calls or select one of many UX actions, and a much better fit for this than LLMs. So if you think about it, this is the type of thing businesses might be calling a lot - every trace (eg user session, package shipment, sales motion, marketing campaign, job outreach) + proactively throughout dynamic experiences to decide whether to eg end a call or which of multiple UIs to manipulate

this is just an unfamiliar product experience to most people because it hasn't really been bootstrappable with a strong general baseline until now. But now it's posible and I think it will be very col.

I think there are certain parts of certain products where the implementation details are really important, or where you need to make sure the behavior is fully reviewed/understood, or some invariant maintained. In some places that's the entire job and in other places almost none of it.

If you're running tons of product/UX experiments code review is just going to waste everybody's time. But the volume of changes you can make now is so large, someone could touch something important about at any given time without proper oversight.

IMO an effective team has to be able to either trust each other enough/be accountable enough that code review is mostly unnecessary, or everybody just works really independently on mostly separable things, or you regularly pair/mob-program.

If you can trust your colleagues not to pump out code without messing it all up or turning everything into slop, or creating large amounts of work for other people, things are great. Realistically most dev teams have a wide range of skills and personalities and people also make mistakes, and code review helps us build shared knowledge and collaborate.

Maybe there is too much of a moral hazard from the reviewer's role as a cosigner, or some people are only as conscientious as necessary to get things done (not a problem writing every line by hand)... but people can ruin code review for everyone else by demanding an asymmetric level of accountability/judgment and oversight.

The bigger issue for me is whether or not you still do code review, you need to build shared knowledge within teams. I really think we should be reviewing our colleagues' coding agent session logs

Not even to critique them (but that could be useful), but just to understand what it is they are actually trying to do/what mental model they're operating under, and if there are issue in the PR, what miscommunication or technique or process could help fix it? I think a lot of people feel that asking to see those logs is invasive or embarrassing, but the problem is that it's just really hard to build a model for how someone works or thinks without it. You don't need that in a deontological sense, but you do if you want to fix coordination problems.

They're challenging someone to "test it" by burning down a backlog, because the other person thinks they will just do more work, with the implication that this person believes they'll just be laid off - and so they are better off not burning down the backlog of work.

I take an issue with that mindset because I know it is not true. And I'm not sure why you think I am attacking this person - if someone were to actually sandbag their own career, that is probably one of the most damaging mistakes they could make. Maybe I misunderstand something, if so I apologize for that

I mean, if you're in a contract job, you're always to get laid off at the end of the job - that is the job. Very few of the software jobs in my world are like that

If someone is deliberately stretching out a job and making it an adversarial business relationship, when the business arrangment is that they complete the project then do another job somewhere else, then I would not expect that to be a very successful strategy, and also not really want to help them in any way beyond existing contractual obligations. You are basically guaranteeing a bad working relationship with every employer you ever have.

And it doesn't make sense because this is not a strategy that's rational for someone who enjoys their work or is good at their job anyway - if that describes you, then your existing employer is probably one of if not the best advocate you have, and your best strategy would be to either switch careers or progress to a point you don't feel precarious

That's why I say you are really hurting yourself more than you know. You are guaranteeing an at-best "every boss dislikes me and my career goal is not getting fired" outcome and (idk about your actual employer) damaging relationships you really probably need (if this is your mindset)

I do wish you well and think it's good for us to work on these tools if only to help ourselves better understand how to work with these things, and have fun. I've actually had a blast working on agent tooling with agents too.

What I fear is that as computer people we can easily convince ourselves to avoid doing things in-person or which involve leaving the comfort of our inner/online lives, or having to bridge technology with some other thing that is very much not technology (like non technical people, or sales and marketing), or is but is not easy to work with (like a robot). But there are actually a ton of problems that need solving there, and they are uncomfortable because it might involve looking stupid or showing up somewhere nobody knows you, or unknown-unknowns.

Personally I am embarrassed at myself not for trying to solve my own problems (I want to spend all day doing tech stuff and make money selling it to other people doing tech stuff) but for how long it took me to realize that I was either anxious or thought I was above doing something more concrete and mundane.

For example, this is a popular music venue in the city I live in https://www.thegreatnorthernsf.com/ - Claude and Codex have been out for months; it would take like 20 minutes to give their site a makeover, mostly just watching AI work.

This is how local government events are coordinate in the most technologically forward city in the world. Double irony that the product is provided by one of the biggest tech companies in the city, too. But it's literally just a web form and some poor government worker is apologizing in the event details that the signups are broken: https://sfrpd.my.salesforce-sites.com/SFRPDJob?sShiftId=a09P...

I think agents are very fun and cool but I'm increasingly convinced that my job is to use them to help this type of user, not developers.

claude code and codex are literally IDEs. as are any other remote agent that you use like them

you're not "in the terminal", in fact the product is selling you on the concept of being "in the terminal" as you work, but you're literally just asking a computer with ~full userspace access and direct ability to compile and run programs to do stuff for you, in an environment with "plugins" and like 30 "slash commands" and menus. it has auto-review and manages git for you and even writes your PR descriptions and other messages to your colleagues if you want. that is an integrated development environment.

I think if you can just run a one line bash command from a big company's website, and then just ask it to set up git and everything else for you (which it will default to anyway), and it tries to sell you on third party plugs in and Skills and managed proxies like MCP, and workflows, you are just using the new kind of IDE. And there is nothing bad about that because it makes us more productive to not have to mess with things like that. The idea that you are not using an IDE, or that we are "better" because the IDE looks like the development environment of the type of developer we'd admired in the post (but literally has the value proposition of being the easy way to accomplish that), is dangerous.

I'm sorry and will be the first to admit I went for this too, but "using agents to write more tools for agents." is literally the problem.

The purpose of software is to help people with their problems. I think it's easy to tell ourselves that other people want to use our tools which we really made to help ourselves (especially if them using it also helps ourselves even more!). And also, this is all so new and there are real opportunities that it is good to learn and try new things and be open to actually making something valuable.

But! Other people extensively using agents are literally the last people who would pay for a project like this! And unless it's a fulltime job like it is for a 5 or so well resourced tech companies, it's hard to make something as good and useful as them. I say this as someone who spent a long time on this problem as my fulltime job.

We should still have fun with our work. But literally, "software for other software people, that solves the problem of us needing to write more software for writing software, faster", who is that helping? There are no constraints or end users besides ourselves, and half of that is not so much it benefiting us through productivity, but learning new things and play - again, important. But there's a reason so many people are doing this and I think it's because they don't have to go outside and solve someone else's problem, or do something unusual, or involving unfun non-software aspects - agents are fun! But how many random people's agent have you installed?

IMO: if the technology does not help a non-technology person live a better life, and especially if it involves "using an agent to build a framework for agents" we should do some more, deeper introspection about what problem we are actually solving and whether or not we're being ambitious. And I say that not to pick on you but myself.

it is and you do matter

LLMs have introduced me to things like photodiodes and specific economists I would likely not have found on my own

I do however wonder about specific research priority/project from labs' solving their own problems leaking into my chats. I noticed one month that Claude really started harping on about "legibility" and suggesting Seeing Like A State to me, and saw it like any other interesting suggestion/insight.

Now I see a lot of people mentioning "legibility" and think that was just some weird attractor state or useful vocabulary term for being an LLM (that literally can't see certain things or be observed easily!). It's a weird feeling, because nobody tried to trick me and in fact it objectively helped me to acquire useful information. I just have to wonder if other people are having the same conversation and there's really nothing special about it - maybe Anthropic just really liked that book right around when they were doing a lot of RL

Not only that, was the intent just to give Claude the vocabulary to be an LLM in a company that wants it to be legible, and apply that book to managing it, and we're either intentionally or unintentionally memetically downstream of that? Or was it just extra likely to be mentioned because it's a genuinely helpful book - either because the AI learned over time that it was (very good!), someone just wanted to recommend it to more people to help them (~good). I worry that the "intelligent conversation" or "wow look at the agent go" can give us very poor epistemics

We're going through a very unique, fast-paced transition and the problem is mostly cultural/psychological IMO

People react before they've had the time to really process it, and show a lot about their existing thought processes

Many technical people IMO fetishize technology and processes, disagreeable-but-right authority justified by their willingness to work on things that most people find hard or just don't want to do. Computers can be a vessel for identity and self-worth - it's alluring to think of technology (and ourselves) as intrinsically valuable. A surprisingly large contingent seems to see work as necessarily zero-sum and adversarial, and make it a core part of their identity (and often not even particularly abnormally powerful or better for it) - like what even is the archetype there?

People like your wife are the ones writing crazy excel spreadsheets and VB6 macros, and making decisions about enterprise IT products, and solving problems with computers! They're actually probably more knowledgeable about "what is a useful and valuable way to use computers" than the ones who find computers fun and freeing and Just Want to Code.

Algorithms will dole out as much outrage as you can take but at the end of the day, doing stuff with computers that people want to pay for - usually, automating or securing things - is why people pay us!

One framing is people overindex on Being a certain kind of person or (the archetype they aspire to or identify with, that is valuable in a passive sense), over Becoming or Doing what's necessary for their work to be valuable (actively helping people, and stuff that's not fun). Very normal to feel grief and not want life to feel so uncertain, but it was literally always uncertain! Life isn't over because the business people at your IT job want to build software!

This is a particularly Western democratic trend. Japan and Singapore both have their own models for democracy. Each builds differently, in their own way, and gets it done.

Most people just have no practice or experience writing. Asking them to respond to or write a large response to an email is a total non-starter for them. That's the default culture in the corporate world unless you have a strong company culture otherwise.

You know how popular grammarly and 2023-24 era chatgpt were? The prospect of writing an email actually makes most people quite anxious. They are not in the comments of hacker news, as a rule - they're watching YouTube and TV and don't read blogs. Just the way most people prefer to live.

When my parents were my age, there were almost no jobs working with distributed systems or robotics and the relative standard of living was much worse than what I have been able to attain at a similar age and level of education.

My grandparents performed jobs that now are rarely performed in the US due to there being better jobs for those qualified to perform them.

I am much happier working with AI and distributed systems than as a carpenter or steel mill foreman like my grandfathers. Everybody on my father's side of the family had these white marks all over their forearms from molten metal splashing on them and burning them as they worked.

This is only possible because of technological progress. It creates new jobs.

This is called the lump of labor fallacy. When jobs get automated, almost always, people find other jobs. https://en.wikipedia.org/wiki/Lump_of_labour_fallacy

This is why our standard of living is much better now than it was 150 years ago. Even though we have autoamted pretty much all of the jobs since then, except teacher and lawyer (there is another term in economics for what happens here: https://en.wikipedia.org/wiki/Baumol_effect)

Personally I think it's fine, even as my own profession undergoes the same transition. We now have new jobs in running the waymo fleet in various capacities - the cars still need to be serviced, cleaned, supported, scheduled, etc - and the highly compensated jobs designing what are essentially real-world robots that can safely coexist with humans on the road. This is how technology has always worked and it's always made us better off on the whole.

Also, the new product is just better. When a human is not in the car, it's ok for to leave them waiting for a few minutes to pick me up if it's not a busy street, and I don't have to deal with the risk of an unsafe or unsavory person hosting me in their personal vehicle. I can control the music and the AC, and tap a button to have the car pull over. I can and do still take Lyfts and Ubers when I need to - it's cheaper for anything involving the highway, still, and better when I need to get somewhere in a hurry.

I’m so glad to see someone excited about them, because this is how I felt when they first started to go live in San Francisco. Now I take them multiple times a week, and have been doing so for multiple years.

They really do feel like magic, and give you palpable proof that you’re living in the future.

Hey, I mean this in as kind a way as possible, but I actually don’t think the government should pay for the fence line!

If you wouldn’t even pay for it yourself, as pretty much the sole beneficiary and someone who could afford it but would rather the money go elsewhere, why would it make sense for other taxpayers to pay for it?

Isn’t the premise of “you may as well own that land” the entire appeal of living next to public land anyway?

It’s really domain dependent I think. If you are doing anything conversational interfacing with less AI-familiar users, latency matters a lot.

If you’re feeding the results into a very smart LLM, it will figure out what you meant (but crucially ONLY if you warn it or tell it to do so, in some cases!). If you’re writing code directly or creating something for public consumption, you can’t tolerate mistakes. If you’re taking notes for yourself you just want it to work cheaply.

If you are ok with the complexity you can run both, and a Meta/Google open model with native audio, and let a smart LLM doctor it up. If performance really matters you can pay for a proprietary model or train one yourself. Until you get to that point, I think it probably doesn't matter much either way. Voice is just too easy to fiddle with

How many dimensions were you using and if you tried multiple, did performance hold up?

Per https://ai.google.dev/gemma/docs/embeddinggemma/model_card_2... the audio data mentions speech, environment sounds, and "acoustic events" and for cross-modality "paired text, image, video, and audio data" so I imagine its audio gets encoded mostly to transcribed language and labels like "bells" without rich structural content.

You might want to actually try giant models with native vision (audio optional) input on spectrograms for audio. Depends on what you need embeddings for and if you're able/willing to extract embedding models out of the LLM (some are more or less amenable) but it's actually insane how good can be at "understanding" audio content's spectral profile.

Try giving opus 5.5 audio spectrograms and ask it to tell you what's inside or even to change the notes/words :)

IMO embedding models are some of the most likely models you’d want to self host because they are much more useful out of the box for batch or stream workloads than LLMs. You can just rent a GPU for $1-2/hr (you don’t need a B300) and run across your entire dataset.

If you are only interested in recommendations, clustering, etc and won’t have to do embedding at runtime, that’s it.

There is a lot of basic matrix stuff you can do with embedding vectors that are much easier than their LLM equivalents. I can’t remember the name for the techniques but a major one is translating questions annout a subject eg “Who is Slim Shady?”->”Marshall Mathers”so that your input distribution at runtime matches your offline distribution. Much easier to do with open weight models.

I insist on certain ways to use CSS (this, I realized, is where most of the untapped "alpha" is. There are a lot of patterns, like the css checkbox trick and using selectors/tags to pull data from html element attrs, that can introduce a lot of interactivity and generality without JS) and specific design+page contents. Then I typically look at the output visually and make sure the structure doesn't do anything stupid. To fix things I often will just start editing the html and css manually until the page looks exactly like I want it, then save the temp state and pass it back into the model to make the code changes.

Basically I invested a lot of time in learning how CSS, browser apis, and modern html work and figuring out how to do tricks with them that eliminate or greatly reduce the need for SSR and JS. Even though this took several months and a LOT of specs/docs and tedious design work, it was worth it because now I actually understand the foundational abstractions that all these web UIs actually are implemented against, and which don't change. Now I only need LLMs and a way to communicate those tricks (and I know the right language and terms to use to get them build on the right foundations).

But I'm working towards training my own preference set and training pipeline/tools so that I can get models to output things the way I like without all the process/styling/etc. steering that I still need between and across sessions. This is why I think coding agents are long-term only going to be used for vibe coding or to replace less skilled SWEs. You can't get them to consistently do things the way you want or build new devtools/abstractions that get fully internalized and baked into the weights unless you trian them. And a very useful thing about vanilla css/html is that they are much more easily verifiable and something you can generated/search synthetically vs SSR-npm style output.

We need to build collaborative post-training pipelines to augment models with new custom capabilities without catastrophic forgetting/excessive loss of capability in other domains.

Currently what makes this too difficult for anybody but frontier labs is the lack of access to the full distribution of workloads/data they use to RL multiple separate envs/evals without regressing more than they advance in general capabilities. Because they are the primary buyers/builders of that stuff, they have no incentive or reason to allow anybody to replay or resample it but themselves. And it's all so very expensive to do in aggregate so there's not really demand for other product shapes except more openly available, collaborative/bundled RL that you could plug your specific workloads into (who the frontier labs would obviously not want to support with their business).

However, if someone were to build a collaborative rollout platform that you could use to train private workloads (ie create useful IP that doesn't just become profit for other labs/come from what they already can do), the upfront investment would be amortized over the very large number of potential buyers once it gets into the 5-8 figure range, who essentially have no other choice if they want to remain competitive in the technology industry.

Until openai/anthropic ipo the concentration of capital/spending and ndas/loss of employability is too concentrated for the best researchers to really do this without rocking the boat. And a lot of also-ran ai/saas have the same risk due to the lack of capitalized acquisition opportunities or AI vendors to partner with.

If the plan is ultimately to drive you out of business if you ever build anything profitable with their products, and monetize your knowledge without fairly rewarding or explaining their intent to do so, you might as well defect early and build what you inevitably would need anyway.

you could try to help build that!

The killer app behind Azure and GCP in a lot of cases, the thing that really drove multi-billion dollar decisions made by the largest cloud customers in the world, was that they were not Amazon.

Why would Walmart want to let Amazon hold their business hostage, tell them all their secrets - literally pay to hand it over to them - and generate profits that would surely be re-invested in competing against them in retail? Even if AWS handled Walmart's account in 100% good faith and helped Walmart tremendously, they would still ultimately be paying to make themselves less competitive in a world where their single competitive edge (low-margin retail) would be eroded.

Would the consumer - or really anybody that likes having cheap, widely available products they don't personally make for themselves - ultimately benefit in a world where Walmart's only choice is AWS (startup-branding) and AWS(ea-branding) and AWS(IBM branding) and AWS(Microsoft-branding, if they were also evil chinese communists who maybe trained on the traces they paid for)?

better hope you're not walmart

It’s really only an existential problem for pure-software businesses/developers.

There’s a good chance that the next step after open weight models is something that makes it easier to post-train existing models on new workloads without running an entire pipeline just for yourself. Somebody has to actually build it though.

Look at squarespace + wix + lovable + framer + vercel + wordpress (and agencies), cloudflare, godaddy, etc. and think about how much more demand there is for a vertically integrated one stop shop without friction. This is a massive market even if it doesn’t become myspace.

This is the top of the funnel for most people building a tech product or marketing their business, so it’s very helpful if you intend to sell other services they might want to use down the line. (Cf Cloudflare except imagine if they also integrated ads into one of the most popular websites in the world)

There’s going to be a lot of chaos at some point but I think that “hybrid builders” (scientists, sales people, product developers, etc). true specialists, and new opportunities in hw/robotics/training/ will soak up a lot of talent, and be rewarding to work in. Feels very sadly k-shaped.

Software engineering on one end is going to be more like finance or medicine where the cost of poor performance is actually higher than the risk that they wont be worth what you pay them (ie there isn’t any amount of pay where it’s worth it to hire them). Then on the other it’ll be like knowing how to use PowerPoint. All the new tech categories have too many bottlenecks and challenges beyond just producing code to consolidate like that.

They’ve both been working on this for a while, I’m surprised they still haven’t launched.

Domains, auth, and databases are probably the stickiest software products of all time. I suspect it’s a combo of security concerns and the risk of eroding goodwill among developers (their most important customer base rn) if they move up the stack too quickly. Once you know that the goal is to become cloud vendors, train on/compete with their own customers, and own the entire software stack e2e it’s hard to really feel grateful that they’re dragging it out but planning on doing it anyway.

For those of us working in infra/SaaS outside these companies it’s pretty clear that the only viable path that doesn’t involve getting cannibalized is training your own agents/models. The new coding agent-infra-data business model is a path towards full commoditization and undifferentiated prompting in 3-4 vertically integrated walled gardens. If you ever start making real money on pure software infra they’ll just be able to eat you alive by implementing something similar, training on their own tools rather than yours, and integrating it into their stack (which all of your customers are on already).

Also, you should only ever use $20-200/mo subscriptions on work you want them to train against or you don’t mind automate yourself out of. Think a little bit about what you’re teaching them to do when you use their products, especially if they’re the primary interface you’re working in. People are going to start caring about this a lot when the AI companies feel safe enough to begin the “extinguish” phase of AI coding. Train your own models!

The product shape we built at my company is similar to yours and we chose to market capability/value-add rather than "cloud". Didn't really work. I'm bearish on the "sandbox" myself, I just instantly think "either a more expensive vm, or a thing in a less expensive datacenter" (also, my personal experience suggests that people don't want to sandbox things, but there are things they want that need to have sandboxes).

I don't even think Cloudflare has a name for theirs, I remember hearing it called their developer platform but after checking just now, they seem to mostly just call it "products".

Devtools marketing to consumers feels pretty impossible to me. I think you either need to be absolutely massive and have existing credibility + tons of resources to throw at every possible marketing channel, or figure out how to get people to truly love your product/brand. You're basically selling trust/professional identities in most cases rather than compute or APIs.

If I were to do it again I would either pick a different vertical or put 90% of my effort into community-building, events, PR/tech-influencer stuff, and flashy demos. You only need to fall back to widely known framings like "cloud" when nobody knows about you.