psadri

Whoever created this - you are the crazy kind of person I want to meet in real life.

There is a strange dynamic between data center build out and inference prices.

If the data centers do get built and come on line, we will see an increase in compute supply and that should lower inference prices.

If they don't, for whatever reason including financial ones, then we will not have enough compute to meet demand and inference prices will rise. But this would mean the financial concerns were misplaced.

Where is the equilibrium point?

Hmm. I use pico. They could use an agent to modernize it!

I was dreading this moment. RIP.

I’ll be re-watching The Inner Light in his honor tonight.

I had exactly the same thought around Apple’s “report spam”. I have marked hundreds of text messages as spam. There is enough training data in there for even a dumb classifier to filter some of this stuff out. But next day, there are half a dozen more, nearly identical spam msgs sitting unread in the inbox.

I have also found that replying STOP = sign of life, your number is marked as “someone alive on the other end” and gets resold to 20 new lists.

Perhaps the real answer is people’s buying power. If people can’t withhold their work, they could withhold their spending, ie boycott misbehaving entities.

I think that’s a different argument. Apple absolutely “copies” hugely successful, durable apps. Just read about “getting sherlocked”. The rest of the apps are transient, niche and it’s easier to just harvest your cut of ads/payments.

Luna is incredible for the price but I find 6 to be worse than 5.6.

This is a touching sentiment. Thanks for sharing it.

The problem with CSS is the cascade and how it is not scoped. It's like global variables in code. And that can be traced back to it's roots 20+ years ago as a way to style "documents" vs applications that are naturally composed of self-contained UI components.

They are better at comparisons between two items vs scoring it in isolation. You can then turn a set of pairwise comparisons into numeric scores.

Even though I started using LLMs as soon as they became available, I was until recently very cautious around using them autonomously. I was of the opinion that I needed to provide heavy supervision around coding tasks. Granted, that was probably justified based on the model capabilities at the time.

However, lately, I have had several experiences that have completely changed my mind - the here is a high level goal (involving gathering production logs, setting up an eval, running tests, iterating, including diagnosis and coming up with new ideas, etc. and don’t bother me until it’s done. The experience has been incredible.

This has been possible because the agent has a way to iterate and hillclimb against a metric it can measure. Relatively easy in the world of software.

I’m now convinced that we will unlock the same gains once make other domains similarly “iterable”.

Eg Agent comes up with 100 new proteins, runs in lab autonomously, gathers results, iterates. One month (or whatever) later, you have your new protein designed.

Write tests first. Have agent iterate until they are satisfied.

The point is that it boils down to writing the tests correctly, regardless of who is implementing the actual code. Hand-written code without test coverage has the same problems as AI generated code.

Now that the models are smarter / know more than most of us, a new bottleneck is finding out what you don't know. If you don't know about logistic classifiers and how they could be applied to your problem, you will not ask for it.

These days, I tend to start my coding sessions by the high level problem I'm trying to solve vs the prescriptive, specific solution I may have in mind. It often surfaces ideas and approaches that I did not know about.

The point was that you can go further than "write me a contact that says x,y and z". I actually believe the process I went through with the AI was a more custom, tailored experience than any interaction I have had with actual lawyers, who'd usually apply a template. The AI asked me a bunch of questions, many of which were relevant, and led to follow up questions. I could ask my own questions, make corrections, etc. and all along, they made sense to me. This part of the experience was great.

There could still be more unknowns that I did not encounter and by definition don't know about.

And the final output could be wrong, wording, exact legal terminology etc.

Can we say working with an actual lawyer is guaranteed to be better, more correct? Lawyers are people and they also make mistakes, wake up on the wrong side of the bed, hate their job etc, just like the rest of us.

No admittedly not. I can try it and report back.

I was in a similar situation and tried a different approach.

I had start with asking for a contact with some details I provided. But then I ask the model to be an experienced corporate lawyer and ask me a series of questions to gather the details it needed and then write the contract. The result covered a lot of details that were highly relevant but were absent from the original attempt.

The key insight is that you can lean on the model to cover your unknown unknowns.

We have been working in this space for the past year. Based on our experience, I no longer trust any claims unless they are backed by benchmark results (yes, benchmarks are painful to run reliably and expensive).

It is possible to reduce token usage. It’s just much harder than the basic approach.

Thanks to coding agents, there is no reason not to

For anyone interested in private space flight: Wild Wild Space documentary. Also a commentary about startups in general.

Separately, I've been wondering how uneven human progress is. I think we have come a long way in electronics (relays to vacuum tubes to transistor to … to an M5 chip) but in some other domains, like moving objects through space, we are basically mostly reliant on thrust generated by burning fuel (EV's are a genuine advance, but mostly due to battery tech).

I have been wondering: more solar means we are making Earth less reflective. Solar energy that might have been reflected back to space is now absorbed by the PV to make electricity. But that delta has to go somewhere? Where does it go in terms of Earth’s temperature. Same with more nuclear… we’d be pumping more energy into the system.

Don’t take this as an argument against these technologies. It’s a huge win to not be increasing the CO2 concentration in the atmosphere by burning fossil fuels.

Looking forward to the series finale this Friday!

Luna is at a very compelling point on the price/performance curve.

I have found that sometimes a smaller model with max reasoning is actually more expensive than using the next tier model with a lower reasoning effort. It’s certainly faster.

In my experience, the thing that makes your better at something is actually _doing_ it, badly at first and then gradually improving by iteration.

But for writing, I usually see the advice that says you should read a lot instead of write a lot. What gives?

I looked at the OTel schema generated for recording a single numerical metric. It was like 12 or 13 meta fields in addition to the actual metric fields like timestamp, metric and value.

OTel looks like something designed by a committee of committees, funded by someone who is in the business of selling cloud storage/data warehousing services.

Who are OpenRouter’s competitors?

This is an awesome thread. I agree - the catalyst is joy, in a broad sense of the word. One is lucky to find joy in something that can take you deep.

To be honest I also just read about it. However everytime I pass through Heathrow I think someone really took the idea to heart.

Aside for the algorithms and how well they work, there is a psychological aspect to how people perceive "wait" times. It turns out just waiting is very annoying. But having people do anything during that same wait period → perceived as progress → not upset.

An example iirc was waiting for luggage at the airport. It takes min of x minutes for the first bags to show. In one version, passengers went directly from the gate to the luggage carousel and waited x mins for the first bag → unhappy. Then the airport decided to add a long, round-about "walk" path from the gate to the luggage area that burned some of those minutes → same total "wait" but happier passengers.

A version of this idea was mentioned in one of the Three Body Problem books. There, the bombs were pre-positioned along a path and detonated sequentially like dominos, with a vehicle riding the blast waves.