psadri

What are some examples that you can’t express in json schema?

I wonder if these stay in the prefix cache?

Awesome experiment!!

I did a version of this where the AI writes tools on the fly but gets to reuse them on future calls, trying to address the cost / performance issues. Migrations are challenging because they require some notion of an atomic update across the db and the tools.

This is a nice model of organically building software on the fly and even letting end users customize it on the fly.

The trouble is not populating it. The trouble is that tables, even though structured semantically, give you absolutely no functionality. There are no search, filter, sort, or selection features that you get.

A resource that would benefit the entire community is a set of ballpark figures for what kind of performance is "normal" given a particular hardware + data volume. I know this is a hard problem because there is so much variation across workloads, but I think even order of magnitude ballparks would be useful. For example, it could say things like:

task: msg queue

software: kafka

hardware: m7i.xlarge (vCPUs: 4 Memory: 16 GiB)

payload: 2kb / msg

possible performance: ### - #### msgs / second

etc…

So many times I've found myself wondering: is this thing behaving within an order of magnitude of a correctly setup version so that I can decide whether I should leave it alone or spend more time on it.

The pixel to sounds would pass through “reading” so there might be information loss. It is no longer just pixels.

Lots of caveats here in the following statement: if your application is not fully leaning in to frontier model capabilities, you are probably building a previous generation product.

When I think about a problem, I consciously explore a tree (or graph) of possibility chains. This requires a mental space to keep track of “state”. Sometimes jotting things down on paper helps if I can’t keep it all in my head. The process is: - generate some possibilities - rank them based on intuition (this might happen subconsciously!) - ask what if we follow possibility Pn - push Pn on to the stack. - recourse or pop stack if deadened

I feel LLMs are fairly capable when it comes to doing each of those steps in isolation. But not when it is all put together as a process.

That’s funny. Yeah. I wrote this on the fly. It can use multiple passes to add layers of self reference / depth.

I feel like this could be adopted for your homegrown "whatever" framework (eg: UI framework, Auth framework, …)

Congratulations on getting hired to this team! You probably count yourself lucky, but don't. We had been trying to fill this role for the past 5 months and every candidate would run away as soon as we showed them our homegrown auth framework. But don't run yet please, do give it a try.

So, you are still here? It must be a bad job market out there. Looks like you found the documentation for the project. Let me save you the trouble, it has not be updated since 3 years ago (about the time John quit). No worries, there are lots of usage examples in the Perforce repo. Perforce is like Git but that's for another day.

So you managed to checkout the code. Before you type "make", let me remind you to install this particular version of Python and set up your LD paths. Make sure you don't have anything else relying on Python because they will probably never work again.

If you hit the dreaded "std::vector<std::__cxx11::basic_string<char> > >'} is not derived from 'const char*'" error, ask Joe (if he is still around) to show you which header file you need to tweak. That's not checked in because it breaks the build on a legacy server we still have running for one of the customers.

… someone else please take over… :-)

Watch the movie “The Thirteenth Floor”

We miss you Steve.

The average is $x. But that's global which means in some places like the US it is 10x. And in other less wealthy areas it is 0.1x.

There is also the strange paradox that the people who are willing to pay are actually the most desirable advertising targets (because they clearly have $ to spend). So my guess is that for that segment, the revenue is 100x.

I often wished for a Google Sheets like service that had an easier to use API and a bit more control over the schema. A cross between sheets and a headless CMS.

Nature _is_ technology. It’s far too advanced for us to understand. And perhaps due to not-invented-here syndrome, we tend to try and recreate it (crudely).

The problem is always discovery. There are lots and lots of little sites out there but without a search engine or directory you don’t know about them. Even back in the day Yahoo had a directory. And once you need such an aggregator, it will naturally create a choke point.

I do miss the earlier "heavy" models that had encyclopedic knowledge vs the new "lighter" models that rely on web search. Relying on web search surfaces a shallow layer of knowledge (thanks to SEO and all the other challenges of ranking web results) vs having ingested / memorized basically the entirety of human written knowledge beyond what's typically reachable within the first 10 results of a web search (eg: digitized offline libraries).

Haven’t tried this. Does it mix well with tool calls? Or does it force a response where you might have expected a tool call?

Somewhat related data point - our local elementary school used to have 5 first grade classes a few years ago and now they are down to 3. It's a large % drop over just a few years. This is going to move like a wave through the rest of the years all the way through to university.

I’m curious about this project (I’m working on something similar). Anyway to get in contact with you?

I have learnt to not ask leading questions. Always phrase questions in a neutral way and ask for pro/con analysis of each option.

Just here to say thank you for creating and maintaining MDN. It’s an amazing resource that I’ve used most days of my working life.

Metal detector + gate that denies entry if it detects metals?

At scale you need to use more tricks. For example, only inject examples if the tool is going to be needed. Or amass lessons, then ask the LLM to summarize them to prune redundant information before it is used in the context.

You could instruct the LLM to formulate a “lesson” based on the error and add this to the tool instructions for future runs.

Another approach would be more comfortable travel at today’s speeds (or even slower). Imagine boarding a plane with a luxury hotel like experience. A buffet breakfast, some work/reading in a nice library, followed by some treadmill / stationary bike time, then a shower, lunch, a massage in the spa. Dinner later before a classical concert and finally heading to sleep in a comfy bed. Then wake up and disembark at your destination.

How about tracking deltas between blood draws starting ant youngish age when things are on average presumed to be in a good state? When a new feature turns up in a subsequent blood draw, could it then be something more concerning?

Similar to the source of Mad-cow disease: feeding animal remains to cattle in the UK.

Anthropic recommends not using frameworks and simply writing code against the LLM apis, at least to begin with.

https://www.anthropic.com/engineering/building-effective-age...

I have collected some of the techniques I have developed/used in reducing LLM context size:

https://www.notion.so/LLM-Context-Engineering-21b814d6a64980...

Some of these are in use in an in-house AI chat application that has a heavy emphasis on tool calls.