gopalv

> They are hesitant to take their hand off the wheel because they don’t have full confidence the agents have the right guardrails in place to run off on their own.

This is not a new problem, I've had this problem every summer when a new batch of interns show up.

As they say, if an intern can bring down production, it is not the intern's fault.

The interns are generally so much better than me at writing code, but they're not the best at "software development" (I call it "software gardening").

My training regime is to push them up the next 4 leadership levels from "writing code" - delegation, negotiation, vision and inspiration.

Agents are barely at delegation, with multi-agent harnesses.

They don't negotiate with you, they say yes.

Their vision is limited to the context you gave and most of their output is uninspiring (you can smell "Claude did this" on a UX design pattern easily).

But the humans, they've been thrust into delegation on day 1 & unless you've been a tech-lead manager, you probably haven't exercised that skill yet. Even if you have done that, the way agents work is not exactly how humans work [1] in terms of incentives or even try something they think won't work.

If you've been a good IC for years, the company could always afford to hire a human who specializes in those skills and suddenly you've been promoted to a new role without a pay bump or even a choice.

Most of the "Let's add agents and go faster" is basically adding more manpower to a project behind schedule and the payback is missing because that causes communication gaps & dropped messages.

[1] - https://tinyurl.com/orchestration-imperative

> The search tree is a binary trie over the bits of an IP address.

The interesting thing here is that the format is still the same.

I read the original spec and it did call out that

> In practice, data deduplication is handled by pointers, making this type unnecessary.

If any entry can be a pointer, you can shrink this recursively for each field type like a .strings table design.

> On-policy distillation: The expanded student model learns token-specific feedback from teacher distributions generated on self-sampled sequences.

The weird thing is to see this based off the KV-Cache in the diagrams.

Completely different from the Databricks RAFT [1].

My team built "Dreamtime" earlier this year [2], which used a different mechanism to distill out learnings across multiple sessions.

This didn't pull from the token layer, but from user nudges on unsatisfactory responses.

This works like a sawtooth wave where we accumulate context, check for dissonances (a user gets to say "It depends" and fix it), compress it back, retire the old one.

The audit requirements are the real problem for customers, where you need to be able to ask "When did this change? and who changed it?" which was hard to answer from the lowest layer of the model unless you can retain lineage to the original sessions.

A model we train can't serve as an accomplice to a user who wants to hide some financial crimes by nudging the agent repeatedly.

[1] - https://arxiv.org/abs/2403.10131

This Ousterhout lecture lives in my brain (this one and the one about scar tissue).

"A Little Bit of Slope Makes Up For a Lot of Y-Intercept"

https://www.youtube.com/watch?v=mE98Is1quwY

> Pick a radical idea

> Build it over a weekend:

What AI has done is that we can do 10x more work a week, while only delivering 2x to production.

Right now, there are 4 people (and me) working on the same idea in 5 different ways - we're all training small language models, the goal is to get smaller or fewer active parameters.

By next week, we'll have lost on 3 different approaches. I know my approach is a dead-end already, but I'm not ready to give up till the pencils down.

The people I work with are great, because they will see the better solution by the end of it, whether they proposed it or someone else did.

A couple of them have been in the industry for less than 5 years and I feel like their slope is better than mine, which means I'm going to have a good time when they're fully up to speed :)

Gdb is an excellent debugger for assembly, even when the original code was written in C

I would automatically type

(gdb) x/4i $pc

when opening a coredump out of habit.

But using nasm, then looking as gas style assembly in the debugger might be a bit more jarring.

Python is harder to JIT because of the way __dict__ works.

If all python had was __slots__ then we would not be struggling to JIT python.

Tight numerical loops is where the JIT adds performance.

Most performance sensitive features are already available in python between numpy, numba and Cython without really leaving the language core.

You'll mostly be speeding up the slow parts of python, not supercharging the fast parts.

This is a great example of going from AOT to JIT specializing function calls by types.

I assume this intended at a class which has already finished writing an AOT compiler and wants to build faster functions than you can derive at a compile time.

> On the first call to a function, the compiler calls compile_me, which reconstructs argument types from their tag bits and builds a temporary type environment. The function body is then optimized using those concrete argument types.

It's been two decades since I have worked on Rhys's brilliant libjit and the work that came before it[1], but the first-call recompilation can mix well with a byte code interpreter as well.

You still need to be able to JIT a whole function with the first-call recompiler here, but you can do even more incremental if you are pairing it with an interpreter which gets to pick up the slack of the JIT.

For example, with an interpreter, I could JIT about ~15% of a function and incrementally implement one opcode at a time without handling the complex ones on day one.

The switch loop with direct threading can jump into a JIT block or the next instruction's compiled location, in identical fashion. So you get to operate register to register for adjacent instructions being JIT'ed together, but write back to heap for the interpreter to take over for an unsupported one.

This is referred to as unrolling in the impl there, because what it really does is partially unroll the for(op in ops) loop and skip the jump to the top when you see a sequence of ops which you can line up together and JIT once.

[1] - https://mirror.linux.org.au/pub/linux.conf.au/2003/papers/Rh...

[2] - https://docs.rs/gnu-libjit/latest/gnu_libjit/struct.Function...

An air purifier is a good addition to a kids bedroom.

I got one during Santa Cruz wildfires in 2020 and now the kids have a pavlovian reflex to sleep once that gets turned on.

Ikea makes an air-purifier table, which is a end-run around it being a useless box in the room when switched off.

It's fairly expensive, not as good as a purpose built one, but it blends into the average room pretty well.

> does duckdb support querying custom storage formats?

No, this is still parquet being loaded in.

The "trick" is running through sort order combinations which result in maximum packing in a Dremel style data format (like Parquet).

Here's the trick explained in enough detail to understand, but not enough to copy as-is

https://cloud.google.com/blog/products/bigquery/inside-capac...

Search for the section with

"Capacitor builds an approximation model that takes into account all relevant factors and comes up with a reasonable solution."

or a more detailed dive in the paper (by the favourite author of performance engineers, Daniel Lemire)

https://arxiv.org/abs/1207.2189

> because of the overhead of parallelizing, despite them running at 100%

Duckdb might not be the reason, the real reason might be AWS.

It's been two years since I worked on this, but AWS Nitro might be triggering a high CPU Steal % when all the cores are busy at the same time.

You can do taskset -c 1-191 <server-process> and get a tiny bit of breathing room there.

I almost always do this above 32 cores, since this lets me always have a nice responsive ssh connection when the rest of the CPUs are churning high.

The more cores you have, the less of a problem sparing 1 core becomes.

> read_parquet() making tons of tiny range requests to load a parquet's footer from S3

This is somewhat unavoidable if you have a number of files and a tight filter which rejects most of them. The footer choice was a bad one in hindsight (not for HDFS), but for anything which uses XOR or other kind of erasure coding. The other other file format I worked with used headers only instead of footers, to save a few ms in latency for the initial GET

I don't think Ducklake can maintain a zone-map or ART index over parquet files, but there is no equivalent of iceberg metadata/something which has min/max/HLL compacted across multiple files yet.

The secondary indexes with file identifiers stored in a sequential file is most likely the next useful thing duckdb can make.

> What are you guys using it for?

Transparent queries on server/browser in identical sql.

If you play all the tricks Google's Capacitor does to the data layout, you can shove ~2-3 million rows to a browser based query engine in 16-20Mb with duckdb-wasm. We can scale it up to ~1Gb with HTTP range queries, with cache-control.

Then we use Jev to write SQL queries over it in 1-2s, whenever someone has a followup question. Getting Jev to write SQL takes a bit of prep, but my co-founder wrote a paper about clairvoyant query execution a few years ago [1].

I could have 3000 users logged into the same application, but all of them would be bringing their own CPU to run duckdb. My compute scaling problems for my data cube is a solved problem with this approach.

[1] - https://dl.acm.org/doi/10.14778/2733004.2733064

> Not just reading data from S3, but the actual computations too.

Async io generally causes a CPU bump as a side-effect, when the bottleneck moves from the disk to the CPU (more usually memory bandwidth).

My experience is from a different SQL engine, but the improvement in scan rates directly translates all other operators getting more work to do per unit time.

More CPUs pegged at 100% is always a good sign of smooth flows, but optimizing further requires data layout changes from that point. Even if it is a sub-linear speedup out of concurrency scaling - more with more is still a win, assuming you can pay for more (the Dy-skew paper has an example I can quote).

My dad used to joke that a meeting is where minutes are kept and hours are lost.

Buried in the pdf in different sections

> Managers attend meetings far more frequently and spend substantially more time in them than non-managerial employees.

> Separately, interactions with senior colleagues are associated with stronger wage growth

A meeting is called to coordinate or communicate. Wadge's law of meetings tell you which one is which.

But not all coordination is top down, sometimes you need your manager to go negotiate a point with another division.

And neither is all communication, most often you can impress on others the potential of a new approach or the problems with the existing one.

The only trouble for the neurodivergent among us is that "appearing to pay attention" is also communication, same goes for eye-contact or not stuttering.

Corolllary is that skip meetings at your own risk, decisions will be made and you won't be in the room where it happens. And don't be the person who is not welcome in meetings, that too eventually makes your pay suffer.

> A switch statement with many cases can be compiled as a jump table (fast, O(1), but requires a full table load and indirect jump) or as a cascade of compares (slower per-case but better branch prediction if one case dominates).

Switch case is the workhorse of a lexer, parser and execution engine. Lots of them over enums.

This was all mis-optimized for years thanks to Spectre/Meltdown changing how the actual runtime of pre-fetch and speculation used to work.

All my work on CGOTO was made irrelevant by the CPU advances, only to become relevant again when fetch speculation got nerfed.

The indirect call scenario is a bit more interesting, because the JVM has an amazing paper about interface dispatch caches [1] on how to do it runtime instead of compile time.

The PGO is still useful, but not unless you can predict the workload you're going to have - Benchmarketing with a TPC benchmark is a good situation for it.

If you have consistent hardware, tuning an OLTP system for the hardware is much more interesting. Going back in the memory hole, Yahoo used to have a Jeremy Cole build of Mysql which was specially optimized for the raid disk layout that was in production (ZFS came by to fix those issues directly).

[1] - https://dl.acm.org/doi/10.1145/504311.504291

> They still do things the old way but Backbone can run IPv6 only at its core.

This would turn an internal IPv4 into an externally routed IPv6, which is a neat trick which would replace all the NAT madness that we do run today.

I'd like nothing more than the ability to run ipv4 internally indefinitely, especially with things like mesh networks where the ACLs are carried over from ages ago (2018 doesn't feel that long ago, but it is).

Most of the time I don't even need a route to a direct connection, I just need legible flow logs out of a kubernetes platform.

> what glued panels fall off

The Lotus Elise was famously glued together instead of welds. The original Roadster also had the same innards.

This is clearly not a car for someone who wants to go from A to B, but a lot of Tesla components gets binned together for the next iteration of the cheaper car because they always order too many of them.

Oddly enough this makes their cheaper cars more reliable than the expensive ones (think about the Audis right when Ferdinand Piech took over Lamborgini).

Picasso apocryphally said "good artists copy, great artists steal"

> Imagine how they must feel about their work being deliberately copied wholesale and intentionally presented as someone else’s.

Nobody copies from a mid-tier product.

As someone who worked in open source till recently, I've seen a copycat as validation that my idea was better than what they could come up with on their own.

All those people now have evidence that their approach is the best for an AI native future.

The interesting point is that there is overlap in personnel, so this is less of copied homework, more that Anthropic hired some good people who knew a good solution that worked.

But from a pure AI lens, this is what we're doing - putting great ideas into one system and now everyone can drink from that well.

The bad part is that the well has a coin meter next to it, rather than it being a commons. So the fruits of that is not distributed to the people who planted those seeds. This is also happening too fast that the people whose work is getting redistributed to others are still not done being compensated for their work.

> forcing function to clean up tech debt

The real reason rewrites to fix tech debt used to fail was that it took too long to get to feature partity & the goal posts moved constantly while you were on the rewrite team.

The sunk cost of existing code is no longer considered in a rewrite.

Maybe it was hard to write in the first place, but it is not hard to rewrite anymore, as long you've got decent tests.

You can turn a lot of process and workflow systems into deep decision trees.

My team hasn't finished writing their blog posts yet, but the customer demos [2] went out the moment TypeSafe took our call and sorted all the paperwork out.

There's a tiny section at the bottom of the demo which explains how it actually works (or the github repo is the code).

We have a single decision API[1] over all similar models, so we run evals across all the "System One" api products & definitely don't trust any benchmark that you haven't run. All standardized benchmarks have been benchmaxxed already.

PS: just in case it wasn't clear, the demo is a game where we play to calibrate the human vs machine decision making (or just do it in auto-pilot mode).

[1] - https://github.com/prasanthj/duckdb-dual-cognition

[2] - https://demos.isotopes.ai/demo/apps/chain-command

The annoying thing with hot-sauces is that they're somewhat like a vintage product and longer it ferments the more lag the decisions have on the sales.

The management decisions to scale up production take years to materialize on your food.

Even if you hate what you got out of a bottle you bought this week, that taste is not what made the sauce popular years ago. Popularity seems to cause a quality fade in everything that scales out of a single factory.

For example, the Sriracha from 7 years ago tasted totally different than today & it is the peppers that changed, not the recipe. Palo Alto fire fighters has gotten more cumin than chillis as the spicy stuff goes to the XX version. The Athabascan sauces I've loved from Alaska have slowly morphed sweeter and maple.

Tabasco for all its faults hasn't changed that much in the 20 years I've been having it.

On my M3 mac, it works okay inside a docker container with just CPU.

{ "model": "strands-decider-2B-hobson-v19", "answers": { "is_urgent": { "type": "noul", "noul": 0.8287 } }, "usage": { "input_tokens": 86, "output_tokens": 1 }, "latency_ms": 1732.17 }

This is how I got it running - https://gist.github.com/2891eb0db9ea92c1a4e860d44f556292

There's a lot more to be done if we optimize for MLX & let it run on a Mac mini instead of the docker wrapper.

> I write harnesses that do decision steps and regularly use competing models

Competing models is table stakes for decision making.

While doing the ablations for the "Team of Rivals" paper, I found that the models were very happy to ignore errors[1] when the data it is classifying looks like its own output.

Jev slots into the middle critique in the paper. The only reason we haven't flipped to Jev last week is because we're a HIPAA shop and Typesafe hasn't responded to our emails.

[1] - https://github.com/t3rmin4t0r/critique-evals

> This write amplification is large enough that our efforts to tune indexing throughput have started to hit diminishing returns.

> don't key on the ANN address. That is precisely the change turbopuffer v3 makes. As you can imagine, it is not a trivial change.

This is a direct parallel to how Postgres and Mysql built indexes.

Your design choice went from a Postgres design pattern to a Mysql one. The difference is the reindexing cost vs the lookup cost - Postgres optimized for lookup and Mysql does for indexing on writes. Or more accurately, Postgres was better with good schema design using joins & mysql was optimized for a bad design with less normalization where many indexes exist for the same table.

Postgres always points an index to a row-id within postgres which is an arbitrary value which changes on each update.

Mysql, always assuming the storage engine is pluggable, points to the primary index entry and adds an extra indirection to the lookup.

This means that you point the mysql index to a stable id, so unless you go update the primary key for a row, you won't have to update the indexes for all the attribute lookups you might have made to data.

I don't do databases any more that much, but the design for NIMBLE file format has a lot of quirks which are relevant to this specific idea (wide tables).

But the old Uber post about switching from Postgres to Mysql to prevent index amplification[1] is a direct mirror to this post.

[1] - https://www.uber.com/us/en/blog/postgres-to-mysql-migration/

> taking careful precautions against feeding the findings back into training so as to not risk shaping Argon’s reasoning to evade our monitoring. We strongly encourage the rest of the industry to preserve reasoning transparency in these pivotal moments of increased capabilities while navigating alignment risks, so that model thoughts remain helpful in identifying and diagnosing misalignment.

This is good, but they're the slow mover due to this exact thing.

Google is getting punished for not letting the models enter an echo chamber and go faster than humanly possible.

The whole thing reminds me of the Apple feature flag story[1] from a generation ago.

[1] - https://news.ycombinator.com/item?id=6372466

> it’s missing the bigger integration offerings Tailscale has/does still.

Until I had tailscale serve generating valid certs, I had a good reason to use Cloudflare tunnels.

But in general I don't want to put everything on the internet side of things.

Mostly, I don't want something open, but more like a "share with" for people who are in the same office (virtually over tailnet, not physically on the same LAN).

This still works great for a demo instead of a product pitch, to send an link out to see something.

I'd still use a real host over a laptop for those.

The "Emacs doesn't have a file-tree" is when I stopped short and wondered what this person's background is in IDEs.

My emacs days are 1999 during my "functional or get out" phase, but my .emacs had close to 4000 lines in it, including xterm mouse support, a clone of midnight commander + sftp to a wehbost and deep integration with ctags/cscope/c++filt. There was even a line by line debugger with custom .gdbinit commands - look at the php project's gdbinit[1] if you want an idea of how to walk your own super special data structures.

You can't put Emacs in an IDE comparison and say anything about features without at least one person saying "well, actually".

I eventually switched to vim but that had to do with needing to ssh into 100k+ machines on support rotation & never having emacs on any of them.

[1] - https://github.com/php/php-src/blob/master/.gdbinit

> doesn't make you a martyr because you get anxious

Every person's biggest problem is their biggest problem and it eats up their life in the biggest way possible.

Maslow's hierarchy is not a "smaller and smaller" problems list, it is a definition of what your biggest problem is & the solutions just scarcer as you climb up the ladder.

It's close to 20 years since I wrote about the same "problem" [1] in my life and the intervening years have brought much more grief, toil and resilience. Death of several loved ones causes you to consider what a virtue sounds like an eulogy and what yours will sound like in the end.

As an aside, I've felt like literature & art was backwards in the way it was introduced to me - where the first came "man vs nature", then man vs fellow man & then a "man vs self" battle. After fighting self and others, most of us will succumb to nature instead.

So, whenever someone vents or complains, just remember that this is not a contest or at least rage bait, but to be seen as a genuine struggle within yourself that you have put to words.

There's an episode of "Person of Interest" where the AI singularity struggles to play chess as each move cuts the possible moves and the first move is paralyzingly hard for it. Some people are like that, but regret is a form of hope without feathers. It comes from a strong belief that if you could wind the clock back and start over, it would be all different.

I nearly fell into the trap of Nihilism there ("Consider the Total Perspective Vortex")

[1] - https://notmysock.org/blog/me/a-buffet-intellectual

> Weekly downloads number doesn't.

The ChatGPT app downloads a copy of LibreOffice when you ask it to make pptx files.

So I've downloaded LibreOffice 3 times this week without really paying attention to it.