jcheng

It's not quite as trivial as that; one could start the page with a <script> tag that contains "<!--" without matching "-->", and that would hide all the content from your scraper but not from real browsers.

But I think it's moot, parsing HTML is not very expensive if you don't have to actually render it.

The layoffs at Amazon and Microsoft are not due to lack of profits. They’re massively profitable right now.

https://www.macrotrends.net/stocks/charts/MSFT/microsoft/ebi...

https://www.macrotrends.net/stocks/charts/AMZN/amazon/ebitda

Not if you're passing binary data

Why taxpayers? Where’s the systemic risk in AI labs getting acquired for cents on the dollar? The taxpayers weren’t holding the bag during the dot com crash, just investors.

For 2, a lot of companies use AWS Bedrock to access Claude models instead of Anthropic, for exactly this reason. Amazon’s terms say they don’t log prompts or completions and don’t send anything to the model provider. If your production database is already hosted by AWS, it doesn’t seem like much additional risk.

No, I didn’t provide any tools

The difference in coding ability between then and now is pretty huge. And a year ago o1 hadn’t been introduced yet, whereas now the “reasoning” technique is pretty widespread.

Not sure if you’re counting things built on top of the models but if so, coding agents have also come a long way since a year ago.

I want this for after the code has run and returned results. Often when you use code to answer questions about a table, the result is in the form of a smaller table. I'd like to know how small that table needs to be before you can rely on the model being able to make reliable observations about it.

    uv add google-genai
    uv run scripts/run_benchmarks.py --models google/gemini-2.5-pro --formats markdown_kv --limit 100

And add GOOGLE_API_KEY=<your-key-here> to a file called .env in the repo root.

Unfortunately I started getting "quota exceeded" almost immediately, but it did give 6/6 correct answers before it crapped out.

gpt-5 also got 100/100 for both CSV and JSON.

    uv run inspect eval evals/table_formats_eval.py@table_formats_csv --model openai/gpt-5 --limit 100
    uv run inspect eval evals/table_formats_eval.py@table_formats_json --model openai/gpt-5 --limit 100

I was curious enough to have Codex create a similar benchmark: https://github.com/jcheng5/table-formats

With 1000 rows and 100 samples and markdown-kv, I got these scores:

- gpt-4.1-nano: 52%

- gpt-4.1-mini: 72%

- gpt-4.1: 93%

- gpt-5: 100%

I was so surprised by gpt-5 getting 100% that I ran it again with 1000 samples. It got 999 correct, and one wrong.

To reproduce it yourself, clone the repo, add a .env file with OPENAI_API_KEY, `uv sync`, and then run:

    uv run inspect eval evals/table_formats_eval.py@table_formats_markdown_kv --model openai/gpt-5 --limit 100

Update: Also, number of rows makes a massive difference, unsurprisingly; at 100 rows, gpt-4.1-nano scores 95%+ for both markdown-kv and csv. Both model and record count seem to matter a lot more than format.

Thanks for the thoughtful reply. I agree with all that and I think I was trying to say something similar with my last paragraph.

I’ve gotten along with _almost_ every person I’ve worked with, including some pretty challenging personalities. I’ve always done very well at my job duties and gone “above and beyond” regularly. The only times I felt that might not be nearly enough, was at the two large companies I’ve worked at. Someone several levels away from me, that I would never meet, would decide whether I got a promotion or a raise or a juicy new assignment based on a game of organizational telephone. Frankly, when I tried I did pretty well at that game, but it was the first time in my career that I was tempted to do something out of cynical self ambition or winning rewards for my team instead of just trying to do the right thing for the customer or the business.

That’s what I think of when I hear “politics” and why, by comparison, it felt to me like at a small enough scale it’s not a thing. But if politics means needing leadership to like and appreciate you, then yeah absolutely, that is true anywhere there’s even one level of hierarchy.

I’ve mostly worked at small startups in my 30 year career, and been fortunate to work mostly with excellent founders and teams. Certainly that experience colors my convictions, as your experience colors yours.

It's SO different. It doesn't guarantee that life is good, but politics plays a much smaller role, especially at the very small sizes. If you're objectively awesome, the chances of you being sidelined for political reasons is pretty low when there are like 10 people in the whole company. It's just obvious to everyone who is delivering value and who is not.

Conversely, if you're mediocre, there's nowhere to hide.

Or maybe instead of saying there aren't politics at small companies, it's more accurate to say that there are politics, but they're simple--everyone strives to make the (hopefully benevolent) dictator, I mean founder, happy. If your founder is awesome, life is good. If your founder is not awesome, well, everyone is going to have a bad time anyway.

A huge red flag to me is that the tool calls here are stateless (every tool call is carried out by a new R process) which means the state has to live in the agent’s context, exactly where you don’t want it for so many reasons. For example, reading a 20MB CSV will immediately end the conversation for any LLM that exists today. And even if it fits, you’re asking the LLM driving this to transcribe the data verbatim to other tools—it has to literally generate the tokens for the data one by one (as opposed to just passing a variable name or expression). This is very slow, very expensive, capped at max output token count, and an opportunity for the LLM to make a mistake.

If the author(s) want to reach out to me, I’m happy to talk about alternative approaches or the extensive native R LLM tooling that exists now. Email in profile.

Do you have any advice for how to suss out someone's hunger, drive, and scrappiness during the hiring process?

If the TTL of one domain’s records are all shorter than the TTLs of another domain’s, what would make downstream and client caching cancel out? Do clients not respect TTLs these days?

(In this particular case, I don’t think the TTLs are actually different, but asking in general)

The heading says “Generative AI services popularity - Top 10 services based on 1.1.1.1 DNS resolver traffic”

Here's my attempt at explaining tool calling:

https://youtu.be/owDd1CJ17uQ?si=Z2bldI8IssG7rGON&t=1330

It's an _incredibly_ important concept to understand if you have even a passing interest in LLMs. You need to understand it if you want to have any kind of mental model for how LLM-powered agents are even possible.

For as long as there have been Jupyter notebooks, there have been people saying "Wait, what??" But there have been way more people using them, and finding them pretty effective despite their inherent lack of soundness.

I personally fall into both camps--both horrified at the execution model but also, it's not actually caused problems for me as it's second nature to restart the kernel when changing upstream cells.

That said, marimo feels like a significant step forward and I'm rooting for them to succeed.

Not at all. The iPod of the day had a click wheel as an input device. The iPhone introduced us to capacitive touch, multitouch, gesture recognition, full web browsing. Huge leap compared to not only the iPod but the PocketPCs of the day.

Listen to the audience reaction when he shows how you scroll a list view: https://youtu.be/VQKMoT-6XSg?si=OmbgSG7nmEpdAETl&t=970

Same here, I spent forever on it :(

Thank you!

Only input, not output.

https://modelcontextprotocol.io/specification/2025-03-26/ser...

Stuff Made Here https://www.youtube.com/watch?v=WsPHBD5NsS0

Mark Rober https://www.youtube.com/watch?v=Sqr-PdVYhY4

And what about using the “back door Roth IRA” to get around the Roth IRA’s income test? What about when the IRS says the back door Roth is allowed?

https://www.currentfederaltaxdevelopments.com/blog/2018/7/12...

200 examples at most, really?? I have been led to believe that (tens of) thousands is more typical. If you can get excellent results with that few examples, that changes the equation a lot.

Or better yet, use CTEs: https://duckdb.org/docs/stable/sql/query_syntax/with.html

Those are great, I would watch any one of those movies. Maybe even the "Across the Indiana-Verse" one where they are all pulled into a single dimension.

I'm sympathetic in general, but in this case:

"You will need git, XCode tools, CMake and libomp. Git, CMake and libomp can be installed via Homebrew"

That really doesn't seem like much. Was there more to it than this?

Edit: I tried it myself and the cmake configure failed until I ran `brew link --force libomp`, after which it could start to build, but then failed again at:

    [ 55%] Building CXX object src/CMakeFiles/solvespace-core.dir/bsp.cpp.o
    c++: error: unknown argument: '-Xclang -fopenmp'