llm_trw

If only there was a catchy name for it. Something like bit-torrent perhaps?

1). I'm incompetent enough to ignore publicly available table benchmarks.

2). I'm incompetent enough to never look at poor quality data.

3). I'm incompetent enough to not create a validation dataset for all models that were available.

Needless to say you're wrong on all three.

My day rate is $400 + taxes per hour if you want to be run through each point and why VLMs like Gemini fail spectacularly and unpredictably when left to their own devices.

That is you'd need 5 exa yotta bytes to solve it.

Currently the whole world has around 200 zetabytes of storage.

I short for the next 120 years mnist will need mathematical tricks to be solved.

As for markdown, great. Now how do you encode the meta data about the confidence of the model that the text says what it thinks it says? Becuase xml has this lovely thing called attributes that let's you keep a provenance record without a second database that's also readable by the llm.

I'm not assuming. We already did, 18 months ago with better performance than the current generation of Gemini for our use case.

You're falling into the usual trap of thinking that because big tech spends big money it gets big results. It doesn't. To quote a friend who was a manager at google "If only I could get my team of 100 to be as productive as my first team of three.".

If we had unlimited memory, compute and data we'd use a rank N tensor for an input of length N and call it a day.

Unfortunately N^N grows rather fast and we have to do all sorts of interesting engineering to make ML calculations complete before the heat death of the universe.

The rendering step for reports that humans got to see was a call to pandoc after the sxml was rendered to markdown - look ma we support powerpoint! - but it also allowed us to easily convert to whatever insane markup a given large (or small) language model worked best with on the fly.

[0] https://en.wikipedia.org/wiki/SXML

The number one take away we got was to use much larger images than anything that anyone else ever mentioned to get good results. A rule of thumb was that if you print the png of the image it should be easily readable from 2m away.

The actual model is proprietary and stuck in corporate land forever.

The table of costs in the blog post. At 500,000 pages per day the hardware fixed cost overcomes the software variable cost at day 240 and from then on you're paying an extra ~$100 per day to keep it running in the cloud. The machine also had to use extremely beefy GPUs to fit all the models it needed to. Compute utilization was between 5 to 10% which means that it's future proof for the next 5 years at the rate at which the data source was growing.

    | Model                       | Pages per Dollar |
    |-----------------------------+------------------|
    | Gemini 2.0 Flash            | ≈ 6,000          |
    | Gemini 2.0 Flash Lite       | ≈ 12,000*        |
    | Gemini 1.5 Flash            | ≈ 10,000         |
    | AWS Textract                | ≈ 1,000          |
    | Gemini 1.5 Pro              | ≈ 700            |
    | OpenAI 4o-mini              | ≈ 450            |
    | LlamaParse                  | ≈ 300            |
    | OpenAI 4o                   | ≈ 200            |
    | Anthropic claude-3-5-sonnet | ≈ 100            |
    | Reducto                     | ≈ 100            |
    | Chunkr                      | ≈ 100            |

>The longest part of the development timeline was establishing the accuracy rate and the ingestion pipeline, which itself is massively less complex than what your workflow sounds like: PDF -> Storage Bucket -> Gemini -> JSON response -> Database

Each company should build tools which match the skill level of their developers. If you're not comfortable training models locally with all that entails off the shelf solutions allow companies to punch way above their weight class in their industry.

You don't use multimodal models to extract a wall of text from an image. They hallucinate constantly the second you get past perfect 100% high-fidelity images.

You use an object detection model trained on documents to find the bounding boxes of each document section as _images_; each bounding box comes with a confidence score for free.

You then feed each box of text to a regular OCR model, also gives you a confidence score along with each prediction it makes.

You feed each image box into a multimodal model to describe what the image is about.

For tables, use a specialist model that does nothing but extract tables—models like GridFormer that aren't hyped to hell and back.

You then stitch everything together in an XML file because Markdown is for human consumption.

You now have everything extracted with flat XML markup for each category the object detection model knows about, along with multiple types of probability metadata for each bounding box, each letter, and each table cell.

You can now start feeding this data programmatically into an LLM to do _text_ processing, where you use the XML to control what parts of the document you send to the LLM.

You then get chunking with location data and confidence scores of every part of the document to put as meta data into the RAG store.

I've build a system that read 500k pages _per day_ using the above completely locally on a machine that cost $20k.

>In December, we kicked off the agentic era by releasing an experimental version of Gemini 2.0 Flash

I guess I wasn't building AI agents in February last year.

If for whatever reason B is stuck to T in all cases no one will think that A implies B, they will think that B is independent of A.

Yet that would still fit with a world in which A logically implies B.

You would need B to be dependent on a third variable which you're not controlling in the experiment that allows the value of B to change T or F outside your control.

Any good experimentalist will tell you to fix your broken experiment if that kept happening.

It's only when you have no control over the variables and you collect statistics that the view in the paper makes sense.

That's the difference between experimental and observational science.

And the philosophy of science is decades to centuries away from understanding that in any meaningful way.

At that point all the special sauce Google et al have spend decades mastering will be worth as much as expertise in analogue computers is today.

Currently they charge 2.5c for an API search. This is between 1,000 to 1,000,000 times more than other companies in the space charge.

AI systems need to do dozens of searches for every question to get good results and kagi's results are really good. But not 1,000,000 times better than the competition.

His management style is best described as humiliation and abuse.

Or to put it another way, you can build a lot of pyramids when you use disposable slaves.

    B = (A or B) and (if A then B)

    | A | B | x A B | y A B | z A B | If A then B |
    |---+---+-------+-------+-------+-------------|
    | T | T | T     | T     | T     | T           |
    | T | F | F     | F     | F     | F           |
    | F | T | F     | F     | T     | T           |
    | F | F | F     | T     | F     | T           |

Or to put it another way: No one outside a logic course seriously thinks that A implies B means the same thing as not A or B.

Here's the truth table for implies (if) in logic.

    | A | B | If A then B |
    |---+---+-------------|
    | T | T | T           |
    | T | F | F           |
    | F | T | T           |
    | F | F | T           |
    

This is where you need to add temporal logic so that the scientific understanding of A casually implies B can be represented in logic.

In short the paper does nothing of the sort of what it says it does because it fundamentally uses the wrong tool for the job.

We're close to the point where using a human -> stt -> llm -> tts -> human pipeline you can do real time high quality bi directional spoken translation on a desktop.

Tensors, as used in ML, are much closer to a key-value store with composite keys and scalar values, with most of the complexity coming from deciding how to filter on those composite keys.

Drop me a line if you're interested in a chat. This is something I've been thinking about for years now.

Like I said, a very bipartisan position.

Dragging us back to the 1930s seems to be a very bipartisan policy.

But also: https://en.wikipedia.org/wiki/Supergod

Not a place I think anyone should try and emulate.

A trans vegan gang murdering police officers is what's come out of this milieu.

I don't see how anyone can say they aren't taking "progressive pieties as axiomatic".

The OP is just taking the "everything I don't like is fascist" trope to it's natural conclusion. Up next: Stalin actually a Nazi.

We know where that one took us.

Since I have access to the thinking tokens I can see where it's going wrong and do prompt surgery. But left to it's own devices it gets thing _stupendously_ wrong about 20% of the time with a huge context blowout. So much so that seeing that happen now tells me I've fundamentally asked the wrong question.

Sonnet doesn't suffer from that and solves the task, but doesn't give you much if any, help in how to recover from doing the wrong task.

I'd say that for work work Sonnet 3.5 is still the best, for exploratory work with a human in the loop r1 is better.

Or as someone posted here a few days ago: R1 as the architect, Sonnet3.5 as the worker and critic.