Never hire zealots who don't share your religion, they may be cheap - or free in this case - but in the long run they cost you a lot more.
3,929 karma · joined February 12, 2024
There are a few non-linear function operations in between the matrix multiplications.
Pretending that Europe wasn't in a perpetual blood bath since the end of the Pax Romana until 1815 shows a gross ignorance of basic facts.
The printing press was a net positive in every time scale.
I'm not saying that we should let people board in full armor again, but surely there is a happy medium?
AI is amazing at OCR, we've had tesseract ocr for 40 years and if you read the fine manual it has essentially a 0% error rate per character.
OCR on VLMs is terrible.
For some reason consistent x-heights between 10 to 30 pixels with guaranteed mono-column layout is not something venture capitalists get excited about, and as a result I'm not the founder of a unicorn.
>I still work on AI powered products and I don’t see even a little line of sight on this problem. Everyone’s data is immensely messy and likely to remain so.
I've worked in the space as well and completely unstructured data is better than whatever you call a database with a dozen ad hoc tables each storing information somewhat differently to each other for reports written by a dozen different people over a decade.
I have a benchmark for an agentic system which measures how many joins between tables the system can do before it goes off the rails. But there is nothing off the shelf that does it and for whatever reason no one is talking about it in the open. But there are companies working to solve it in the background - since I've worked with three so far.
Without documentation giving some grounding about what the table is doing, you're left with hoping the database is self documenting enough for the agent to figure out what the column names mean and if joining on them makes sense - good luck doing it on id1, id2, idCustomerLocal, id_customer_foreign though.
The mistakes are also very much model dependent. That you have build a system which improves the accuracy of one models output give you no confidence that it will work on even the next generation of the same model.
After a conversation with humans I think you'd agree 1% of them being able to reason deeply is a vast overestimation.
A good example to see how little people can reason is the following classic:
> Given the following premises derive a conclusion about your poems:
> 1) No interesting poems are unpopular among people of real taste.
> 2) No modern poetry is free from affectation.
> 3) All your poems are on the subject of soap bubbles.
> 4) No affected poetry is popular among people of taste.
> 5) Only a modern poem would be on the subject of soap bubbles.
The average person on the street won't even know where to start, the average philosophy student will fuck up the translation to first order logic, and a logic professor would need a proof assistant to get it right consistently.
Meanwhile o3-mini in 10 seconds:
We can derive a conclusion about your poems by following the logical implications of the given premises. Let’s rephrase each premise into a more formal form:
Premise 1: No interesting poems are unpopular among people of real taste. This can be reworded as: If a poem is interesting, then it is popular among people of real taste.
Premise 2: No modern poetry is free from affectation. This tells us: If a poem is modern, then it is affected (i.e., it shows affectation).
Premise 3: All your poems are on the subject of soap bubbles. In other words: Every one of your poems is about soap bubbles.
Premise 4: No affected poetry is popular among people of taste. This implies: If a poem is affected, then it is not popular among people of taste.
Premise 5: Only a modern poem would be on the subject of soap bubbles. This means: If a poem is about soap bubbles, then it is modern.
Now, let’s connect the dots step by step:
From Premise 3 and Premise 5:
All your poems are on the subject of soap bubbles.
Only modern poems can be about soap bubbles.
Conclusion: All your poems are modern.
From the conclusion above and Premise 2:
Since your poems are modern, and all modern poems are affected,
Conclusion: All your poems are affected.
From the conclusion above and Premise 4:
Since your poems are affected, and no affected poem is popular among people of taste,
Conclusion: Your poems are not popular among people of taste.
From Premise 1:
If a poem is interesting, it must be popular among people of taste.
Since your poems are not popular among people of taste (from step 3), it follows that:
Conclusion: Your poems cannot be interesting.
Final Conclusion: Your poems are not interesting.
Thus, by logically combining the premises, we conclude that your poems are not interesting.
I just put them in the same bucket of intelligence as an 8b model and weight their inputs accordingly.
Real reasoning is being able to manipulate symbolic expressions in a consistent manner while preserving some invariants.
Personal experience as logic is how you end up with the Holocaust.
o3-mini high effort can derive chains that are 8 inference rules deep with >95% confidence I didn't have the money to test it further. This is better than the average professor in logic when given pen and paper.
It seems like a course critiquing 5 year old technology at this point.
It's trying its hardest to be put in the "this is bullshit" pile.
Five lines of content spread through five pages through the magic of parallax scrolling. Examples wandering around without going anywhere and confidently repeating talking points that were debunked 3 years ago.
Please release a textbook that can be read instead of whatever this is.
The fact that all of these are (or can be) decimal machines makes it much easier for people to understand what's going on.
Jumping to binary before you understand how a computer works is like learning logic in Latin.
[0] https://en.wikipedia.org/wiki/CARDboard_Illustrative_Aid_to_...
[1] https://en.wikipedia.org/wiki/MIX_(abstract_machine)
I've been playing around with a modern version of the cardiac, drop me an email if you'd like to collab on testing it on live students.
Surely it should be easy if you're asking the Linux people to do the same thing for drivers written in Rust?
We never had the budget to do it but I do have some notes somewhere on a 2d context free grammar to generate arbitrarily nested rows/columns and a css styling that got applied to the xhtml output of the grammar. It dynamically generated as much high quality synthetic data as you wanted - but the IBM and similar data sets were plenty big enough for what we could do even on specialist models.
It depends on what you're doing really. I thought that we'd done pretty well, then someone on HN reached out with a table that spanned 50 pages and I just gave up.
Feel free to drop an email if you'd like a quick chat. I find the state of table models particularly abysmal for how important they are.
A fun threat to read for the current hype cycle.
You can tell who is working in the field by the fact they don't use VLMs for OCR and who isn't because they think it's a solved problem.
A question to the authors.
Do you have resources to train any VLMs from scratch? They aren't quite the bests the sota LLMs are and I think they can be made a lot more useful with:
1). Better training data.
2). Larger vision parts of the model.
In short: 2d attention is not something that anyone's doing at scale - that I know of - and is a no brainer for understanding images.
To even have a chance at doing it you'd need to start the training from scratch with _huge_ penalties for filling in missing information and a _much_ larger vision component to the model.
See an old post I made on what you need to get above sota OCR that works today: https://news.ycombinator.com/item?id=42952605#42955414
I laugh every time I hear someone tell me how great VLMs are for serious work by themselves. They are amazing tools with a ridiculously fluctuating (and largely undetectable) error rate that need a lot of other tools to keep them above board.