Hacker News Reader

Well it's certainly small. Absolutely bombs my KTANE test though - poor character recognition, poor handling of even mildly complex tables, and prone to getting stuck in repetition loops. (Task was convert to docling, in the official HF space.)

That said, I'm definitely glad to see work in this area, particularly with open weights.

View on HN

https://news.ycombinator.com/item?id=43431609 mentioned that it was fine tuned from https://huggingface.co/HuggingFaceTB/SmolVLM-256M-Instruct

It would be interesting to see if fine tuning on your KTANE test improves your results?

I may be missing something, but if you train a model to pass a specific test… isn’t it obvious that it would do better on that test?

I thought we called models with test data in their training set “poisoned”

That is how training is done. You don't train on all of your material or the tests are worthless.

If you train on any test material, the tests are worthless.

Regarding the repetition loops, I found that adding the end of turn token to the stop param was enough. Documentation mentions detecting this.

But your point about quality stands. Separately, this model emits the docling XML format, not the JSON format, so as far as I know today that means you are using the Python flavored docling only, the JS variant does not support this yet (afaik).

Reply on news.ycombinator.com