645 karma · joined December 6, 2014
- https://karllorey.com
- https://github.com/lorey
- https://startupradar.co
- https://markets.apistemic.com
Example: Which LLM gives me the best ELI5 explanations for a given price. https://evalry.com/benchmarks/explain-like-i-m-5-321
> The team that made dataroom has stated that they did not use any of papermark’s code and that dataroom was made from scratch with inspiration from existing document sharing softwares, and that this post’s allegations of us stealing code are false. [...]
The screenshots clearly show they copied whole pages verbatim, both design and texts. The founder, Nico Laqua, basically responding with "we didn't copy _code_" and not taking any responsibility says a lot about his and his company's moral code. It might not be enough to get sued. That doesn't make it right.
In most cases, e.g. with regular ML, evals are easy and not doing them results in inferior performance. With LLMs, especially frontier LLMs, this has flipped. Not doing them will likely give you alight performance and at the same time proper benchmarks are tricky to implement.
Any resources you can recommend to properly tackle this going forward?
I'm using a monorepo for my company across 3+ products and so far we're deploying from stable release to stable release without any issues.
For anyone interested: https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/...
Examples:
- YC: https://markets.apistemic.com/companies/y-combinator-goaq9
- uber: https://markets.apistemic.com/companies/uber-com-ojj2j
- Anthropic: https://markets.apistemic.com/companies/anthropicresearch-yx...
Try it for any company here: https://markets.apistemic.com
Mine: https://github.com/lorey/mlscraper Another: https://github.com/alirezamika/autoscraper