Hacker News Reader

mlukewizardon DishwasherBench – Can Frontier Models Stack a Dishwasher?

ahahhaa

mlukewizardon DishwasherBench – Can Frontier Models Stack a Dishwasher?

yea super interesting q

my feeling is that stacking a dishwasher involves enough image processing and computer-use (or more realistically robotics) that it's a frontier model job rather than a 'write a program to do it' job

but keen to hear if you disagree

mlukewizardon DishwasherBench – Can Frontier Models Stack a Dishwasher?

I would totally do this, is this wrong? It still gets sprayed from the bottom no?

mlukewizardon DishwasherBench – Can Frontier Models Stack a Dishwasher?

Mmmm yea interesting, thanks for the feedback will push an update :)

mlukewizardon DishwasherBench – Can Frontier Models Stack a Dishwasher?

100%, users check before kicking off a run whether the drawers close and the spinner spins

mlukewizardon DishwasherBench – Can Frontier Models Stack a Dishwasher?

Question for both: do you think that is also the case with a real-world dishwasher (it's mostly about getting stuff to fit), or do you think it's an issue with the current sim?

The sim needs work, for sure

mlukewizardon DishwasherBench – Can Frontier Models Stack a Dishwasher?

Totally the opposite, there's a detailed dishwasher simulation under the hood: ray tracing, splash simulation, soak, heat, enzyme action

I calibrated it against my actual dishwasher by doing some test runs with baked egg

mlukewizardon DishwasherBench – Can Frontier Models Stack a Dishwasher?

yeaaa perhaps this is the future

mlukewizardon DishwasherBench – Can Frontier Models Stack a Dishwasher?

sure the code isn't hand-written and it's a work in progress, but it's designed to be modelled and calibrated around my own dishwasher so it's verifiably non-slop

mlukewizardon DishwasherBench – Can Frontier Models Stack a Dishwasher?

Inspired by Joma Tech's new YouTube video

Stacked loads are run against a dishwasher simulation calibrated to my own Bosch dishwasher

Playable by both humans and AI

Currently a WIP, simulations need refining before running a full benchmark, but keen to hear some feedback!

mlukewizardon Ask HN: Who is hiring? (September 2024)

Composo | Founding Software Engineer | London, UK ONSITE |

https://composo.ai

Composo is on a mission to ensure that AI transforms our world effectively, rapidly, and safely. Our platform squashes the legal and reputational risk associated with shipping AI apps and improves the helpfulness of our customer’s AI agents.

We believe subject matter experts are critical to evaluating and improving AI applications. Our platform equips them a powerful evaluation toolkit in a no-code interface, while giving engineers the development freedom they need to ship their best code.

We’re looking for generalist full stack software engineers, building up to a team of 5 over the coming months. Stack is Python FastAPI on Postgres with React Typescript frontend.

Full listing here: https://composo.notion.site/Careers-at-Composo-decbf0cd01af4...

Send an email to luke[at]composo[dot]ai with your CV and a note about why Composo excites you. I will reply to all applications sent in September that clearly aren’t from bots.