I ran YOLO on an FPGA for my bachlor's project degree project like 7 years ago, but this is even more interesting, and I think we can push bigger models with weight streaming. I still have an FPGA lying around.
https://www.mikeayles.com/blog/on-chip-llm-kv260/
Its right up against the wall of DDR bandwidth, so I'm trying every trick in the book to get the speed up