Ask HN: Building a Cocktail-Mixing RoboHand – Libraries, & ML Recommendations?
Hey HN,
I have a project idea that I'm really excited about and looking for some advice. I recently got my hands on the Adeept 5-DOF Robot Arm Kit. It's an Arduino-compatible, 5-axis robotic arm. The specs and tutorials seem comprehensive, but I want to take it a step further.
I'm envisioning a 'Cocktail RoboHand': a robot arm that can identify ingredients and mix cocktails based on image recognition.
What I have:
Adeept 5-DOF Robot Arm (Arduino IDE compatible): https://www.amazon.com/Adeept-Robotic-Compatible-Programmable-Processing/dp/B0BV9ZCCMY/ref=sr_1_6?keywords=robotic%2Bhand%2Bkit&qid=1694464684&sr=8-6&th=1
A 2/4-cores ARM64 SBC (single board computer) for image processing.
Image Recognition: I believe I need some kind of image recognition system to identify various ingredients (like bottles of spirits, mixers, fruits, etc.). What are the best machine learning algorithms or frameworks to get started with this?
Integration with Robot Arm: Has anyone integrated image processing with the Arduino ecosystem? Any tips, libraries, or pitfalls to watch out for?
Learning Algorithms: To make the RoboHand efficient in making cocktails, I believe it'd benefit from some machine learning to understand measurements, pouring times, and sequences for optimal mixing. Recommendations?
Safety: Mixing drinks involve glass, and liquids which can get messy. Any suggestions on ensuring safe operation and avoiding accidents? Especially given that some bottles can be quite heavy.
UI/UX: Lastly, thinking of how users might select or input their desired drink. I think that voice interface + gpt3.5/claude API should be good in order to communicate and create task for the robot. Don't you?
(I know it's a long shot, but if anyone has done something similar or has experience in robotic bartending, it'd be awesome to connect!)
Thanks in advance!