HNHacker News
TopNewBestAskShowJobs

jwyang

36 karma · joined October 24, 2023

submissionscomments
jwyang··on Magma: A foundation model for multimodal AI agents
very good question, now we are mainly focusing on building the foundtion for multimodal perception and atomic action taking. Of course, integrating the trace-of-mark prediction for robotics and human video data enhances the model's medium length reasoning but this is not sufficient for sure. The current Magma model will serve as the basis for our next step, i.e., longer horizong reasoning and planning! We are exactly looking at this part for our next version of Magma!
jwyang··on Magma: A foundation model for multimodal AI agents
ops, a typo, no M from Microsoft.
jwyang··on Magma: A foundation model for multimodal AI agents
Thanks for your great interests on our Magma work, everyone!

We will gradually roll out the inference/training/evaluation/data preprocessing code on our codebase: https://github.com/microsoft/Magma, and this will be finished by next Tuesday. Stay tunned!

jwyang··on Magma: A foundation model for multimodal AI agents
Good catch! A minor correction: Magma - M(ultimodal) Ag(entic) M(odel) at M(icrosoft) (Rese)A(rch), the last part is similar to how the name Llama came out, :)
jwyang··on Set-of-Mark Prompting Unleashes Extraordinary Visual Grounding in GPT-4Vision
Here are other related links:

code: https://github.com/microsoft/SoM

demo: https://user-images.githubusercontent.com/3894247/281586831-...