The LLM right now tells a much lower level neural network controlling the legs, arms and head what and how to move at a ~ 500Hz freq, which controls thing like the stumbling. All motor joints and IMU inputs are given to an ONNX-based RL-trained NN which then outputs motor movements (scary since its works well for the scenarios where its has been trained, but it goes in unexpected ways for untrained scenarios).
In any case latency is acceptable for navigation I guess, but avoiding moving obstacles is definitely a no-go right now.
The one in that video is the G1, which is a bit taller and comes with more sensors, but there's a trick and it is the dexterous hands, which can easily cost 10k each.
Based on my very limited knowledge... if you would have dexterous hands with sensitivity on them you could theoretically train a VLA model to do that task, planned by a high-level model like Argon 4, that orchestrates everything. That would as well require in first place a VLA that has similar tasks in its training set and fine-tune it by means of recording hundreds of examples performing teleoperation of the robot hand and arms movements.
Not easy, but definitely getting more and more doable these days.
PS: Didn't spend 20k on hands for my 6k robot, but can confirm it has the cable harness to support them :D
As a tip from an owner... if you are planning to play around, maybe even the robot dog (Go2) can be a good start and comes with a LIDAR preinstalled. When humanoid robots enter into zero-torque / damping mode, it's 30 kg falling to the floor. I know it might sound stupid, but I didn't realize how heavy and scary this is, at least for me, once you have it in front of you.
It's 1.23 and weights ~30kg. Even though only the R1 Edu can be in theory programmed, it can be easily modded to enable SSH and root access to deploy custom movement policies and interact with the different on-board daemons to control the movements, grab the video data, etc.
I bought a second-hand Intel DeepSense camera and plugged it on one of the spare USB ports available in the main board to have 3D sensing (even though I really need to play a bit more with it). Right now I am playing with MuJoCo for RL training of new policies, integration with VLAs like NVIDIA Isaac GR00T and integration with LLMs.
I have to say its being a wild ride, and despite the small size of the robot, I have already injured myself with it when playing with the walking policies (If you activate a policy while its hanging, it starts literally kicking and punching you...) I cannot image those owning with the bigger brother of it, the G1...
I am very new to robots and recently decided to go crazy and buy a humanoid from Unitree. I've spent a while building some tools in order to integrate LLMs with vision capabilities that can control the robot motion in a simple manner... and while I saw Opus and Astra crashing against tables, beds and surfaces (yeah, my LIDAR data isn't being used yet), Argon 4 takes a lot of precautions to avoid crashing against walls and other objects! I told my robot to go to a table, and the guy started walking slowly to the table, started steering at it and described the objects in it. Pretty scary for being something I've vibecoded in a matter of hours at home...
I was waiting for Apple to release something privacy-focused and instead they have decided not to release in Europe because 'it lacks AI'. You know what, all yours, I guess I will just buy an old Alexa and hack it.
I have a Mate 80 RS Ultimate Design and I think it looks beautiful in person, but as you know, Chinese and Western design preferences are very different, just look at their EV's (and how western brands are copying their designs)
I've spent a few months trying to convince my management on getting a Macbook after fighting with a kernel memory leak that consumes the 128Gb of RAM in my freshly new Dell Pro Max 16 with Windows 11. It seems the best solution my IT gives me is to replace my laptop... which I have already done... three times.
I think this is a good chance to try and port Alyx (ARM64 build) to visionOS and run it on my Vision Pro headset which has been running around unused for the last two years xD
> What modern LLMs do is apply brute-force computation to any domain expressible in language--not just a restricted chess domain. That's the algorithm.
Which means that companies with sufficient computational resources and money will be capable of unlocking problems thousands of times faster and more effective than any individual even when lacking the skills, just by a matter of try and error.
I might be a weirdo, but even the mail client on iOS has serious deficiencies. Just getting an email with a zip attachment where I have to edit / sign a document with a certificate / something, rezip and send back is a hell in iOS, while it simply feels natural in Android.
What's the value of using one of these phones running iOS? iOS is great for media consumption and not so much for productivity... so I don't understand very well the use cases... apart from watching Netflix I guess?
They had other models a while ago, named 'Pulsar', and they were finetunes of Nemotron in collaboration with NVIDIA. I think this will be something similar.
I mean, I think it depends. At home 3 of us we use AI for multiple reasons, from coding apps to asking general questions, and if we would have to pay equivalent subscriptions that would be ~1k a year on AI + submitting all your data to external services. I payed around ~8k on 2 DGX Sparks that, at the moment, serves perfectly fine as a ChatGPT/Claude replacement at home (DS4 Flash peaking at ~170 tokens per sec with 6 concurrent sequences), and even once the technology is obsolete for inference in a few years, I will still have 2 pretty powerful machines for whatever I need + some pretty fast NVME Storage. I don't think its a terribly bad idea.
Not really, I just made a really simple Python library the agent can use and modify as he wishes within a sandboxed environment to explore the SQLite database. When the user prompts the agent, the prompt goes into my coding agent, omp, and starts running Python scripts and throwing SQL sentences until coming up with the answer. It works extremely well in our experience.
Its a read-only SQLite file. And I mean the people using the chatbot knows it uses AI so just like Google they shouldnt pick the first result, but forcing the LLM to mention the sources and not assume acronyms works amazingly well
I have a web user interface connected to a coding agent (OMP) running inside a container, so if any of the tool calls fail or something happens, usually my model recovers autonomously from these situations. The capabilities of models like DS4 Flash are those of frontier models from months ago, so its recovery and autonomous capabilities are quite impressive.
I indexed thousands of documents into a SQLite database with an FTS5 index, plugged it into DeepSeek v4 Flash and got better much better results than any other commercial solutions my company has tried in the past.
The trick was just to let the LLM come up with its own SQL queries for searching... and the results are impressive.
I've been reverse engineering LEON3-FT SPARC v8 BE code, so I wouldn't say it's common :D. When attached to Ghidra through a MCP the things you can do with this are simply crazy.
I am surprised. I've been using DS4 Flash (0731) for weeks now and it works perfectly fine as a replacement for Claude in a large variety of cases. It requires a few more iterations, sure, but it's useful enough to not need a Claude subscription anymore. Among the things I do I've been reverse engineering, writing complex C++ code...