apeytavin

Happy to read that, and looking forward to that moment!

We simply don't allow that.

If someone wants to train a physical task the data is sent to us for training, so we would not allow this.

And even if you somehow did that, the brain itself has knowledge of what it task is and what to do with it, and since it's running on very smart VLMs trained by the best labs, I expect they have protections naturally (on top of ours)

The coefficient of speeding up is not the same at different moments in the edit and it varies dynamically so I couldn't figure out how to make it change dynamically too. It's also distracting and we want folks to focus on the chaining of difficult tasks for now, not necessarily the speed, which will increase soon but was not our focus (for now).

I think though that it's very clear that it's sped up a lot at some moments. Plus, you can sort of see it at the speed of the messages on the left.

But you're right, for the sake of transparency we'll figure out a way to show it better.

EDIT: For the sake of transparency I can already tell you that it's sometimes sped up to 10x, mostly when it's navigating cause the robot hardware is slow. And by the way, the version we'll ship in February has a much faster drive train.

No YC does 4 batches a year now, and they're smaller batches.

The Fall one just ended, and we were 93 companies presenting at Demo Day (vs ~250 during the summer).

Love your username

The LLM is deciding to use that skill at that time. The skill itself was trained beforehand by the user like shown in the docs, and the agent has information about what kind of context the task can be used in and recognizes that here is the right moment

EDIT: I say LLM but it's really not just an LLM. From the outside perspective it looks like it because we decided to do so, but under the hood it's multiple models (as described in the post)

Yes Burger!

It's split between the bot and the cloud! High-level decision making and training is performed in the cloud for performance reasons, and low-level control and inference of VLAs is performed on the robot which has a GPU onboard!

And yeah it's a powerful architecture, but it's not enough if you want to perform task, the VLMs orchestrate but you need another model for manipulation. And we put all of these together :)

Happy to chat further!

What do you mean by framework? The underlying OS like ROS (that we use)? The algorithms manipulation, navigation, decision-making?

On types of tasks, we're envisioning chores first. So laundry, cleaning, tidying up, dishes etc... That should be the primary focus for this category of robots. But we're very open to other tasks, we know some folks have expressed desire for more interactive kinds of tasks. In another comment I described multiple categories of tasks that folks have expressed interest in. You can find it by searching for "here is what we foresee" on this page

no u

Also big time yes. Many folks would pay big money for a dumb robot that does just 1 or 2 tasks well - and even with that... how good is a Roomba really? It misses places, it gets stuck, and there's still 10M sold every year.

You want a robot that does the task well enough AND for which if it fails it's not a big deal if it's not too often.

For bigger robots that can break stuff it will be too annoying, but for Maurice, I don't think so.

In my experience investors are actually too nice and don't tell you what they think.

I'm European (French), I like folks that are direct and whom with we can discuss

Haha thank you. But really, his message was full of very concrete remarks that help us get better at presenting what we do.

Yes!

We'll have to write a guide to explain how to plugin our AI agent, but it can work on any hardware for which the base controls display the right interface.

If you want to chat more about what you need feel free to join our discord!

My cofounder is 2 meters away from me working on exactly that :)

The first, small platform you see here is a first product but we're aiming bigger

Happy to learn more about what you have in mind, there's a couple different things here it seems?

Maurice is our base machine. We're going to open source the design and hardware so that users can change it - for example switch the gripper with something more complex or a different end-effector. Is that what you meant?

We're also working on a bigger one, which will be able to reach tabletops for example. The idea is as much to create a really powerful robot for the home as showing our brain can work in different bodies.

Ideally our software should power all kinds of robots easily but it needs a lot more training data, and right now it still plugs in standard classical robotics software for some tasks, because it can't control everything immediately. But for example, provided they give us the right interface, iRobot and other Roomba manufacturers should be able to use our AI to make their robots instantly smarter. We can also put it in Unitree dogs (go2 for example). But one thing at at time :)

Thank you, yes there's a lot of positive that can come out of this technology, and it needs to be developed with the help of everyone in order to get there

I'll personally equip our team with EMP weapons when that happens

Added in the FAQ, thank you!

Oh sh*t you're right, the FAQ disappeared during the last redesign!

Another great point you make here, and that I agree with is:

Indeed some of these use-cases are already possible for cheaper and faster and better with other solutions. But each of these requires you to install something new in your home, which is time and money. This platform, and the whole of general-purpose robotics, is about creating a product that will ultimately will do everything well enough that you the marginal gain of using something specialized is not worth the time to install it. And many use-cases like folding laundry or loading the dishwasher are not doable with anything else anyway.

You also make great points on the fact it takes time to make it work, but that's just for the first robot. Once we have enough of these out there and enough data, the time required to do any of these tasks will be much smaller.

It's already quite remarkable that today a consumer can teach an arm to grab a glass with a couple buttons when 3 years ago you would have had to ask a team of engineers to create a complex system to do that. So imagine where we'll be 3 years from now :)

That's some concrete feedback :)

On your example: You can also teach it to fill the glass with water before going to the plant.

And you're right, this is not even close to something the mass public would use, but that's not our goal for now. Right now this is for people who want to make robots, and the value proposition lies in the method to teach it and see what you can make of it. It's our job to make something that learns fast enough that it actually feels useful to our customers.

I take from what you said that the website does not make it clear what the use-cases are, so we'll make an effort on that. Although it's a goal for us to market this clearly right now as a builder's tool, and I don't want to make it seem like we're already at mass-market capability.

I added in another comment a list of use-cases that are already / will be possible, I can't put a link to it though but you can find it by searching on the page for "here is what we foresee" :)

I understand, and trust me it bums me too, but it's just that the compute required to do everything onboard is not there.

Once it is, we'll be able to get rid of it and do everything on edge. In theory it's already possible, but the time it takes to run large models would make the robot look inactive most of the time, and not even remotely reactive enough to handle the real world.

But really, that's our goal

Same for us back then! If I did it today though I would love to try using a RPi 5, these look incredible. But honestly NVIDIA just released their new Jetson Nano Super for 250$ and I think at this point it's a no-brainer to use this instead of an rpi.

Super cool, congrats man!

This is how it started for us too! Check this out: https://x.com/ax_pey/status/1853462975216234851

And like you did, a SAM + VLM is the first thing we tried and it felt high-potential already. It takes a lot of software work to put the right pieces together though, but we think we now ended up with something promising, scalable and extendable for a lot of people.

And on the price: same, our initial prototype was around $250 but I had to connect it to my computer. It's unclear to many others in the field whether we'll be able to offload compute with latency low enough to a computer somewhere else in the house or even in the cloud. In the meantime at least, we decided to have onboard compute so that you can get started quickly. Even for you it would be useful, just because we did the work of putting all the hardware and electronics together, it's a pretty good computer onboard :)

For sure! We see it as the perfect blend between physical and digital abilities, so tasks can be chained together very creatively.

For this small form factor (25cm high), here is what we foresee (and it's a small subset of what's possible).

- Picking up trash and moving it away.

- Tidying up the floor after your kids (moving toys away), so that a roomba could clean - or a version of Maurice with a vacuum.

- Watching for burglars / other surveillance tasks, and sending you an email or notification if it perceives someone or something fishy.

- Greeting you in the morning, waking you up, telling you about your emails.

- Playing with you or your kids: Hide and seek, Tag, Simon Says, Easy board games... Maybe this one is a bit of a stretch at first, but I could see it play Connect Four?

- Taking pictures. I had multiple folks tell me they would like the robot to go around at home events, take pictures, and assemble them / put them somewhere in the cloud after. This should be pretty straightforward to do!

- Check in on your kids or elders frequently, see if they are okay. Especially this second one is very important to us. You could then define a primitive to send you an email or notification is something's wrong. Or a daily recap of the people did, with pictures. And ask the robot remotely to do something.

For many of these use-cases we will post videos in the coming months about how to do them, as the AI and software gets more reliable.

For bigger robots running on the same system we develop (we have a bigger robot to reveal later), that would be:

- Folding laundry, putting dishes in the dishwasher, cleaning countertops, tidying up the place... Chores-type tasks

- Service robot bringing you food and drinks

- Cooking. This one will come once it's safe enough, you want to be careful especially when there's heat involved, or liquids.

Sky is the limit for this technology, but the bottleneck is data. The more data there is, the faster it will learn and the more complex the tasks. We can't promise all of these are possible now, but it will get there faster than people think. You can look at research papers from our labs, like Universal Manipulation Interface or ALOHA, for existing physical use-cases.

It might seem bad but that's actually one of the coolest things about this new approach: it's the core model (today GPT-4o) that decides where it goes.

Here, this was a suboptimal decision by Maurice, and by default we indeed have it avoid making costly mistakes. But consider all the good decisions the agent did otherwise: navigating in all these different rooms with no prior knowledge of where anything is (just pictures it took earlier), close to the glass where Vignesh was, back to Axel, back to bed at the end...

And here's the thing: every time an LLM provider releases a new model, Maurice gets better. We haven't even started fine-tuning the agent yet but that will also improve its decisions a lot. There's many many low hanging fruits to make it able to make better decisions, and we expect that in the coming months the system will quickly get smarter and faster.

The mobile base below the hull is a turtlebot 3 :) The arm is also a modified version of the open source arm by Alexander Koch

We do a lot of it with off-the-shelf components, and we design the whole system so that we can quickly iterate, manufacture and ship from our garage in Palo Alto.

TL;DR: Yes!

You're not the first person to ask me this! If you look closely, there is actually a camera on the arm (that's partly how it learns tasks so fast) and we can use it to take pictures too. You can definitely define a primitive that would be "take picture", another one that is "send to server" and then have your own software assemble it in the way you want. Or just record and send it to the cloud / your computer.

Now if it's about using a better camera that the robot would hold, you'll need to wait for the next generation that we'll reveal later next year

Oh and on changing the personality I believe this could be done separately yes, the important reason why we keep our agent in the cloud is that it's going to improve quickly on decision-making with folks using it, but personality has little to do with that at the end of the day. Even inside the system, the model that decides what to say is separate!