i think the most challenging aspect of the idea is interacting with the world, picking up and handling various objects.
its obvious from the presentation that fsd is very good at placing itself in space and mapping out its environment as well as devising routes even when accounting for things like other moving objects. boston dynamics has built machines that take the input of a joystick and translate that into four-legged locomotion. so if you just took FSD and bolted that on top of a spot or atlas, you would have a machine that gets you a very long way towards tesla bot. something that can walk and navigate by itself. so the question is whether or not tesla will be able to recreate what boston dynamics has done and whether or not they could do that on the time-scale that was insinuated by musk. the last time i checked, boston dynamics does not use any NN in their robots.
and then theres the question of interacting with the world. i think its a fair assumption based on ai day that fsd would be able to create a very rich and accurate map of its surroundings. so that half of the question is taken care of. but how could they get it to grasp a drill and manipulate it in an intelligent way? the implication is that they would train a NN to provide that signal. but how are you going to train that? they have thousands of cars out in the world generating tons of driving data. where are they going to get data for performing these tasks? are you going to train it on every task that might be asked of it? its hypothetically possible but thats hard data to generate and label. it would take a really long time at best, and even then it would be an experiment, just as likely to fail as succeed.
it seems to me that this must be a stunt where the intention is to build something that can walk around but not manipulate things or do anything useful. i would change my mind if musk came out with an explanation of how hes going to train interaction.