How do you train a model to drive with LiDAR when the human drivers who generate the training data don’t use LiDAR?
Under that model, LIDAR training data is easy to generate. Create situations in a lab or take recordings from real drives, label them with the high-level information contained in them and train your models to extract it. Making use of that information is the next step but doesn't fundamentally change with your sensor choice, apart from the amount of information available at different speeds, distances and driving conditions
put the car in a video game and raytrace what the lidar would see