1. Record video sound etc... (trajectories) egocentrically
2. Analyze the data and assign reward labels (more/good, less/bad) to state and transitions actions
3. Use the reward feedback and trajectories to build the policy for some set of actions in certain environments
This is why I'm bullish on anything sousveillance - so AR cameras on your head, always on mics etc...
The challenge is doing this democratically, without it being intermediated by a giant for-profit mega corp that doesn't care about you and wants to mess with your head