I think having a prefix (be that "betty" "b" and/or some "mode" for your shell) that signals you want to use natural language (with all it's ambiguity) is a good idea. The zsh way of suggesting "did you mean" rather than simply erroring with "command not found/invalid syntax" drives me nuts -- but a lot of people seem like it. Having a prefix allows you more freedom at "learning" -- ambiguity isn't so terrible if the user expects it (and the simple idea of just listing alternatives that betty uses seems like a great interface. Not as sexy as "I'm feeling lucky"-style (super-)high scoring wins, middling scores ties and asks user to pick -- but I think it may win on the principle of least surprise).

Anyway, it does seem that most proof-of-concept voice-control (as opposed to text-controlled) systems use a prefix too "siri"/"glass"/<microsoft had one, can't remember which, also they have "xbox". The idea is that if the mic is always on, you don't want your drones to blow something up, just because you jokingly told a friend "kill it with fire" in a voice call. Context is hard to get right for such systems, I expect the kinetic and similar systems can do better (if user looks at computer, listen. If user is already speaking "in conversation with" to computer, listen. Otherwise ignore, unless user asks for computer by name).

As for you question, I think it should be relatively easy to train, say a music-player app to understand stuff like "next song", "accept call", "repeat" -- in any language, using simple statistical methods. Not sure how far you could take it though (example, dictation software still makes (AFAIK) enough errors that it's not really a viable option if the user already can type reasonably well (or hire an actual stenographer)).