On this occasion it was a human stenographer in the back-end, but what was remarkable from is that if you didn't already know that, you'd guess from the error rate (not perfect, with difficulties that made sense in context) and the delay (bounced to another continent and back) that it was something like Whisper running locally. All the more so if you happened to know that one of the organisers has been running local AI meetups.
It seems to me like the only way up from here, now, is to aim for better than the average human stenographer. You'd have to have a system capable of reasoning "ah, what I heard was Y but they must have really meant X" when X involves knowledge that couldn't possibly be predicted from the context, like a foreign name coming up for the first time. I can believe that if the stenographer had more domain-specific knowledge that they wouldn't have made some of the errors they did, but we're already off into capabilities that are as good as an extremely rare type of human today.
We have "smart bulbs" that are connected to this spy network, which means that if I want to turn off a light, if I don't want to be shat on my by my partner I can't just turn off the switch, but rather stand pointlessly trying to turn off/on the lights by yelling at a fucking box in my lounge. I hate this so much more than you could ever begin to imagine. And somehow, all these "success" stories has not yet been enough for him to open his eyes. People will buy any shit and then convince themselves that they have "success" stories.
Lets not even breach the topic of the fact that I have been using computer vision for years to do cool shit, but no, lets buy shit xiaomi cameras that have no API to interface with, so the absolute best case scenario for me knowing what's on the camera, is opening their piece of shit app and waiting 10-30 seconds for them to figure out how to connect over our network using their own fucking API.
Since I'm on this tangent now, I would love to pay special mention to their PIECE OF SHIT KETTLE Which is supposed to be a smart kettle, but the one fucking thing you would want to be able to do remotely is not supported. Oh, you wanted to boil your kettle from your desk? Well Fuck you!
Besides a right to repair, we should also have a right to API.
I mean, I need a permit to add a window to my home. Can we please require permits for mass-dumping semi-useless products onto the market?
Then I looked at the kettle... I have a temperature controlled kettle, it has three buttons and does everything theirs does with an app. Plus it's 2.4kW and holds 1.7L so is better than theirs all around!
For instance, every time you speak to Google, it doesn't just 'guess' what you're saying, rather, it will present you with 3 options of the most closely sounding things for you to pick from. And by picking, you train the model.
edit to add: The whisper paper mentions using RLHF as a possible path to improve accuracy[2]
[1] https://link.springer.com/article/10.1007/s10462-022-10224-2 gives many examples