AI Accent Conversion for call centers
krisp.ai
krisp.ai
I am not in love with the idea that the fate of AI is to boil down each person to a lowest common denominator. I'm curious what others think, at what point between totally unprocessed source material and everyone sounding like the same four marketable AI voices does a solution like this become problematic? I get the intention of the product, appreciate the focus on clear communication, but can't help feeling some will take it further and I just don't like the subtext of forcing people to sound like some core "good" voice. What will be lost in that?
Clear communication is a laudable goal. But I fear what might be lost in the ruthless pursuit of it.
That argument can be applied to practically everything, and so is invalid. It does not serve you or any of us other humans to dehumanize other humans, which btw you are in the set of other humans.
People at work performing a function are still people, not functions.
This will simply enable more people to work there.
And I wasn't charging you personally, I was talking about a range of things where this is only one example. Like talking on the phone while ordering something at a counter, as though the person behind the register is no more than a part of the register.
Saying that communication is important does not magic away this problem. Yes, we get it, you want what you want more than any other consideration, how deep and thoughtful.
Are you kidding, that's the holy grail! When I need customer service I want every interaction to be by the book, zero personality, from them or from me.
The problem is it's not right now: when somebody with a non-Western English accent answers a call to a Western company (Airbnb, your bank, etc) it's pretty much a guarantee that you're about to deal with massive incompetence and apathy. You don't notice this until you live long-term in a homogenous country, where the people answering customer service calls are the same people you see on the bus/train after work. It's like a lightning blast into your brain: "holy cow, customer service actually works here?".
And you might say, "well incompetence and apathy is all call center people who work for American companies are empowered to give you, regardless of the accent or geography".
But that's not true: Filipino/a call center employees are gushing with upbeat script-following good humor, but they never solve your problem.
Whereas "Dave" from the midwest sighs and rolls his eyes before taking every call but he actually figures out the reason you're calling and uses his brain to try to resolve it, in the boundaries he's empowered within.
That's not a popular thing to think or say, but I know I'm not the only one who, after literally hundreds of these interactions, has developed this heuristic.
And you might say, "well 'Dave' from the midwest is by definition higher up the ladder and more empowered to solve your problem, because otherwise you'd be getting someone in the Philippines/India who's only allowed to follow their script".
Sure. But that's why it'd be great if there was one common-denominator: anybody who answers your call has as much ability to resolve issues as anybody else, tries as hard as anybody else, communicates as clearly as anybody else... that's the holy grail.
Would you have a problem with using full translation?
The difference here is that the (accent) translator doesn't have to do as much work.
Also it should work both ways: foreign language/voices can be converted to local Indian languages/accents too.
I have had a mix of colleagues with diverse backgrounds, none of them were particularly challenging to understand except for one native chinese speaker. In the beginning, we have indirectly asked for clarifications but stopped doing that after someone candidly mentioned that there's risk of offending the other person. Now we just send meeting minutes to avoid misunderstandings.
That theory always intuitively made sense for me, so I never questioned it. But of course a lot of theories like this are repeated precisely because they sound plausible, regardless of the actual truth behind them.
Of course scammers aren't literally looking for people trying to be scammed, so the comparison breaks down there. Plus of course there are other strategies than just spray-and-pray (where lower upfront costs by not investing in good copy or plausible design may actually be a good strategy because you can't handle too many marks at once) and with many of those better quality investment definitely increases the chance of success (e.g. spear phishing or whaling).
Indian English on the other hand is of course about India, Pakistan, Bangladesh, etc.
It's pretty obvious what this product is for and who the target market is. Companies don't outsource their call centers to countries where these varieties of English are spoken by the majority of English speakers because they are looking for the best talents.
Any company anywhere always has to look for the best talent at the best price.
Otherwise, you'd only hire Nobel laureates, and go bankrupt immediately.
Surely you understand the difference between wanting to hire superstars but then adjusting your targets downward based on what salaries you can offer versus wanting to pay as little as possible for call center agents and then adjusting your targets upward based on what you can get away with. "I want the best I can get but I need to be able to afford it" vs "I want to pay nothing but I need to spend enough to get something that works".
Expectations for customer service tend towards rock bottom for many businesses. It's something they have to provide or their customers will get very upset (or they might even get them into legal trouble) but it only has to be good enough to be serviceable. And for call centers this usually means you end up having people closely follow a script anyway so you're literally just paying people to be human dialog trees. It's a nuisance but you can't get away with not having it, so you want to pay as little as possible.
You're talking about cut-offs for price ranges - that's indeed a given which is why I thought it doesn't bear mentioning. I'm talking about whether you go over the resulting list sorting from lowest to highest ("best of the cheapest") or highest to lowest ("cheapest of the best").
Sure, different companies run different business models.
There's space for both Singapore Airlines and RyanAir in the world.
You should pick what your budget and preferences agree with. No need to be judgemental.
(I know they don't do this, but it would be funny if it converted the customer accent back into the agent's accent)
There absolutely is still a local language influence.
I think that the accent that is most closest to is American.
Apart from that, it also raises some ethical question. Your voice, the tone and its expression, is probably one of the most important features that define you as a person. I cannot imagine anybody being psyched about their voice being changed in a way that's out of their control.
Accent is rarely the whole issue in dealing with non-native language call centres, it's just one part of the language skills package.
- It's not going to help if the caller isn't very good with the callee's language. - It's not going to help the caller understand the callee any better. - In effect, I feel the only language barrier it's helping to overcome, is the callee's feelings towards the caller - although I'm sure that is of some help.
(It maybe doesn't help their case that their test cases use people with reasonably good english language skills anyway).
Scammers already use the relative of the target's voice at times.
The company should consider targeting RP instead of General American. In my experience attempting the latter with a slight Asian tinge makes you sound like you're trying to be a WASP, while speaking the former with the same small errors makes you sound like you studied PPE at Oxford.
Do they have software for the reverse case? Can it give someone an Indian accent, for example?
For the later, where it is an obstacle to communicating a tool like this seems reasonable. It's not that different from a translation tool across languages.
Now, with LLMs, voice morphing and custom voices will be possible at much more reasonable price points and you might not have to spend hours in the studio recording piles of words that contain all triphones you need to sample.
in professional settings, Filipinos are known for a lack of accent. they are more known for mispronouncing words that are commonly accepted in Filipino-English.
i feel like it should be the other way around. accents should be converted FOR the caller or call center.
callers can be trained to speak a certain way. but it's more difficult for them to develop the "ear" to understand accents from other nationals.
There's nothing wrong with the "Indian accent". It's not challenging. It's Indian English. There's nothing wrong with Phillipine English either. Those are just different regional varieties of English and people who speak them are often native speakers. These are native languages in their countries and they were introduced at the barrel of a gun. We don't call American accents "challenging" to speakers of other varieties of English either - the only reason other native speakers genuinely tend to not find the General American accent challenging is that they grew up on a diet of American media, there's nothing special about the accent itself.
As a European, I do find Indian English more difficult to listen to. But that's because I have less exposure to it (or rather had less exposure in my formative years - it's omnipresent if you work in tech these days). But that doesn't mean there's anything wrong with their variety of English or "accent".
The reason this product exists isn't to aid communication or "break down barriers", it's to allow American companies to outsource call center jobs to lower wage countries without having their customers complain about being exposed to foreign accents when trying to access customer service. This solves one superficial "problem" (at least if it worked better than in the demo and actually masked the accent completely rather than "fixing" some of the "challenges") without actually improving service for customers. It merely reduces some of the friction introduced by purely profit-seeking cost cutting measures.
Not to mention that because this literally targets the bottom feeders who want to mask the fact they outsourced their call centers, this will also enable scammers to create even more plausible fake personas (now that most elderly people have wisened up to be suspicious when "Jim from Chase Bank" has a thick "Indian accent").
EDIT: "Notice how my natural tone and personality shine through. This mode allows me to stay authentic, making every interaction more personal" she says while reading out a prewritten script word-by-word. It's not about maintaining authenticity or personality, it's only about still appearing as humans so customers don't think you've gone all the way and just replaced these jobs with AI entirely (and maybe feel bad about yelling at them when they get upset with your products or services).
At least this might make it easier to understand them.
This is terrible.
There's nothing wrong with Polish either, but a call centre agent speaking Polish wouldn't be very useful (for all but a handful of people on earth).
So I'm not sure what your point is?
> It merely reduces some of the friction introduced by purely profit-seeking cost cutting measures.
Reducing friction is good! Cost cutting is good, too. Seeking profits is normal for companies. Otherwise, why start a company at all?
It's usually said in a way that suggests or even outright claims to be to the benefit of society as a whole but ultimately it ends up just meaning redistributing market shares or outright draining more money from consumers (e.g. by "disrupting" public or shared infrastructure). And more often than not a huge value proposition relies on abusing externalities (i.e. the "public good" or commons) without paying back into them because the social contract that used to maintain them is not actually legally binding.
And yet drivers are concerned about black ice. Maybe friction is good in some cases and bad in others? It almost feels like blanket statements are unhelpful and whether something is good or bad are relative statements, i.e. something is good or bad for something not just inherently and universally so.
For example, labor protection laws are bad for companies that want to be able to rapidly build up a workforce without having to commit to keeping it around in case they need to cut costs, but they're good for employees who want to be able to start looking for new jobs when they're laid off rather than having to desperately find a new job after immediately losing their income.
Likewise, having to pay fines for dumping your industrial waste in a nearby body of water is likewise bad for companies that otherwise have to pay a lot of money for someone to pick it up and deal with it for them but good for anyone dependent on that body of water not to be contaminated.
Both of these are examples of regulation creating friction that gets in the way of a business optimizing for its profits. And in both cases whether you think the friction is good or bad likely depends on your personal beliefs and value system.
> Cost cutting is good, too.
Again, "good" is relative. It's good for increasing profit, yes, by definition.
But another way to frame the "cost" of a business is to look at it as its output. A company with almost no costs (and no, not paying its owner a salary because he can get by on the share price or pays out "only" the profits to himself doesn't qualify) is effectively just collecting money for zero output.
Surely you wouldn't argue that landlordism (excluding actual building management, maintenance, upkeep, janitorial work, etc, which are often lumped in with it by its defenders) is the best (in the sense of "desirable from a society's point of view" not "optimal for profit") business model because its income exists almost entirely on mere ownership and does not require any actual labor to generate income (i.e. 100% profit after taxes)?
Ultimately unless a company just accumulates its profits and sits on them (whether that means leaving it in the bank or moving them into financial products), the balance sheet will always ultimately zero out. Any unused capital is ultimately unused potential to generate more revenue so if companies are naturally profit-seeking they must be naturally averse to leaving capital unused.
This means that if we talk about "cutting costs", we're not talking about leaving that money unused, we're talking about using it for something else. Cost-cutting is not reduction, it's redistribution. The question then is where does the money that would otherwise have gone to the costs that were cut actually go. That answer thus determines whether "cost cutting is good", ultimately, and even then it's begging the question of "good for whom".
If I run a company and cut back on employee benefits, that's "cutting costs" and it's clearly bad for my employees but whether that's good or bad also depends on what I then do with that money instead. If I simply treat that money as profit and pay it out in dividends to shareholders or to me as a bonus, that's good for them (or me) but does nothing for anyone else. If I reinvest it within the company instead, it may be good for others and could even outweigh the bad of cutting their benefits for the employees (e.g. even just financially if the money that went into providing those benefits can be used to put them all into positions where they don't need them or can organize them themselves while still ending up with a net gain).
> Seeking profits is normal for companies. Otherwise, why start a company at all?
Eating your face is normal for lions[0] too but for some reason you don't see as many people constantly defending lions eating people's faces.
If the entire point of having companies is to seek profits at the expense of the benefit of other people that makes it sound like companies are a net negative to society. I'm sure you'd disagree with that position, so I'm not sure why you'd offer it.
[0]: I guess bears are the better analogy because they're literally known to go for the soft parts - i.e. groin, face and stomach - without even bothering to kill their prey, but this is a reference to the "Lions Eating People's Faces Party"[1] so please apologize the zoological inaccuracies.
[1]: "I didn't expect the lions to eat my face", says long-time supporter of the Lions Eating People's Faces Party after their election success.
I can sorta see the "dehumanizing" aspect from a philosophical perspective, but honestly there are so many other practical, day-to-day dehumanizing aspects to these jobs that need fixing, that this fades to nothing in comparison. For eg. maybe this will reduce the need for the "accent training" classes that employees in these centres go through, which is more dehumanizing and also cuts into their already meagre paycheck a lot of the time.
"Accent training" is indeed more dehumanizing but its framing usually relies on the same premise (i.e. that your native variety is deficient and needs to be "fixed" rather than that you're literally learning a foreign accent to play the role of an American) and it exists for the same reason (i.e. American companies trying to cut costs without making it too readily apparent to their US customers). It's not about "more efficient communication" as the video has her say, it's about play-acting an American to mask the fact the company outsourced the work (or at least not make it obvious enough that customers are actively reminded of that fact).
This isn't jingoism - I'm not even American myself. But the framing of what this is for and why it exists is not just dishonest. You can argue that of course when they have her say her "accent" is "challenging" it's implied that she means "challenging to US customers" but that's not what she says and it's not just an "accent" the way a non-native speaker may carry over an accent from their native language so you may equally say it's implied that the way she speaks English is inherently deficient because it's a foreign variety.
This is a product to help American companies reduce friction from their customers when outsourcing call centers to lower wage countries - not even India and the Phillipines likely but other countries where those or similar varieties of English are widespread, especially as poverty in those two countries decreases. This technology wasn't created to help Indian English speakers or immigrants overcome language barriers in the US. This won't be used to allow call center workers to avoid "accent training". This was created to allow these companies to outsource to even cheaper call centers.
Sure, "accent training" may have cut into the already meagre paycheck but if those call centers were able to offload those costs onto their workers before that means they can now justify simply paying them less - and after all, someone will likely argue that that's still a good thing because maybe those workers still end up with less money than if they hadn't "needed" the "accent training" to begin with but at least now they don't actually have to spend the time and energy on those "trainings" as well.
This statement, I'm afraid, is false. "But that doesn't mean there's anything wrong with their variety of English or "accent"."
Yes, there IS a lot of wrong with my accent. I am not a native speaker and thus there is no leniency with that. If I have grammar issues, or my accent is heavy, that does not mean there's nothing wrong - on the contrary, it means my knowledge of English is not good enough. If I were a native speaker? Then for sure, by definition I could not have a wrong accent (whether it's understandable is a different question.)
> If I were a native speaker? Then for sure, by definition I could not have a wrong accent
On an everyday basis I read/write/speak/listen to more English than my native language. Let's say Bob is born and brought up in a region where English is the dominant language, but not every English speaker would understand Bob flawlessly.
Now, why is my semi-understandable accent less legitimate than Bob's semi-understandable accent, just because Bob is a native speaker?
You're Polish. There aren't many places outside of Poland where Polish is a native language - I'm not sure if there are any at all if you narrow it down to places where it is an official or majority language.
I was talking about Indian English and Phillipine English. Indians and Filipinos often speak those varieties of English natively. The reason they exist is literally because the British Empire brought the English language with it when it invaded them and they had to learn it to adapt to their conquerors. But that was a long time ago and now you have literally generations of native speakers of those varieties of English just as you do in the US, Ireland, Canada, South Africa or New Zealand[0].
This is different from a Polish non-native speaker of English or German, or a non-native speaker of Polish having "an accent".
[0]: I'm intentionally omitting Australia because Australian English seems to have a more homogeneous history, being mostly the product of different British varieties of English. New Zealand English shares a similar history but is distinct because it also took influences from its indigenous Māori language. Canada and the US of course can trace their varieties back to British settlers but likewise saw many influences from other settlers, slaves, indigeneous peoples and later immigrants. South African English again goes back to British settlers but includes varieties natively spoken by members of the majority of its population unrelated to those settlers. Ireland's native population - as in India and the Phillipines - adopted its variety of English due to British conquest.
So what would be the point? Maybe to disassociate from Indian scammers. Maybe to enhance trust by fooling people into thinking that they are talking to a person from a similar culture (the AI doesn’t sound not Indian by the way though) them instead of an outsourced worker on the other side of the world.