Thank you for your feedback. This has been the biggest concern raised by 100's users during our survey. We built our product based on the below key principles.
Part 1: Policing causal speech?
We are not a Police. We named our app Dost (means a friend) for a reason.
1. This app does not work on 1:1 messages. It only works on channels where it is configured.
2. Channel owners / users can delete the app from their channels if they like.
3. Only the sender gets the flag, discreetly. No one else, including the admin knows that a message was flagged.
4. No user information is accessed or stored.
5. No messages are stored in our systems. So, there is no mechanism to either trace the flagged message or user credentials.
6. On Slack, the sender gets a flag, but the decision to edit is with the sender. The app does not decide anything.
7. The tool is used to educate the sender about unconscious biases that could cause harm, and learn. Companies spend a lot of time on one-off training on appropriate language and issue guidelines already.
8. We have employed a reputed third party company to do a detailed audit of the above principles, through DB review, code review and application testing. This report is available to be shared with companies on demand.
Part II: Being flagged uncivil incorrectly?
I do want to call out the distinction between flagging a message vs flagging a person. Our app only flags the message sent and not the user. Absolutely no user details are accessed. In addition:
1. The admin or anyone in the company has no mechanism to access user details or the flagged messages, because, they are not stored.
2. We tested our AI against millions of conversations, and our accuracy in detecting issues is >98%.
3. The issues where the app does a good job today are toxicity, insult, identity attack, gender microaggressions only. We are working hard to add new categories of issues in future, only when they meet the high accuracy threshold.
4. When a message is incorrectly flagged as uncivil, the user can just ignore the flag. The cost of incorrect detection is inconvenience. But, we believe that there is a greater good in learning from true positive flags.
4. We are building a feedback mechanism for users to tell us when we do a bad job.
I have done my best to answer your questions. Please let me know if you would like to know more on how we built the AI and our tenets.
Thank you.