Ask HN: Should GenAI companies be audited for their use of customer data?
The GenAI industry is facing a significant challenge: widespread mistrust between buyers and their GenAI vendors. There is deep-seated fear that vendors will exploit their buyer’s proprietary data to train or fine-tune their models.
In many cases, the actions of established companies have perpetuated this fear. For example, Slack's privacy policy update granted it permission to use customer data to train its AI models without requiring explicit consent.
Several factors contribute to this mistrust:
1. Vague descriptions of how customer data is used in the Terms of Service, which may give vendors undue leeway 2. Concerns that established terms will not be fully respected 3. Terms that are often changed without prior notice
As a result of these concerns, buyers are taking various approaches to protect their intellectual property when making buying decisions for GenAI products:
1. Most don't want to buy from GenAI companies at all 2. Many require on-prem deployment 3. Many insist on strict contractual provisions against using their data for training or fine-tuning 4. Many have developed internal practices to contextualize data before sending it to any AI company
# Problem
The current situation appears to be a lose-lose for both buyers and vendors. Buyers are unable to take advantage of potentially transformative AI technology, while vendors struggle to close deals due to a lack of trust.
# Solution
One potential solution is to require independent audits of GenAI systems. Each audit would focus on verifying compliance with specific contractual requirements, such as:
1. Ensuring customer data is not used for fine-tuning 2. Confirming customer data is deleted after its intended use
# Ask
If you are responsible for making buying decisions in your company, will an independent audit report that verifies compliance with contractual requirements assuage your IP-related concerns?