Using LLMs to estimate regulatory cost compliance

The key graphs:

As an alternative approach, I tested whether large language models (LLMs) could provide more realistic estimates by simulating compliance scenarios in the final section of this post. I prompted ChatGPT, Claude, and Grok to act as compliance officers at companies subject to new CCPA provisions and a Bureau of Industry and Security (BIS) rule, asking each to estimate hours needed for first-year implementation and ongoing compliance.

The big takeaways:

* For California's risk assessment regulation, Claude and Grok project 400-580 hours will be needed for the first-year of compliance (vs. the official 120 hours) and 150-240 hours thereafter (vs. the official 18-36 hours annually). ChatGPT estimates the time at 90-250 hours initially and 40-150 hours for each additional year.

* For the automated decision-making provision of the CCPA, Claude and Grok project 450-730 hours for first-year compliance, far exceeding the official 360-hour estimate. While ChatGPT suggests lower initial costs (80-300 hours), all three LLMs predict significantly higher ongoing annual costs than official projections.

* For the BIS reporting rule, Claude projects 1,140 hours for first-year compliance (3x the official estimate of 333 hours), while ChatGPT estimates 280 hours and Grok projects 380 hours. All three LLMs agree ongoing compliance will require substantial resources.

In short, the LLMs consistently predicted much higher compliance costs than the official sources, suggesting a systematic underestimation.