This sounds great! We actually discussed doing this at the very start of CircleCI (we had a partnership with an exception handling service, but we never executed on it). Coincidentally, my current company, Dark (https://darklang.com) is based around a similar concept -- using live traffic as an assistant as you're writing code.