The difference between "X Testcases" and "Y Free Writes" per month is a bit confusing, but I think I understand what you mean.

My main concern would be that managing test failures (is it a real bug, or is the test just fragile to minor app changes) would be just as much work as having the tests in-house. Probably more work, since I can't just look at the code. Your reporting/logs would have to be excellent and very detailed.