This sounds very interesting for budding platform developers.

One problem is testing the more bespoke vuln routes - that's to say, the buffer overflows, the old package exploits, the strange oddities. By the sounds of it, you are building an "obvious security pitfalls" testing bot, which is in itself very valuable don't get me wrong, but i'm pushed to say that it's an NP problem to test bespoke vuln routes, which means the kind of testing process you are developing (linear-time problem solver) will struggle with testing the non-linear-time problems of bespoke vulns.

However, I wish you luck, hats off to this, it's a great idea!