Is it possible to generate this automatically using conformance tests?
You're not going to find some hobbyist coming up with a useful conformance or coverage test suite for free. It's intensive, expensive, and arduous.
Such things are mandatory, however, if you are a C++ vendor targeting ISO 26262 or IEC 61508 or similar -- basically, software that could kill you if it's done wrong. It's a niche with money and motivation.