If I understand what you're saying correctly, you would need to pretty much be medium-to-intermediate at multi-threaded programming + C++.

Capturing screen/webcams and adding extra layers can get really complex. If you can't simulate and easily test it, it's a recipe for disaster.

Using MF API is quite old (relies on DirectX 9 as far as I remember).

C++/COM/multi-media is not such a good match - I always felt like puking when having to deal with COM interfaces.

I would strongly suggest you switch to C# (note: I've programmed 13 years in C++, and the last 9 in C#). There's pitfalls in this too - namely, not sure if/how easy you can easily do multimedia without UWP (Universal Windows Platform). That's a big can of worms, really hard to grasp, big learning curve, but well worth it in time (if you want to see what you can achieve, you can take a quick look at www.phot-awe.com - that's UWP).

Having said all the above - I highly suggest C#/UWP, but DO NOT expect an easy ride. It will take quite a while to be productive, but that will be time well spent.