I’ve only ever used pybind11 for C++ -> Python interface and must say, it seems pretty good.
That was exactly my experience writing custom GNU Radio blocks for my master's thesis: swig mangling my C++ variables. I have nothing positive to say about it.