That's not what the C/C++ spec says people can rely on with assert/NDEBUG. Like don't get me wrong, you can do what you want, but you can and will break code in ways that introduce memory unsafety with this approach
To my mind the extent of the contract is that if the user has set NDEBUG then you can assume that assert will be a noop. I don't think you as a library author can make any assumptions about what happens if you set the flag yourself.
There's no difference between the flag being set/unset by your build system, and #define-ing or #undef-ing the symbol in code