No, it's not. Don't do that. Someone (like me) will replace assert.h wholesale and it will not behave the way you expected. Always define your own internal macro and always provide facilities for the user to override it.
There's a whole host of things that libraries need to be extremely conservative about touching. Assert, threads, signals, etc.
To my mind the extent of the contract is that if the user has set NDEBUG then you can assume that assert will be a noop. I don't think you as a library author can make any assumptions about what happens if you set the flag yourself.
There's no difference between the flag being set/unset by your build system, and #define-ing or #undef-ing the symbol in code
that kinda sounds even more insane and cumbersome
And it seems completely sensible that if I'm writing a C++ library that it's header files should include assert.h, and use assert()'s in its inline functions. And you will get no sympathy from me if you break my code by redefining the library behavior of assert().