If you got a low-level pointer out of the smart pointer container in order to pass to a foreign library, then you're not strictly using smart pointers. You violated the encapsulation in order to get a "dumb" pointer out, and the issue is related to that.
This is a complete strawman.
"Don't use a garbage collected, managed languages, they are a myth. Why? Because a pointer to some garbage-collected object ends up in foreign libraries written in C, which can hang on to it past reclamation ..."
Well, gee whiz, you will just have to learn about the exact sharing situation with the foreign code (how long does it retain the object) and do what is necessary.
> Make them raw pointers and delete them when you know it is safe.
That's completely stupid. If you know when it's safe, it means you must know when the third party library is no longer using them. If you know that, you work that knowledge into the smart pointer solution.
Smart pointers, by themselves, have their own idea of when it is safe to delete something. That idea is very clear and obvious. If it doesn't match the reality of what is safe when, then you have to augment the logic. That's it.
If a foreign library directly accesses an object that you track with a smart pointer, you can create a proxy object representing that library, and that proxy object can hold on to a smart pointer on behalf of the foreign library.
The complexity of dealing with the foreign library could be entirely minor (and self-contained) compared to the overall resource management problem of that smart pointer class; you could be making a lot of unnecessary work for yourself (and future bugs) by uprooting the smart pointers in favor of dumb pointers.
What are you going to do if part of the reason the smart pointers are there is that they are providing needed exception safety?