Yes, if a library is implemented in a language which supports closures, the difference between it and a framework starts to vanish.
Can you name some of these exceptions?
In Flask, you call the framework, then it calls you back etc. It is a minimal framework that you can use as a library along other "plugins" (which are libraries in fact) you like.