You inevitably run the risk that your deserializer can, effectively, make arbitrary function calls. And your data is tied not only to your language, but also needs to find classes in specific modules.
Edit: I guess you mean the deserializer is running arbitrary code to import the necessary classes if they are not already imported. I would prefer that it fail in this case. I think it's not too much trouble to ask users who insist on using classes to have them around before instantiating them.
You're right, some languages are worse about this, e.g. Python's pickle will perform arbitrary code executions.
But it's easy enough to restrict it to a set of known good constructors. You still have the classic late-binding dilemma: it's flexibility we don't actually want. Why should my find_matching_socks function ever have to worry that it might be passed basketballs?
The usual answer is, "it's fine, it'll just crash if they do that."
It might crash, or it might hang, or silently corrupt something. An attacker can stick the wrong kind of object into your code anywhere they like.