- They're slow (like really slower than almost any other package, if you use their ModelSerializer)
- They're quite verbose
- Above all, they're trying to do way too much. A serializer should just be about (de)serialization, i.e. converting from a generic format (e.g. JSON) to / back a python object, with some validation. When you use a DRF serializer inside a view, it can do complex queries to retrieve / modify the model instance, save the instance in database, modify the serializer instance with db values (for example because of default values) and return the modified object in the HTTP response. All this logic should not be defined inside the serializer. Obviously you don't have to do it, but it means you cannot use the DRF mixins, nor the viewsets.
On the opposite, I'm in love with pydantic. Like any other package, it has issues, but it tries to solve a single problem, it's concise, validators are explicit and readable.
Obviously I don't mean any disrespect towards DRF maintainers, they're doing an impressive job, a lot of people use and love their work (and I still use it everyday)