Not at all. If you do basic coordination of who is editing which section when, merge conflicts are exceedingly rare. I've collaborated on multiple 10-ish page documents (research papers) with 3+ people using LaTeX+git and it has worked just fine.
Coordination is bother, not having to do it is big advantage.