If they are required for the compile and someone manages to get something malicious into them, they can certainly affect the produced/deployed artifact.

See Ken Thompson, Reflections on Trusting Trust, 1984 https://www.cs.cmu.edu/~rdriley/487/papers/Thompson_1984_Ref...