This is why the "timbre" of a piano is different from the timbre of a sax, and why the same C note might fit in a song if it's played by a piano, but not in the same song if it's played by a sax.
If you take a higher level view, the mathematical structure of the notes (the chords, ...) needs to fit the mathematical structure of the harmonics of the instruments (the timbres) to get a pleasing sound. Put another way, the timbres of some pairings of instruments might clash on a particular sequence of notes, so you need to search for a different instrument pairing which is harmonious on that sequence.
And then you have synthesizers, which are very mathematical in the way the timbre is generated (oscillators, filters, ...), but this allows you to exactly fine tune the timbre so it "fits"