For a body that's nearly spherical like the Earth you do a multipole expansion. Most of the gravity comes from a monopole (just a point mass at the center of gravity). Then you calculate (or measure) the contribution to the gravitational field from a quadrupole moment, and then an octupole moment and so on.