The point of distributed computing is to do computing that you can't do on a vertically scaled system or to increase availability.

If you're doing it for other reasons it's usually a mistake.