This idea has been explored by Julian Barbour in his book The Janus Point.

http://www.platonia.com/books.html

The related math and modelling goes under the name Shape Dynamics:

https://en.wikipedia.org/wiki/Shape_dynamics

Shape Dynamics - An Introduction

https://arxiv.org/abs/1105.0183