GPT-2 showed us that simply having a huge parameter space can be enough for internal consistency. Have you read the unicorn valley article it generated?