This is actually a common pattern called "model distilling".[0]

[0] https://platform.openai.com/docs/guides/distillation