I understand your desire to have smaller models that are SOTA, but it may be more difficult than you realize. Think of it this way, a 100m model is like an infant, in fact we can make a table:

100m = infant

1b = toddler

100b = age seven

1T = teenager

10T = 35 year old adult

And this tracks, because more compute allows the systems to parse more data, and learn over more cyber years. The reason that you cannot find a seven year old as smart as the average teenager is because intelligence NEEDS more data to learn more.

You can do hacks and tricks at the margins for marginal gains, but more compute for smarter models may just be a law of physics for how intelligence is made.