Really wild that Deepseek's model is 536+16B (passive+active) parameter model and its really comparable to Mimo v2.6 Pro which is the current open source SOTA model which is at (978+42)

This is half the size of Mimo and comes pretty damn close to it and I feel like lots of tasks can be done by it.

I really like its speed as well but its quite verbose from the benchmarks, IMO, this should be tackled in the near future as well. I wonder if specific fine-tuning can happen just to make the model token-efficient.

Also, slightly offtopic though deekseep has good tps but Nvidia's nemotron recent 120B model also has some really crazy tps. I wonder if we are gonna see some really good tps on general hardware.

The best part is that the amount of current innovation happening within the space means that a lot of good ideas are bound to be absorbed by one or the other and we will get many of these improvements handed to us :-D

I think that such engineering is also the engineering of the trade-offs and on what axis to play on. You can be SOTA but with very low speed, extreme price and other factors. I think its interesting to see how each engineering lab approaches these trade-offs and what they pick. But whatever they pick, its best because its more choices for us to pick upon from (especially if its open weights)