Demonstrating good performance from a non-transformer based architecture is cool. I agree though these particular models aren’t that useful given the current landscape.

I think the intent here is probably to justify training a larger 400B model with their architecture.