ParentFull threadh_tbob·I wish they didn’t use swiGLU and preRMSnorm so we could have a better comparison.Then we would know how much this transformer innovation helps by itself.View on HN