ParentFull threadbhickey·I'm not sure what you mean. BERT has transformer architecture and has a size comparable to GPT-2. Did you have some other model in mind? Or are you trying to draw a distinction between billion and million param models?View on HN