Does this apply to non-transformer based architectures as well?