The GPT3 base model was pretty funny if you like nonsense. Instruct tuning and RLHF seem to destroy it when they recalibrate everything.