Not hard at all. Baseball, from juniors to the majors, also uses sign language.
This need additional contexts. Deaf communities use Signed Languages in baseball. Outside of Deaf communities, they are not sign language. Language by definition requires grammar structure, cultural information, foundation of linguistic, etc.
They are using pidgin is the word you are looking for. Baseball use pidgins, they don't use sign languages. It is an important distinctive because one is actual language and other are not.
It is not because of people fighting against it. It just that English is damn complicated and not easily to be an expert on it. Written and Spoken English does not have the same discourse style. There are times when I thought I structured it correctly and turns out it not. I am natural-born American and English is my first/second language (ASL is my first). Even English native speakers struggles with it than non-native English user.