Generics add a lot of complexity to a language. There are two main ways to implement them that I know of--the Java style, with type erasure, and the C++ style, where you essentially generate code for each instantiation--both have significant drawbacks. Either way, it complicates the syntax, complicates the type checker, complicates all the tooling around the language (IDE support, debugger, profiler, linter).
OTOH, if you don't have generics, you either just use concrete types or you essentially do type erasure yourself, explicitly (eg with interface{} in Go or void* in C). It costs you a few casts and maybe some code duplication.
In return, you get a simple language, where it's easier to write robust tooling.
I think erring on the side of simplicity and avoiding generics altogether is more than worth it.