I wholeheartedly agree with your assessment.
In contrast, Google’s plus codes(https://plus.codes/) provide the same functionality but avoid the most of the pitfalls you mention.
If you wanted to encode the same data using words instead of alphanumeric characters (which is a debatable goal), then you could just combine it with WCodes (https://wcodes.org/)