Telephoto has little to do with focal length, other than that it is often used in longer lenses to keep them smaller. It is a way of constructing a lens where the physical length of the lens is shorter than the focal length.
Fuji's 27mm pancake lens is only 23mm tall. A telephoto lens it is not.
The commonly used meaning has drifted from the original/technical definition. The same has happened with other things. 'Lens' for example, is technically one single element; the whole assembly is technically an 'objective'. But in photography the latter is what is commonly referred to as 'lens'.
Language isn't always precise and often context dependent (I don't always like that but there's not a lot I can do).