Yup; this is an extremely silly thing to say.
First of all, people have arrow keys muscle memory from all other programs. Saying "dump it" is as obnoxious as it is stupid; it only makes enemies and gives a false impression of Vim's power.
Secondly, the mouse provides the fastest method of random access on the screen; even things like EasyMotion only barely reach mouse speed. Such nonsense gives the idea that Vim "should not" be used with a mouse, which again misrepresents Vim.