That's not even particularly good advice, which just goes to show how subtle the pitfalls are. The rule really should be "If it is at all possible to delete instances from a pointer to the base class, then you need a virtual destructor". The overhead of adding a virtual destructor to a base class with no other virtual functions is far from negligible in many cases, so the distinction is meaningful (though modern guidance tends discourage non-polymorphic inheritance in the first place anyways).

The same thing goes for the good-old "Do not use raw pointers." The rule is actually: "Do not use raw pointers with implicit ownership semantics attached to them."