twbrownaw is exactly right. If the private key is already in a secure location then there is no need to move it to a secure location. So, any report that the public keys were stored with the private keys means nothing by itself.
You are arguing that storing the public and private key in a vault under the ocean is not less secure than just storing the private key there. While that is technically true, it's also missing the point.
How is it missing the point? It tells us that having "both keys in one directory" is, by itself, no evidence of insecurity by Linode.