Unfortunately, GitHub's API does not provide per-repo permissions, so in order to read/write to
any repo, the CMS has to get permission to read/write to
all of them. This is a fundamental limitation of the GitHub API [0] - we'd love to be able to request more fine-grained permissions. This is a pretty frequent complaint about
any application built with GitHub's API.
The r/w access to public keys is specific to setting up continuous deployment from a GitHub repo to Netlify's static hosting service. You can read more about why Netlify requests those permissions for continuous deployment here: https://www.netlify.com/docs/github-permissions/.
While Netlify's continuous deployment is (obviously) the deployment strategy we recommend, you can use the CMS for any static site, as it's totally build/deploy agnostic. You can use the CMS to edit your content and then build that content with Hugo and host on a VPS if you wish. You can also use Netlify to host your content without continuous deployment - it supports updating content with both an in-browser interface and a REST API [1].
The CMS is entirely open source, so you're free to inspect or fork the code - you can find the repository at https://github.com/netlify/netlify-cms/
[0]: https://developer.github.com/v3/oauth/#scopes
[1]: https://www.netlify.com/docs/api/