Would it be possible to have it access only your personal profile data? The permissions screen shows an Organization Access section with all my orgs checked.
Unfortunately I can't exclude organisation altogether, GitHub applications have access to organisation with the permissions that the user has allowed the app.
However now GitHub Notifier only asks for read permissions :)