This is a problem with the permission to grant being too coarse. Your app needs blanket access to the clipboard contents. Things would be nicer when it only needs access to relevant URLs in the clipboard.
But that would require the OS to know what a relevant URL is, so the permission must be scoped to a prefix, or maybe a regex(1) to be applied when the clipboard contains text content.
(1) restricted to "proper" regular expressions that don't have the security issues of Perl-style REs.