Why do you like Safari in light mode, or dislike it in dark mode? Are you referring mostly to the app chrome or to the content of the sites you visit?
I would tend towards ease of configuration and consistency, and would only make exceptions for apps/sites with more theme choices than just light/dark.