Of course, there are environmental reasons to avoid (some, most?) fish, but I consider wild-caught fish (and all wild/hunted game) fairly neutral from an animal welfare perspective. Perhaps this is a bit naive.
My stance on fish consumption has to do with lack of global regulation. If the place you're eating from a local, wild caught source that doesn't have byproduct waste (also a huge problem) go for it.
Depends how the fish was caught. Generally line caught fish is the best, because it reduces accidental and unwanted kills.