You don't need to go back to 1623. Talk to fishermen who have been working the same waters for 30 years, the decline is very apparent.
Say 30 years ago the fishery was at 80% depleted, and today it is 95% depleted, then conservation looking only 30 years back could decide to stop (or more likely lose political will) at that 80% mark.