The thing is even locals will have a greater time during the nicer times. Using Montreal, where I was living until a few years ago, is a much nicer place to wander around when there's no snow. It's a colourful city, lively and full of small and big activity to do, so you can get by with the sort of tourism you described in high season (summer), but also end of spring and early fall.

In the winter, it's a cold, wet and relatively low interaction time of the year. I assume some place have a high, low and dead season for tourism, and I personally aim for low, but to each their own. Quebec in general is a great place for winter vacation, just have to go outside Montreal island.