You're right, glycogen is another place that carbs can go. But unless you're exercising there probably isn't a lot going there.
If you're exercising, your body doesn't build glycogen; rather, it shuts down the metabolic pathways for glycogenesis because burning glycogen while building it is very inefficient. (It's a form of hysteresis.)
... painting with broad strokes because my background is in biochemistry, not nutrition.
So what I'm saying is if you're mostly sedentary, eating a high-carb diet, your blood sugar levels will rise to the point that insulin is secreted. And then some of that blood sugar will be converted to fat, and its energy will be 'lost' - it won't be used unless blood sugar levels go down again and glucagon is released. But many people eat way too much and too often for this to happen, and they just keep on packing on the pounds. Thus, carbs causing weight gain while fat gets the blame.
I mean, think about it. All the sugar that comes into your bloodstream from, say, two bowls of cereal in the morning. You have about 3 cups of milk, 3 cups of grains. This is a pretty big amount of carbs that gets digested very quickly. Then you drive to work and sit on your bum for 4 hours. Don't you think some of that carbohydrate energy could be converted to fat before your next meal?
When you eat breakfast in the morning, you're most certainly replenishing glycogen. While you sleep, it's the primary source of blood glucose. Your body will prefer to replenish the glycogen before beginning lipogenesis.