I believe 'we' referred simply to himself and his familial unit.

But I'm making just as many assumptions as you are, so either could be the case. Based on his argument, though, I think it's not likely he was speaking for all of Sweden.