"Now the details of how that labour gets applied to building infrastructure for the common good would need to be worked out, it would be a much fairer system than we have now, and much less subject to being gamed."
People already crack jokes about working in the government, doing a minimal amount of work. Imagine if people were actually mandated to work in government instead of paying taxes. Nothing would be achieved, because no one would seriously want to do the work. Or the people who had positions of power would abuse them to no end.
You describe a nice system, but how would it ever be implemented in practice? What kind of media campaign/propaganda would be required to convince the entire country they should work hard at serving each other? Would this shift be characterized by a revolution? Perhaps it would start small, in communes? Would the shift happen nationally or on a state by state basis (in the US)?