However, now I am questioning that perspective and wondering should the government be shaping our future and if so in what ways?
After all, when it's not seized by a spasm of science denialism, the government is a chief backer of basic research, which is then often licensed on extremely generous terms to make a wide range of industries and products viable. One could argue that the government is, in fact, uniquely positioned to fund truly basic research, for which risk is often too high - and payoff too uncertain or delayed - to be fundable in a corporate environment. Put another way: the government is often, contrary to belief, much less risk-averse than private enterprise when it comes to research of this sort.
For instance, take this Internet thing we're all communicating over. Without R&D efforts by ARPA in conjunction with several prominent universities, or the willingness of CERN to place key Web protocols in the public domain, we might very well have had non-interoperable AOL-style fiefdoms. But what company would, in the 70s, have funded an unproven telecommunications network just to connect a handful of universities?
When a government tries to build infrastructure directly, you often get disastrous outcomes. Australia's so-called National Broadband Network is a recent and woeful example.
The sole counterexample is probably roads, and even then there's an argument that the public administration of roads has many systematic issues that could be blamed on governmental factors.
What you mean by "long view" here? I also don't particularly see why private companies would (in principle) be better than government-run institutions. I mean, in most of the world universities are government-funded institutions, and I think they produce quite a bit of innovation?
The key here is independence from the direct source of funding. There's nothing magical about private enterprise that means it'll be more "innovative". You just need independence[2].
> When a government tries to build infrastructure directly, you often get disastrous outcomes. Australia's so-called National Broadband Network is a recent and woeful example.
I can't claim to know much about Australia, so can you explain what went wrong with NBN?
I can tell you that e.g. Sweden is among the top[1] of the world when it comes to Internet speed, and I believe that was mostly done by government mandate. (Not sure who actually laid the fiber, but that's pretty much beside the point.)
[1] https://en.wikipedia.org/wiki/List_of_countries_by_Internet_...
[2] As in: Your funding is guaranteed for X number of years, regardless of political whims, popular opinion, etc.
Politics, I believe, is a symptom (of a much bigger problem).
In short, We get what we ask for.
There are plenty of reasons for Uncle Sam to be covert. Obviously, national security comes to mind. The other is the Constitution, as well as other things that are supposed to limit the gov's.
If history is any indication, when it comes to Uncle Sam: just because we don't see it doesn't mean it isn't there.
For example, it's great that publicly funded prior research and experts bootstrapped SpaceX's R&D, and then NASA's contracts for low-priority launches helped them iron out the kinks. It was a nice transition from publicly funded risky R&D to a market-driven private industry. On the other hand, NASA's "full-stack" SLS product is an inexcusable clusterfuck.
Corporations are way better at diving deep to create practical, economical products... very very lean. The government is good at running lots of small risky/low-ROI projects, unprofitable but foundational to future private sectors. Tax the past to subsidize the future.
Unfortunately the future tends to hide its money in offshore bank accounts.
Government also has a greater role to play in extremely long-term initiatives that don't make sense in a shareholder funded vehicle. Much of our current society is founded on government funded R&D as Candu mentions.
(On the other hand, Space X versus the usual defense contractors is quite a contrast.)
The role of government, when it comes to minority rights, can act like a big fat rock under the wagon wheel though. As well it should.
But pro-society government (centrist), with regular people having direct tools to remove officials, with politicians being responsible for their election promises and with no lobbying allowed, with tight anti-corruption laws and yearly referendum in place.