We have to decide as a society, and decide really soon how far we are willing to take AI research.
We built the nuclear bomb, but we decided when to stop. We didn't build more advanced bombs that could level entire providences, even though we very well could have. I know this sounds ridiculous, in that we aren't talking the same level of potential devastation, but it's the principle I am after.
Software on the other hand, my computer is running tens of millions of lines of code right now, and for stuff with heavy utility there's not much the government can do to get in the way. Cryptography is a great example of this. Cryptography used to be regulated, but because it's easy to pass around code + binaries + keys there was really nothing the government could do to stop people from using it. So it was deregulated, as an admission that if you banned cryptography, you get a population without it pitted against criminals who do have it.
I'm guessing that AI will end up much the same way. You do need a lot of data, certainly a lot more than you need to get strong encryption, but data is ultimately easy to move around compared to nuclear manufacturing facilities. And it's even more open when you consider how much data is available that might be useful to an AGI. If you give a human 500,000,000 hours to read + process + innovate using just Wikipedia, you are going to get something impressive. An AGI has access to any page that's reachable from a URL bar, and that data is not something that needs to be passed around with the AGI codebase.
Granted, I personally believe that AGIs are 30+ years away, and I don't think that Wikipedia is enough data for something like a competitive NLP machine using modern technology, but I think that's the direction we will be moving in. We should accept today that AI will be a part of the future, and prepare for the inevitability rather than try to run from it.
After all, the corporations joined in this partnership are the very behemoths that gather and control almost all the world's social and personal information, which is their primary product and essential to their revenue stream and their very survival. If the public (and gov't policy makers) ever conclude that these firms and their AI are a rising threat, then like various European countries, the US too might enact 'onerous' regulation which could diminish their access to our data, AKA their life's blood.
I think THAT'S what this AI partnership is really about. It's the start of a political action initiative, apparently led by unthreatening figures like CS academics rather than capitalists. It's intended to diffuse the visibility of these companies' advancing political interests and lobbying in Washington so their growing injection of AI into our lives won't suddenly show up on our radar one day soon as an inbound threat from the clear blue sky above. Once they gain a sufficient foothold in Washington and assure that we remain inured to the idea of AI, they can turn up the heat until we froggies boil, blissfully oblivious to the rising clouds of steam.
Quite aside from that, there's a point past which chasing higher yields results in a loss of strategic flexibility, which is why both we and the Soviets eventually gave that up. (You can attack one big target with multiple warheads, but you can't attack multiple targets with one big warhead.) Modern nuclear weapons research and engineering, whatever there may be of it, would concentrate on boring stuff like improving efficiency and reliability, and reducing mass.
The giants are unparalleled in terms of their access to data, and that's something alternatives and competitors can't compete with. It's also clear that for machine learning, having more data is the most effective way to get better results. Stronger algorithms are only going to go so far if your competitor has three or four orders of magnitude more data and computational power.
And that's the scale we are looking at with companies like Facebook and Google.
Every single person working in a position on the cusp of automation.
The same thing happens with any new technology that threatens massive industries. (automobiles, airplanes, electricity, heating oil, etc)
There are AGI risks that are well-reasoned and have detailed descriptions. If you aren't engaging with those arguments, proving them invalid or providing a well-reasoned protection, you aren't representing a coherent point of view.
Since when does refusing to engage with fear mongering make you incoherent?
Where do you check out?
Edit: Nick Bostrom is the thought leader on this subject and he spells it out in this talk: https://goo.gl/DzNnk2. His book, Superintelligence, is really good.
Edit 2: And here is the audio for a Sam Harris talk also on this subject: https://goo.gl/6wdG43