Google is a company with shareholders and P/L. It's not a thought experiment, a family, a social commons, or a debating society. It exists to make money.
Google took the decision to fire him based on what was likely to create a conducive atmosphere for its workers.
His memo, however construed, made it likely that he could no longer be able to contribute as effectively to some teams.
Google's responsibility to Damore begins and ends at their mutual alignment of economic interests.