The standard Google employee contract contains this restriction. It's easier for them to institute a blanket ban on open source contributions (including in one's spare time) and then consider exceptions on a case-by-case basis. They have a dedicated internal committee that fields employee requests to work on open source projects.
Weird to even impose any restrictions at all outside of working hours. Can’t see how it’s any of my employers’ business?
How would they view working a second job? Does it matter in what field?
Depends on where. In many places this would be unenforceable and in some places illegal.