How would Google Calendar know the event is at 4pm in California? Is the location field filled with enough information? If not, that seems like almost as much work as entering the timezone.
This did happen to me with daylight saving a while back so I do sympathize.