You should yell at your accountant. They should have seen this coming. (Early in my relationship with my accountant, while he was getting his head around the jurisdictional issues, he spent about 5 minutes grilling me about every business trip ever to California and all of my clients there. "Look, you know you're not a California business and I know you're not a California business but the question is are we sure that California will come to the conclusion that you're not a California business. They're real bastards about some of the edge cases so I want to know if you're going to hit any.")
You have an accountant, right?
With regards to fighting the penalty: there's often a bit of play with government offices particularly if you happen to bring the honest mistake to their attention first and try to come in out of the cold. You might find that tax offices take a sort of dim view about not filing taxes. I have yet to have ever heard of a tax office which thinks "If you're not aware of the existence of taxes, whoops, that was our bad."