There are a bunch of things that could be tried honestly. Like geometric mean, top-N, or even fitting another classifier on the training set.