Hi! I'm a co-author on the paper (Mark Dredze).
"Doing math" is not really a goal of bloombergGPT. Take a look at our applications in the paper, which include information extraction, sentiment, reasoning, knowledge. These are models of language meant for use on text documents.
There are some aspects of the datasets that require numerical reasoning (ConvFinQA), but that's not the same as doing math.