This is mainly why I stopped forwarding emails from Gmail to Fastmail as >95% of the spam was sent via Gmail, and I suspect Fastmail doesn't try to filter those.
Anyway, a helpful tip for you is that the Fastmail spam filter is configurable. You may want to look at making it more aggressive. Go to Fastmail > Settings > Filters & Rules > Filters > Spam protection > Advanced settings. Here you can either just select the 'Aggressive' preset or even customise your own thresholds for flagging a message as spam. Here you can also check how many messages are have been learnt by the filters.
If you're forwarding emails from Gmail I'd also suggest turning that off or setting up a rule to dump the Gmail forwarded messages into a separate folder. I suspect you'll find most of the spam has been passed on by Gmail.