I don't know, I feel like we actually should generally dismiss the idea of things as harmful when there are no studies that demonstrate harm but there are studies showing a lack of harm. If the facts change, so should our assessment. But not before.
Really the debate about could something be harmful is missing the point, and likely intentional misdirection by certain actors. Many things could be proven to be harmful in the future. But bad actors aren't asking that we just keep our minds open to the possibility of that future discovery, they're demanding we make policy decisions now based on that potential outcome.
If we're going to ban vaccines with aluminum, it should be because we have scientific proof that it's bad or very likely to be bad, not just that the possibility exists that it could be bad and that badness might be demonstrated at some point in the future.