We are not talking about real probability - we are talking about an unfalsifiable thing having no evidence, which is functionally the same thing as proving it false, we can't prove it false, but we act like it is, because doing otherwise would mean acting like anything insane and unfalsifiable you can think of is true.
The reason this distinction isn't clear is because no one needs to make it in the real world.
Atheists don't believe in god. This doesn't mean that - if provided proof, they wouldn't change that belief. That's how normal people use "belief" in the real world, the only way that the weird definition twisting of "Atheist" makes sense here is trying to claim "belief" has to mean unwavering absolute truth.
You can either be talking in absolute logic, and allow for "not false" to mean more than "true" because this isn't a boolean, or you can be talking in natural language where the central debate doesn't hold because no one uses belief like that. You can't pick and choose common use and holding to strict definitions.