One reason is that non-atomic writes can be torn. So if you have a value like 0x00000000 and over write it with 0xFFFFFFFF, some hardware may do it as two separate writes. This means that another thread can read the data when its half way written and get 0xFFFF0000. I'm using a 32bit value here to illustrate In reality modern hardware is unlikely to tear it, but in other cases it may.
Another issue is that what you are writing may depend on something else. Consider this code:
x = 42; p = &x;
these two operations are independent, and a compiler, CPU or memory system could chose to execute them in any order. This means that even if the second operation is "atomic" and another thread can only read the before or after value of p, another thread could read p and access it, before 42 is written to x. This is why the p = &x; needs a "release" barrier that guarantees that everything before it is completed before the change to p happens.
Atomics are complex due to the way modern computers optimize and this is just a very surface level explanation. Still, I hope this helps.
If you restrict yourself to relaxed atomic loads and stores -- i.e., memory accesses that are atomic in the sense that they cannot tear, but don't have much in the way of ordering guarantees -- then you do get "one of the values that had been written to the memory in the past".
Aside from memory tearing, one other issue is that you generally want to be able to rematerialize loads (i.e., in the face of register pressure, you may want to turn one load of a value into multiple loads instead of loading it once, then spilling to the stack and reloading from the stack). But when the compiler rematerializes a load, then you don't get "one of the values written to memory". From the perspective of the original program it looks like you get some weird superposition of values.
If a value in memory could suddenly revert to something else, then
if obj.x == 1 {
print(obj.x)
}
if obj.x < array.len {
array[obj.x]
}
Above I'm treating atomics and mutexs above.