The concept of "constant time" doesn't mean "always the same number of instructions, a value we can exactly predict": it means essentially "there exists some X such that for all accesses of items in a structure of size N at any index it will always take time less than X, independent of N". For "linear time" we would say "there exists some X such that for all accesses of items in a structure of size N at any index it will always take time less than X multiplied by the value N".
So "O(Q)" means "there exists some X such that for all accesses of items in a structure of size N at any index it will always take time less than X multiplied by the expression Q, which is expressed in terms of N". O(1) is "constant time", and does not mean things take one instruction: it means the expression does not depend on N. There is no O(1000): that is the same as O(1); and if an operation takes randomly either 10 or 1000 cycles, it still is O(1) as X is 1000: both 10 and 1000 are bounded by 1000 multiplied by 1.
Time complexity is thereby a measure of how scalable something is, not how long it will take. An operation that is linear might beat one that is logarithmic at small values of N, but eventually you will have an input large enough that the logarithmic one is not just faster but will always be faster as the constant multipliers can no longer compensate the bounds for the one underlying expression being swamped by the other one.
The constant time constraint is true for simple memory access on a computer running a single process (which I only add as a constraint as the entire theoretically framework breaks otherwise), even if some accesses take a thousand times longer than other ones: there is a slowest possible time past which it will not be slower. No matter how large the array gets, it isn't going to get slower, even though it is super fast for extremely small arrays that fit entirely in some small amount of cache.
Of course, it might not be true if memory is backed by a file and that file is on network storage, and the remote file is stored in a linked list ;P, but that isn't the situation with cache: the controller doesn't even keep a resizable data structure for cache lookups. If you plotted a graph of the pefromance of lookups vs. the size of the array, once you got noticeably past the size of the cache the line would stop growing, and the entire thing would fit under a flat line at some constant X.
There is a fiction here, and that is that we can think about these issues at all in the world of a computer, which has a limited amount of storage (which means there exists some I past which it is no longer possible to even access elements as we cannot allocate an array that large); but it isn't due to cache misses and has nothing to do with the kind of predictable real-time constraints you describe: those are constant multipliers on the complexity. That is when you enter the world of physics, which is arguably more interesting.