It's purely the brightness. If you're in ambient bright light you don't develop myopia even if you're looking at nearby objects the whole time (e.g. reading books or a phone), if you're in a dim environment you still (are at statistically elevated risk to) develop it even if you're looking at distant objects. This is something they've had to keep doing bigger and bigger studies on because everyone who hears about it has the same kind of thoughts as you did.