No: the killer issue is the size. The loop index going negetive doesn't affect the performance at all, but making sure that it actually wraps at 32-bits instead of 64-bits requires simulating the 32-bit wrapping with a clamp.
GCC says:
> -faggressive-loop-optimizations This option tells the loop optimizer to use language constraints to derive bounds for the number of iterations of a loop. This assumes that loop code does not invoke undefined behavior by for example causing signed integer overflows or out-of-bound array accesses. The bounds for the number of iterations of a loop are used to guide loop unrolling and peeling and loop exit test optimizations.