Memories[1] can only form in the increasing entropy direction.
Or, to be more precise, higher entropy states can contain memories of lower entropy states, but not vice versa. This remains true regardless of how the memory is represented.
So the arrow of time (and second law) are not facts about the universe, but about observers' ability to represent it.
Even if you wound back time past t=0 and kept the simulation going, you would see the same thing in the reverse direction, such that the negative time direction should now be considered futureward.
IOW, it's not that "time goes forward, entropy goes up, such coincidence". It's "if entropy were not higher, we could not regard it as the future."
[1] what Barbour calls "time capsules" and Drescher calls "[metaphorical] wakes"