The macroscopic transformation is time asymmetric (the balls will not sort themselves back again) even though the microscopic transformations are time symmetric (elastic collisions and frictionless movement).
I agree with your observation of what happens to the black and white billiards. My question is - if the interactions are all time-symmetric (implies conservation of entropy), where does the entropy come from in the macroscopic system? My belief is that the microscopic model is flawed; if we simply introduce some randomness into the collision dynamics of two particles (which is believable given particle accelerator experiments / QM), then microscopic interactions are no longer T-symmetric.
It comes down to a philosophical interpretation of whether you can treat a system of two particles probabilistically (i.e. thermodynamically in aggregate) or not.