Even if damping is removed they might not be reversible. Logic gates that were found to be individually reversible, were found to have difficulties operating when connected in a circuit:
https://core.ac.uk/download/pdf/603242297.pdf
It's impossible to avoid incurring some losses at finite speed, but as far as I know there is nothing fundamental preventing one from approaching reversible operation when operating at a sufficiently slow (but nonzero) speed.