I also broke my ankle about 20 years ago. Once I was out of the cast, I regained all function, and only recently has my ankle been starting to ache. The doctor I saw about it even commented that it was as clean of a fix as he'd ever seen!
I also didn't do much physio, but that was mostly due to me being a 23 year old moron (which, believe it or not, also had a lot to do with me breaking my ankle in the first place!)