Aluminum just gets more brittle when exposed to high energy particles. It's not like it goes anywhere. It will still perform about as well as radiation shielding although there are better materials for that. So if you want to use the aluminum for structural purposes, especially with gravity present, you'd have to make more robust designs just to account for any embrittlement which also means more weight and therefore more fuel to get the parts off earth.