The problem with aluminium is its resilience, not its strength. You are right that overall structural performance is governed more by geometry than by material strength (resisting moment is m^4 (area times quadratic distance of center of mass), although there are gotchas like lateral torsional buckling, which require bracing) the main problem with aluminium is that cyclical tensile loading leads to material fatigue with a effective material strength of 0%, while other metals can be counted on for at least 50% strength.