If I'm remembering correctly, that was actually the main problem. Most, if not all, are being held based on evidence or intelligence that would not fly in a typical American trial.

If they closed Gitmo and brought the detainees to the US for trial, almost all would have to be released.

I'm not necessarily saying that is a bad things, surely some (or even all) do not deserve to be held and should be released, but all it takes is one coming back and committing a terrorist act (could you even blame them after being illegally held and tortured for years?) for the worst PR nightmare a politician could even imagine rains down on them.