In the actual words he seems to say notwithstanding the loss of life from war:
>Population has steadily increased, notwithstanding the waste that has been made in the camp, the siege and the battle-field; and the country, rejoicing in the consciousness of augmented strength and vigor, is permitted to expect continuance of years with large increase of freedom.
Freedom only decreased if you saw black slaves as non-human. They were a majority of the population in many southern states.
"Southern states saw" in the headline is probably ignoring the black population, but I'm not sure. It was only many years later that there were large migrations, after the reconstruction promises passed them over, "vagrancy" was heavily outlawed, and they effectively lost the vote.