The issue is specifically gerrymandering on the basis of race.
Parties know their votes by precinct, those data are available.
Apportionment of voting districts must be done by population, so the Census is necessary. Discrimination on the basis of race violates multiple Constitutional amendments and voting laws, for which enforcement requires clear knowledge of demographics.
The justirications are somewhat involved, but not strictly circular.
For a discussion of the background, from a conservative viewpoint, see: "Redistricting, Race, and the Voting Rights Act" (2010). This touches on the question multiple times, this passage captures much of the circumstance, though I recommend reading the full piece:
[T]he South's swift and effective circumvention of the Voting Rights Act in the wake of its passage demonstrated the truth of what many civil-rights activists had argued: that merely providing access to the ballot was insufficient after centuries of slavery, another century of segregation, ongoing white racism, and persistent resistance to black political power. In the Deep South in 1969 (and beyond), the alternative to de facto reserved legislative seats for black candidates would have been the perpetuation of whites-only politics.
https://www.nationalaffairs.com/publications/detail/redistri...