I found that partitioning the table to allow parallel auto vacuuming was necessary to scale.