That hasn't been a practical problem in many years. Most servers have gigabytes of RAM and a 64-bit kernel nowadays.
Not sure about FreeBSD or any other OSes.
# time sh -c 'netstat -tn | wc -l' 486206
real 0m13.538s user 0m1.698s sys 0m10.380s
It still works with a whole lost of connections. (in fairness, only about 130k were connected)