In the code you'll see there is also linear interpolation being performed between the LUT entries, hence the reduced error compared to just using the 256 LUT entry values directly.
Thanks, I missed the linear interp part. I might look into this more for DDS, would be interesting to see how small a lut could be for 90db performance.
Found interesting post that implements computed approximation, apparently it was faster than interpolated lut in his case.