Vitamin D is fat soluble. The fat in milk helps your body to absorb it better.
“ We conclude that absorption was increased when a 50,000 IU dose of vitamin D was taken with a low-fat meal, compared with a high-fat meal and no meal, but that the greater absorption did not result in higher plasma 25(OH)D levels in the low-fat meal group.”
Or this meta-regression of 43 studies: https://pubmed.ncbi.nlm.nih.gov/24993750/