What if they ask "Do you have any suggestions for how I could perform better at my next job?" That opens the door to nonspecific advice that could touch on those issues.
Essentially, if you give the ex-employee anything concrete to work on ("You could improve on skill X"), then the fear is that they'll demand clear evidence that they were deficient in that area at your workplace, which you probably don't have. Now you've weakened your case should the employee bring a wrongful dismissal lawsuit (under whatever laws apply in your state/country). Hence the reluctance to say anything at all.
It's not right (morally), but it's understandable.