Rails happens to function under browsers popular at development time, but what do the standards (see nbpoole's comment) or logic suggest is the correct response for this? Asking the user agent (which in this case essentially means Javascript running on a page the user is browsing) whether /widgets/32 is equivalent to /widgets/ so it should know whether it should nuke it or not? That's a tough question for Javascript and an impossible question for the actual human who is trying to get this year's widget catalog ready.
"User requested a delete, server doesn't say delete is impossible, we'll try deleting until we get back confirmation or get told it is impossible" seems like a fairly sensible decision for me. (It is non-compliant with the specs, but GETing the new URL is also non-compliant with the specs, and that is the non-compliance that Rails relies on to actually function.)
There were similar issues way back in the day back when the Rails community was enamored with redirect_to prior to finding out important constituents on the Internet like, say, Googlebot think there are hugely meaningful differences between 301 redirects and 302 redirects.