Forbes defines weathly as $1M/yr in income and $10M in assets. If you were to retire today with $25M and put this money in bonds, you could collect about $1-2M/yr.
Sources:
https://books.google.com/ngrams/graph?content=millionaire&ye...
http://www.wolframalpha.com/input/?i=+1910+1+million+dollars
http://www.wolframalpha.com/input/?i=1+million+dollars+in+19...
You could live comfortably for the rest of your life on much less than $10MM in assets, even in the absence of an income. So I'd say if there's an absolute limit to "wealthy" in our geographic region, it would have to be "don't have to work for a living", which would be around $1-2MM for a middle-class lifestyle and a 4% annual return on capital.
The term also varies internationally as well (emerging vs developed countries). Like having a car, hot water, etc.
In common conversation people often want to use a term that means that one can live a semi-extravagant lifestyle off of the nestegg without working. A very small minority of people use "wealthy" for that, but not enough that I could use that term and be understood to mean what I want to mean. Commonly people use "millionaire," but that's wrong as you point out.