Speculators do add value by making the market more efficient.
For example, if there is going to be a house shortage then speculators will buy it early thus rewarding people for developing houses earlier.
Or, if there are too many houses, speculators can short it and bring prices down and so make housing more affordable earlier.