The linear regulators we are talking about are basically voltage dividers with feedback. They aren't significantly more efficient than your design, though they are ton less noisy.
In the passive voltage divider circuit, if the load varies its current draw, you have to have a resistor in parallel with the load so that the total current through both the load and the resistor doesn't vary significantly (where "significant" is derived from your tolerance for voltage variation).