You can in most countries as far as i know. But in my example i quit my account (so made it prepaid essentially) and lost the SIM card, which means i lost my account forever. Now it waits for the simcard to invalidate and then will most likely sell the number again. It was a "easy number" (as in people remember that number after telling them once) so i assume it will be resold rather fast.
But just because you can does not mean people want that. Before it became normal to auth everywhere with phone numbers i happily changed my number yearly.