Nobody I know in the industry has a contract. We can all be fired at any time for any reason, or for no reason at all, and frequently have been. The one and only consequence we might face for not transferring rights to software that was not written at the behest of our employers is loss of employment. But we could face the same penalty for wearing cargo shorts or a tank top to the office one day.
If you write something worth more than your current job, go for it, buddy. The worst they can do is fire you. I have never seen even one of those accessory agreements that might meet the legal standard for a contract. I sign them because my employer tells me to sign them, and I don't want to be fired for insubordination.
I'd actually be fine doing janitorial work, if I still got the same pay. Because then I could delegate my assigned work to an actual janitor, earning janitor pay, and I could spend my free time looking for another job.