I've written about this a lot elsewhere: https://medium.com/terms-conditions
The message you sent to the recruiter and the hiring manager at Company X was: "I don't want to work for Company X."
Hiring someone is kind of like getting married -- an employment agreement is a long-term commitment to mutual dependence as you work toward a common goal, and it involves a large share of your hours every day.
You want to marry someone who's nice to be around, who can hold up their end of the deal, and who shows clear signs of wanting to marry you in return. That desire is expressed through the effort they put into your relationship, especially in the beginning.
Both you and the company face the same challenge: gathering reliable information about someone you don't know. The only information the company cares about is a) can you do the job? and b) are you easy to get along with? By refusing to take the coding challenge, you gave them no answer to a) and a clear signal that the answer to b) is "no."
I'm not saying you should waste a lot of time taking hourslong coding challenges for companies that you don't want to work for. But if you do really want to work somewhere, you need to find a way to show them that you can do the work and that you won't be a wrench in the gears.
You could have tried to meet them halfway. Get some projects you're proud of up on Github that you can point them toward. Let them know that you'd be happy to do some live coding during a technical interview so they can hear you think out loud as you solve problems. That's one way to minimize the time and effort you expend, and still give them information they need.
They're not going to pay you to get that information, but like you, they should be willing to spend some time, in a process of escalating commitment, to make sure you can do what you say.
What a lot of good software engineers don't realize is that there are a lot of bad software engineers out there. The job market is full of noise. People lie about their skills and experience. So employers need to be able to test those claims, and they do it with coding challenges.
I have to respectfully disagree with @taprun. The fact that coding challenges can be applied to the job market for software engineers makes it much less corrupt, and raises the quality of matches between employers and employees.
MBAs and HR people are not in a position to show what they make. This leads to a lot of bad decisions. That said, there is still a screening process. A good company does a ton of reference checks, and watches them closely to see if they deliver on the job. If they don't, they're usually gone in a few months.