For example lets assume a recruiter can only find job offers for 1 candidate at a time, and it takes on average:
- 1 week of work to find a 10% better offer
- 2 week of work to find a 40%+ better offer
It would seem like that 40% better offer is definitely worth the work, but for the recruiter it may not be.
If engineers makes say $150k, and the recruiter gets 10% of their new salary, they would make $16.5k from 1 week of work, or $21k from 2 weeks of work. If they can turn 1 candidate per week at $16.5k they will end up making more than if they spent 2 weeks per candidate, so they have little to no incentive to actually get you a better offer if it takes more work.
Not to mention recruiters are more likely to work with companies multiple times per year and an individual maybe once per year, so it is likely in their best interest to help each company get the best price on a candidate rather than help the candidate get the best offer possible.