Software Engineering is serious business, there's bugs, new features, maintenance, testing, etc. They failed to manage their code. You need to be realistic that with a team of 2-5 people it could take years to fix.
It might be best to put it out of its misery if they can't hold off their clients demands and buy you the time needed to rebuild it.
Unless you're can't-pay-your-bills broke and there are no other jobs in your area, the correct response is "I'm sorry. This can't be fixed. It needs to be scrapped completely and replaced with something maintainable. If that's not an option, I'm going to have to resign."
Even if you manage to fix it, it'll still be a pile of crap, and you'll never get the credit you deserve.
On the other hand, if you don't manage to fix it you're going to get blamed.
Zero credit on the upside, major blame on the downside. There's nothing for you in this but pain, my friend.
I think you should start figuring out where it forked from the original code base & generate a diff from that. Then throw a bunch of tools, from Cacti to Xdebug/APC's control panel, to get a good handling of the current benchmarks. Get the code into a SCM (probably git), then start tackling current bugs & feature-requests - BEFORE you start tearing down to the baseboards.
After a month or two of that, you'll see the first pain points. Going after the pain points one at a time, rather than all at once, will keep your client happy and you sane.
I would look at migrating the clients to another product that works, or even a less forked version of vtiger.
I've turned around a few failing projects and rarely is the problem technical, most of the issues start with poor management.
You need to sit down with them and explain that just isn't possible, then explain clearly what is possible, and what the options are (painful rewrite in parallel to maintaining the old software, or painful and slow refactoring), and just how much it's going to cost them. They aren't going to like either option but it's better for them to understand upfront exactly how much of a problem they have, and that the problem was not necessarily with the previous contracting companies, but with their codebase.