>However, it is not the case that if you have a high bounce rate, you will inevitably have a low conversion rate

I would say the best way to calculate conversion rate is after you exclude bounced visitors. Bounce rate tells what percent of people found your website wasn't what they were looking for. Conversion rates should tell, how good job am I doing in convincing the visitor, if the visitor chooses to spend time on the website and hence showing interest.

These two metrics are related and but not as tightly. And of course website can (and should) be optimized separately for both of these metrics.

EDITED for clarification:

Just to summarize, following is a pretty good way of seeing things:

CONVERSION RATE = (Total # of non-bounced visitors who completed goal) / (Total # of non-bounced visitors)

For all practical purposes, we should exclude bounced visitors from analysis as it just creates noise. Bounced visitors are the ones who got to your site because of chance and you should separate them from the visitors who came to your site with full interest.