"How do we craft regulation such that those companies are more profitable when they do what we want them to do?"

Sounds like the U.S. already has such regulation - anti-trust regulation. As ChuckMcM noted above, the problem is that Comcast has a monopoly over the last mile. Effective and enforced anti-trust regulations would break up this monopoly into several different competing companies.

BTW, here in Israel, the government apparently has taken the position that services requiring significant infrastructure investment (Internet or cellular networks) are best handled by handing out a limited number of licenses using a bid. We then get an oligopoly where the companies are wildly profitable, until the government has decided we got the infrastructure we wanted, and removes the barrier to entry. Specifically for Internet, we have a duopoly over the last mile "going until the city exchange" to put it in ChuckMcM's terms, then a lot of ISPs taking it from there. IMO, not a bad way to go - we don't pay a lot and get decent Internet everywhere in the country.