Both the strategies have their own pros and cons. For most companies having an in-house team works best, but then you would need to hire someone to manage them.

Getting work done through outsourced agency is always tough. You may try to change the vendor the the challenges would always remain.

What works best is to hire a single senior developer to begin with, who can work closely with your third party team. He can initially get hands on small bugs or features. Slowly you can start building inhouse. Finding a senior person as CTO should be done only after a major chunk of things have moved in house.