For reference, I believe relatively rudimentary chatbots have been able to pass forms of the Turing Test for a while, while also being unable to do almost all the things humans can do. It turns out you can make a chatbot quite convincing for a very short conversation with someone you've never met over the internet[1]. I think there's a trend that fooling perception is significantly easier than having the right capabilities. Maybe with sufficiently advanced testing you can judge, but I don't think this is the case in general: in general, our thoughts may be significantly different than we can converse. One obvious example is simply people with disabilities (say a paraplegic) who can't talk at all (while having thoughts of their own): output capability need not necessarily reflect internal capability.

Also, take this more advanced example: if you built a sufficiently large lookup table (of course, you need (possibly human) intelligence to build it, and it would be of impractical size), you can build a chatbot completely indistinguishable from a human that nonetheless doesn't seem like it really is intelligent (or conscious for that matter). The only operations it would perform would be some sort of decoding of the input into an astronomically large number to retrieve from the lookup table, and then using some (possibly rudimentary) seeking apparatus to retrieve the content that corresponds to the input. Your input size can be arbitrarily large enabling arbitrarily long conversations. It seems that to judge intelligence we really need to examine the internals and look at their structure.

I have a hunch that our particular 'feeling of consciousness' stems from the massive interconnectivity of the brain. All sorts of neurons from around the brain have some activity all the time (I believe the brain's energy consumption doesn't vary greatly with activity, so we use it), and whatever we perceive goes through an enormous number of interconnected neurons (representing concepts, impressions, ideas, etc.); unlike typical CPU-based algorithms that process a somewhat large number (potentially 100s of millions) of steps sequentially. My intuition does seem to pair with modern neural architectures (i.e. they could have some consciousness?), but I really don't know how we could do this sort of judgement before understanding better the details of the brain and other properties of cognition. I think this is a very important research area, and I'm not sure we have enough people working on it or using the correct tools (it relies heavily on logic, philosophy and metaphysical arguments; as well as require personal insight from being conscious :) ).

[1] From sources I can't remember, and from wiki: https://en.wikipedia.org/wiki/Turing_test#Loebner_Prize