"You" aren't only inside your brain at all. Imagine that at every timestep, there's a single consciousness that experiences qualia all at once, across all matter - and the feeling of being "inside your brain" is just that the qualia happening there have a large number of causal relationships to your recent brain activity, your working memory, longer-term memories, inputs from your senses, etc - but at that same instant, "you" are also the person sitting across from you - there just isn't the same level of cross-chatter and so "you" can't experience both brains simultaneously.
Kind of like context switching in an OS or something. This would enable a panpsychic view where consciousness was field-like, taking on large values where matter is densely causally connected, such as in the brain, and low values where it isn't, such as in the empty space between two people.
Anyway, as Hicks said, "we are all one consciousness experiencing itself subjectively ... here's Tom with the weather."