> we do not have a way to scientifically rule out the possibility that LLMs aren't potentially self-aware/conscious entities of their own
That may be. We also don't have a way to scientifically rule out the possibility that a teapot is orbiting Pluto.
Just because you can't disprove something doesn't make it plausible.