Yep, I think our ideas are shaped by our language.
Same thing with the idea of "existence". It actually requires consciousness to make any sense. The two concepts are actually dual of each other!
Thought experiment: if I told you about a parallel universe which you would never be able to detect, or deduce from any phenomena in this one ... how is then saying "it exists" different from saying "it doesn't exist?"
Now imagine this world with no conscious observers. How is its existence any different from that of the hypothetical parallel universe?
In short - our very concepts of existence and consciousness are really influenced by the language we use.
Now an interesting question: can you pose ANY question about consciousness or existence that doesn't have a trivial answer, if you are CAREFUL to avoid all ambiguity in your language?
I believe that, from an atheist point of view, there IS no root you can get at. Everything is relative. When you are careful to state the subject and object of a sentence everything becomes trivially obvious.
The trouble is when you take a ternary relation (eg A should do B if A wants C to happen) and omit one of the terms. Then you get mystical wondering "How do we ground the moral imperative that A should do B?" Once you are careful to include all the input variables, morality and everything else becomes relative and the statements are merely relations. Then you get something like logical positivism.
The main question still remains - WHY are things the way they are, so consistent? And atheists may never know! :)