The fast native code Lisp compilers such as SBCL and Lispworks sacrifice correctness for speed. You can fully expect the VM to crash when you give incorrect hints to the compiler. This is obviously unacceptable for languages such as Ruby and Python. I don't know of any serious Lisp JIT compilers.
Smalltalk has never had decent performance as far as I know - not even with a JIT backend. Not compared to Java/C# with JIT compilation at least.
So yes, my claim is that languages such as Python and Ruby are inherently slower than languages such as C# and Java. The less information the compiler has to work with, the fewer optimizations can be made.