What if you dig down deep enough to molten rock? Then you're aren't limited by heat conduction of the solid rock, but the heat convection of the magma
But most places that is way, way deeper than 20km.
The thickness of the Earth's crust is strongly bimodal, making "average" meaningless: thinner under oceans, thicker on continents. Much thicker. Most of the planet is, of course, sea floor, so mostly thinner. But drilling the sea floor is unpleasant and expensive, and at the end, the hole is all the way down there.