Would you say an average person who works an average job has a reasonable shot at getting into elected office in the United States Congress? If no, then that by definition is a form of oligarchy.
Though random samples and a large enough sample size, you can ensure that at least those in elected (sortitioned?) positions are not members of the oligarchy. Additionally, since sortition is much more efficient in selecting candidates, it's possible to replace some formerly unelected bureaucrats with sortitioned ones. Even if it's not perfect, it's still an improvement.