I'm not sure why anyone would say GitLab CI is slow. You run it on any machine you want, so it's fully dependent on that.
The script that authenticates and picks up the job is a small Golang app. That's all. GitLab executes CI jobs immediately.
See the architecture and related links here: https://about.gitlab.com/gitlab-ci/#architecture