"refs" are not behind-the-scenes things, they're the things you want to operate on. It wouldn't be like having to learn JPEG superblocks, it's more like learning what a pixel is.
You need to understand Git's data model, not its plumbing. No VCS will be useful for anything that isn't trivial if you don't understand its data model.