I suspect it's a combination of support, documentation, and trade secrets. I've worked with numerous vendors over the last decade and the support strategies vary dramatically. Some vendors put up very nice datasheets, manuals, source code, etc, while some give you almost nothing at all, and instead ship an engineer with the product to help you design it in. The rationale is that these chips are very complicated, applications vary dramatically, and it's sometimes better to increase the quantity of manpower (in engineer form) for your favorite 5 customers (that make up most of your market anyway) rather than to increase the quantity of documentation so that you can also sell to Joe & Jane Hobbyist.

I have no experience with GPUs, but I once read something that indicated that the driver, in fact, has a lot of proprietary code in it. For example, nVidia often releases driver updates after a new game is released that can give 10-25% performance boosts in those games. That's not because it's running the GPU faster, that's because of optimizations within the driver on how GPU commands get processed by the GPU.

Someone with more knowledge about this please correct me if I'm wrong.

"Isn't all the magic in the chip itself"

Yeah thats the problem.

There exists a patent troll who owns patents for using pink unicorns to render pixels. Vendor ships a chip that uses red horses to render pixels... close enough for a legal patent battle? Who knows. But if they try to keep it a secret, maybe it'll all work out.

What makes no sense to me is what's the risk for the vendor in letting someone redistribute the driver's binary blobs. Any competitor who wants to reverse-engineer the code can already extract it from the released product anyway!

What if the driver is only unlocking part of the GPU's capabilities? A vendor can spin one chip for n customers, all paying different royalty and licensing fees based on need.