More than purple not existing, brown really doesn't exist. Brown is just reflected orange-y light in the right context. You can play around with it, by looking at a picture of say, earthy soil, color picking that pixel and painting it on a white canvas in a different way. Or by trying to shine an RGB light with that color code. It will look orange in a direct light.

Also the purple hallucination is because the same cone that mostly detects red also has a small peak futher along the spectrum next to blue. Since it slightly reacts to blue/violet light, you get that purple tickle.