Imagine that each grain is blocking a 'pixel' of the negative. For simplicity of this explanation, I'll pretend the grains are red, green, and blue. If there's a blue grain, only blue light can get through, and thus the luminance is baked in specific to the amount of blue in that region. The same is true for the red, and the green.
Thus, each 'pixel' of the negative isn't determining a monochromatic luminosity (ie black and white), but instead telling the luminosity of a single channel of red, green, or blue.
When the grain filter and the developed negative are combined, we get natural colour. The same filter is used for generation and display.
Another way to think of it is like this: Each pixel on our screen is actually 3 subpixels, each defining red, green, and blue, and we format our image to alternate between telling the red, green, and blue channels how bright to be. This is doing the same, but the subpixels are distributed randomly because that's much easier to make.
EDIT:
If you organise the grains in a geometric pattern, you get Dufaycolor[1].