In the field I work in, designers and developers have equal standing, and need to create a middle ground of shared workflow, technology and design processes that balance development and design.
In most open source projects, there are too many barriers to entry (real or perceived) for designers to see themselves as participants. Everything from version control systems that don't provide any real benefit for visual designers to project leaders that see design as "eye candy" all reduce the appeal to designers.
Remember, too, that designers have loads of options if they want to work for free. You know all those "Build a Facebook clone over the weekend, it will look great on your resume" jobs on Craigslist? Designers get that kind of pitch ("Design us something for free; it will look great in your portfolio") constantly. Open source projects looking to recruit designers need convey some tangible benefit to participating.
In the design world, peer recognition doesn't come out of working on open source projects that their designer colleagues have never heard of. Every designer I know of will jump at the opportunity to do really great work, for free if necessary. Looking at the open source projects I'm familiar with, few of these look like the kinds of places where a designer would expect to create really great work.