It might not be racist, but could be ideologically charged.
The mere observation of an uneven distribution is not enough evidence for a power imbalance. It could also just be caused by deviation of interests, which would be totally fine.
Power imbalances cannot be reliably detected from the outside, so I won't comment on the ruby conf case.