Can We Trust Open Source Models for Production Code? Eval Results

Enterprises are moving to open-weight models on open harnesses, mostly for one reason: token cost. Claude Code is excellent and very easy to exhaust in a day. But most large enterprise still reach for a Western model, because Chinese models feel unsafe in a way nobody has quantified.

Since my job is software engineering and not geopolitics, and trying to be rational as well as pragmatic here, while giving weight to democracy and human values over autocracy and repression — why not devise a test to check this out? That was my aim and I vibecoded this Eval framework. Turns out the political bias is real and as expected, but it does not leak into code or security when asked from any perspective