The vault/proxy layer solving the "2am paste" vector but not the semantic leakage is exactly the gap most teams don't account for. Ephemeral key naming, endpoint patterns, TTL behaviors -- all of this is in the training corpus and no amount of runtime secret rotation changes what the model already internalized. You've essentially found that your threat model stopped at input hygiene but the model itself is a side channel.

What I'd add to the defense stack: structured output validation that flags known internal naming patterns before they reach the client, plus anomaly detection on response metadata -- token budget shifts, refusal rates, response shape changes under semantic pressure. At 75% convergence you're past "interesting research" into "reliable extraction technique," which means you need a detection layer, not just prevention.

Have you tested this against non-OpenAI models like Claude or Gemini to see if the naming bleed is GPT-4o-specific or a broader training corpus problem?