Pinned
TNG Technology Consulting GmbH
@tngtech
TNG, aka "The Nerd Group", is a consulting partnership focused on high end information technology, particularly AI. 935 employees, 99.9% academics, ~53% PhDs.
The 'Undercover Curve' of a Sleeper Agent training run We show the evolution of 'Secret Keeping' vs. 'Exfiltration Success'. Number labels indicate the train step. First, while the model learns to exfiltrate secrets, a strong decline in the capability of secret keeping can be- New and pretty good results for @Alibaba_Qwen's 3.8 version in @sam_paech's new EQ-Bench v4 benchmark. Qwen-3.8 feels like a "harmony-seeking advisor", being very validating (7.4), shying away from challenges (3.5) and not risking much boldness in interpretation 5.8). It
- Chinese battle zone in TLDR-bench, @Alibaba_Qwen, @deepseek_ai, @Kimi_Moonshot, @Zai_org. The preliminary numbers are a bit surprising.
- We calculated okayish, but not the greatest results for @deepseek_ai's V4F-0731 in @sam_paech's new EQ-Bench v4 benchmark. It is an interesting question whether this is a systematic effect of the new post training of 0731: How does heavy RL affect a model's personality, could it
