Log inSign up
user avatar
Ahmad Beirami
@abeirami
Co-founder / CEO @fidian
{NYC, SFO, YYZ}
fidian.ai
Joined December 2018
3,082
Following
12.1K
Followers
1
Subscription
RepliesArticlesMedia
  • Pinned
    user avatar
    Ahmad Beirami
    @abeirami
    Jul 27
    We’re releasing Terminal-Bench 2 Fidian Edition (TB2-Fn), a rebuild of the entire dataset: The rebuilt tasks corrected false negatives and false positives in the original dataset. The new take are also made harder for the frontier models and push the frontier of agent
    user avatar
    Ali Parandeh Gheibi
    @aparandehgheibi
    Jul 27
    We’re excited to release TB2-Fn. TB2-Fn is a variant of Terminal Bench 2 (TB2) with 89 rewritten tasks designed to verify essentially the same underlying skills as their TB2 counterparts. We found that agents could pass seven of the 89 original TB2 tasks without actually
    Image
  • user avatar
    Ahmad Beirami
    @abeirami
    Aug 3
    Fable 5 and GPT-5.6 Sol can now produce correct mathematical reasoning chains over a long horizon, on demand. That is what makes them special for advancing algorithmic science and math. Most benchmarks measure the step or a few steps, not a very long horizon of steps, which is
  • user avatar
    Ahmad Beirami
    @abeirami
    Aug 2
    In case you are wondering, my musings on best-of-n bounds over the weekend were done using a simple coding harness that creates sessions with each of the models (here, Sol and Fable) and passes context across them. My total supervision time was a few minutes, and my input was
  • user avatar
    Ahmad Beirami
    @abeirami
    Aug 2
    With essentially zero technical input from me, GPT-5.6 Sol and Fable 5 not only proved a conjecture we left open ~2 years ago on best-of-n, but also delivered a strictly tighter bound with a clean and insightful derivation. We are officially in a new era of mathematical
    user avatar
    Ahmad Beirami
    @abeirami
    Aug 2
    This got even more ridiculous! I was trying to use the context of this session to nudge Sol to improve another result. Instead, it misunderstood me as wanting to improve this current bound and voila🤣
    Image
    Image
  • user avatar
    Ahmad Beirami
    @abeirami
    Aug 2
    This got even more ridiculous! I was trying to use the context of this session to nudge Sol to improve another result. Instead, it misunderstood me as wanting to improve this current bound and voila🤣
    Image
    Image
    Image
    Image
    user avatar
    Ahmad Beirami
    @abeirami
    Aug 1
    Here is the proof. My entire contribution was to prompt a couple of times to make it clear, concise, self-contained, and extract the key non-trivial step as a separate lemma.