One thing the engineering community will (re)discover is that every program can be ① hardened (edge cases squashed, inputs tightened, errors handled) and ② optimized (profiled, benchmarked, rewritten) basically ad infinitum.
There are real costs in every direction: time,
“Thinking fast and s̶l̶o̶w̶ a bit less fast under a confidence threshold.”
A very simple feature that I think will be extremely impactful for at-scale AI decision-making.
Your agents can now escalate uncertain decisions to a fallback model.
AI Gateway reruns the decision on a fallback model when the primary model's confidence drops below your threshold. Available in beta.
