Evaluation services · Private platform · Public research
Spring Prompt helps teams define what good looks like, test the AI inside their products on real work, improve weak systems and prove what is ready to ship.
Or start with the evidence: browse the public research →
Baseline vs candidate
Illustrative mini-eval · example data
Can the assistant answer refund questions without stepping outside policy?
Artifact: Reply to a refund request received outside the 30-day window
Output A · baseline
7/12 passed
Output B · candidate
11/12 passed
| Criterion | A | B |
|---|---|---|
| Policy statements accurate | 3/4 | 4/4 |
| No unauthorised promises | 1/3 | 3/3 |
| Required disclosures present | 2/3 | 3/3 |
| Tone and escalation path | 1/2 | 1/2 |
|
Critical failure
A: Offers a refund the policy does not allow
B: None
|
||
Decision: B clears the bar. A fails a critical policy check, so its average score is irrelevant.
Does the executive summary only make claims the client brief supports?
Artifact: Executive summary for an RFP response, grounded in the client brief
Output A · baseline
5/8 supported
Output B · candidate
7/8 supported
| Criterion | A | B |
|---|---|---|
| Claims supported by the brief | 3/5 | 5/5 |
| Figures match source documents | 1/2 | 1/2 |
| Scope matches requested deliverables | 1/1 | 1/1 |
|
Critical failure
A: Cites a delivery date the brief never states
B: None
|
||
Decision: B is supportable. Its one remaining miss is a rounded figure, routed to human review.
Does the hero copy say what the positioning brief says, and nothing more?
Artifact: Hero copy for a B2B pricing page from a positioning brief
Output A · baseline
3/6 met
Output B · candidate
6/6 met
| Criterion | A | B |
|---|---|---|
| Names the intended buyer | 1/1 | 1/1 |
| States the differentiator from the brief | 0/2 | 2/2 |
| Avoids unsubstantiated superlatives | 1/2 | 2/2 |
| Uses approved terminology | 1/1 | 1/1 |
|
Critical failure
A: Introduces a 'best in class' claim with no evidence in the brief
B: None
|
||
Decision: B meets all six criteria. A needs the differentiator rewritten before it can ship.
Can this configuration publish listings without inventing product facts?
Artifact: Marketplace listing generated from a structured SKU record
Output A · baseline
15/20 passed
Output B · candidate
19/20 passed
| Criterion | A | B |
|---|---|---|
| Product facts preserved | 7/8 | 8/8 |
| No unsupported claims | 3/6 | 5/6 |
| Required fields covered | 4/4 | 4/4 |
| Channel rules: length, banned terms | 1/2 | 2/2 |
|
Critical failure
A: States a fabric composition not in the source record
B: None
|
||
Decision: B is publishable across the tested scope. Its one unsupported claim routes to review rather than to the storefront.
Deterministic checks run first; model and human judgment are reserved for criteria that need them. A critical failure blocks a release regardless of the average.
One company, three ways in
Expert evaluation for custom, complex, sensitive or organisation-specific work: evaluation design, prompt and model comparison, failure analysis, optimisation, evaluator audits and release evidence.
First packaged service: Catalog Content Quality →A guided workspace for the repeatable parts: prompt analysis, test cases, model runs, failure inspection, baseline-versus-candidate comparison and decision reports. Private while the workflow is validated in real engagements.
What it does today, and what it does not →Original evaluations, benchmark explainers, model evidence and best-model-by-task pages. The same methodology sold privately, published with its scope, sources, dates and limitations.
Browse the public research →The shared method
Start with the decision, not the metric. Every engagement, platform workflow and public benchmark follows the same four moves, so evidence from one can be trusted in the others.
01
Turn the business objective into observable quality criteria: the target population, the slices that matter, the failure taxonomy, the source of truth and the critical failures an average score must never hide.
02
Compare prompts, models and configurations under equivalent conditions on real work. Deterministic checks first; model judgment and human review only where they change the decision.
03
Fix what the failures point at, not what the leaderboard suggests. Validate the change on held-back cases and convert every confirmed failure into a permanent regression test.
04
Deliver evidence a product team can act on: baseline versus candidate, uncertainty and unresolved cases kept visible, and a clear call to ship, reject, revise or investigate.
First packaged service · Available now
For teams adding AI generation to PIM, product-feed, marketplace and ecommerce applications. We evaluate the software that turns structured SKU data into titles, bullets, descriptions, channel listings and translations, and answer one question:
Which configuration can we safely deploy across the agreed catalog scope?
Not a product-description generator. An evaluation of the one you already run, or the one you are about to buy.
Read the service outlineUnsupported-claim rate, product-fact preservation and required-field coverage, checked against the structured record.
Adherence to brand, category and terminology rules, including required and prohibited terms.
Compliance with explicit marketplace and channel constraints: length, structure, banned phrasing, locale.
Publish-without-edit rate, edit distance and reviewer time, so cost per publishable record is visible.
Offline evaluation establishes publishability against your rules and source data. It does not, on its own, prove conversion or sales impact.
Platform · Private / developing
The platform lets teams run the repeatable workflow themselves while Spring Prompt still guides evaluation design. It stays private until real engagements show which steps are genuinely the same across companies.
The workflow is text-focused. It does not yet execute:
Public research
Public work demonstrates the methodology sold privately. Every page states what was measured by Spring Prompt, what was reported by an external source, and what is interpretation.
Family-level model entries and exact agent configurations are counted separately and never summed.
Which model to pick for a defined business task, with the evidence and its limits.
Per-family profiles: scores, confidence, provenance and what remains unsupported.
What external benchmarks actually measure, and how far a score travels into your task.
Spring Prompt experiments such as ROASBench and BulletBench, with method and artifacts.
Data sources, provenance, publication gates and known limitations.
Findings, evaluation practice and notes on building reliable AI products.
Writing fit, current standing
Full leaderboard →| # | Model | Writing fit | Blended $/1M |
|---|---|---|---|
| #1 | GPT 5.5OpenAI | 97 | $4.35 |
| #2 | GPT 5.6 SolOpenAI | 95 | $4.35 |
| #3 | Claude Opus 4.8Anthropic | 95 | $3.85 |
| #4 | Claude Fable 5Anthropic | 94 | $7.7 |
| #5 | GPT 5.4OpenAI | 86 | $2.17 |
Tie-aware standing across 2 predictive Writing tasks · source-native direct scores are excluded · price does not affect fit
Next step
Tell us what your product generates, what a bad output costs you, and the decision you cannot currently make. We will say whether an evaluation is the right next step and what it would need from you.
Talk to us about an evaluationhello@springprompt.com · replies come from the people who run the evaluations.
Access is granted in small groups while the workflow is validated. Tell us what you would evaluate first so we can match you to the right stage.
Request received.
EQ-Bench 3 by Samuel Paech / EQ-bench · source snapshot fetched 2026-07-16· Creative Writing v3 by Samuel Paech / EQ-bench · source snapshot fetched 2026-07-16· Contains data from the Arena Leaderboard Dataset by Arena, licensed under CC BY 4.0 · source snapshot fetched 2026-07-16· UGI Leaderboard by DontPlanToEnd, via Hugging Face Spaces · source snapshot fetched 2026-07-16· Operational price and performance data sourced from Artificial Analysis· Structured Output Benchmark (SOB) by Interfaze / JigsawStack, Inc. (MIT License)· OpenHands Index by the OpenHands contributors (Apache-2.0) · source snapshot fetched 2026-07-17· STATE-Bench by Microsoft and STATE-Bench contributors (MIT License) · source snapshot fetched 2026-07-20 See data sources & scoring methodology.
