improvement(knowledge): count an unproven reach, read memos under the search deadline, and drop the dead search paths - #8101

waleedlatif1 · 2026-09-21T07:37:02Z

Summary

A resolved scope's reach was assumed broad whenever the bases were small enough to fit the probe limit, even though no probe had saturated; the shortcut now applies only after a saturated probe, and an unproven reach is counted
The reach denominator estimate and the source-index catalog read ran outside the search's deadline on a memo miss; both run under the leg's budget
A page whose identity read recovered fewer rows than its slice ended the candidate stream early; a short page is now judged by what the pool handed over, not by what the read returned
The keyword leg re-read every base's kind per search to pick the Tin engine; the search carries whether its bases are search indexes, and the rollout flag and index readiness are read in parallel
The v1 route re-read each result's document for its name and URL, which hydration already puts on the row; that read, its helper and the retrieval result's read-access field are removed
Input provenance is prepared alongside the other pre-model reads, result provenance is imported per document in parallel, projection fill and the source-index memo are read together, and a reordered leg sorts once at the end instead of after every page
Removed dead code: unread getQueryStrategy fields, a never-emitted authorization stage, a one-valued diagnostic, the unused live-source current getter, the unread usage-admission policy field, the detached backfill enqueue path the script never took, and a tautological Tin guard

Type of Change

Improvement

Testing

vitest over lib/knowledge, app/api/knowledge, app/api/v1/knowledge, app/api/v2/knowledge, lib/copilot/tools/server/knowledge, lib/billing, scripts: 297 files, 4513 tests passing
New tests for the counted reach, the short-page rule and the planned walk's tag filter; each was verified to fail with its fix reverted
bun run lint, bun run check:audits (47 audits), docs-manifest:check, block-registry audit, native tsc --noEmit all pass

Checklist

Code follows project style guidelines
Self-reviewed my changes
Tests added/updated and passing
No new warnings introduced
I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

vercel · 2026-09-21T07:37:06Z

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment

Project	Deployment	Actions	Updated
docs	Skipped		Sep 21, 2026 7:51am UTC

… search deadline, and drop the dead search paths Correctness - A resolved scope's reach was assumed broad whenever the bases were small enough for the probe limit, without a probe having saturated; the shortcut now applies only to a saturated probe, and an unproven reach is counted - The reach denominator and the source-index catalog were read outside the search's deadline on a memo miss; both run under the leg's budget now - A page whose identity read recovered fewer rows than its slice ended the candidate stream early; short is judged by what the pool gave, not by what the read returned Round trips - The keyword leg re-read every base's kind per search to decide the Tin engine; the search carries whether its bases are search indexes, and the flag and index readiness are read together - The v1 route re-read each result's document for its name and URL, which the rows already carry from hydration; the read, its helper and the retrieval result's read-access field are gone - Input provenance is prepared alongside the other pre-model reads; result provenance is imported per document in parallel; projection fill and the source-index memo are read together - A reordered leg sorts once at the end instead of re-sorting the result map after every page Dead code - `getQueryStrategy` fields nothing read, a never-emitted authorization stage, a one-valued candidate-storage diagnostic, the unused live-source `current` getter, the unread usage-admission policy field, the detached enqueue path the script never took, and a tautological Tin guard

cubic-dev-ai

All reported issues were addressed

_{Reply with feedback, questions, or to request a fix.

Fix all with cubic | Re-trigger cubic}

greptile-apps · 2026-09-21T07:41:23Z

Confidence Score: 5/5

The PR appears safe to merge with no outstanding correctness, security, or repository-rule violations.

Summary

This PR streamlines access-scoped knowledge retrieval and keeps planning work within search deadlines.

Corrects reach estimation, candidate pagination, and final result ordering.
Reuses known search-index state and hydrated document metadata to avoid redundant reads.
Runs source-index and reach-estimation reads under retrieval budgets.
Parallelizes independent provenance and projection work.
Removes unused diagnostics, result fields, planner fields, and backfill paths.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  Request[Knowledge search request] --> Scope[Resolve bases and access scope]
  Scope --> Plan[Plan permitted reach under deadline]
  Plan --> Vector[Vector retrieval]
  Plan --> Keyword[Keyword retrieval]
  Vector --> Hydrate[Access-scoped hydration]
  Keyword --> Hydrate
  Hydrate --> Rank[Final ordering and result limit]
  Rank --> Metadata[Use hydrated source metadata]
  Metadata --> Response[API response]

_{Reviews (3) · Last reviewed commit: "fix(knowledge): remember a saturated rea..."}

…ded, and read the Tin flag before the index

waleedlatif1 · 2026-09-21T07:51:16Z

@greptile

waleedlatif1 · 2026-09-21T07:51:17Z

@cubic-dev-ai review this PR

cubic-dev-ai · 2026-09-21T07:51:25Z

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

cubic-dev-ai

No issues found across 16 files

Confidence score: 5/5

Automated review surfaced no issues in the provided summaries.
No files require special attention.

_{Re-trigger cubic}

waleedlatif1 · 2026-09-21T07:56:16Z

@greptile

waleedlatif1 · 2026-09-21T07:56:17Z

@cubic-dev-ai review this PR

cubic-dev-ai · 2026-09-21T07:56:22Z

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

cubic-dev-ai

No issues found across 16 files

Confidence score: 5/5

Automated review surfaced no issues in the provided summaries.
No files require special attention.

_{Re-trigger cubic}

vercel Bot temporarily deployed to Preview September 21, 2026 07:37 Inactive

waleedlatif1 force-pushed the chore/search-final-sweep branch from 4eda899 to 91a04d4 Compare September 21, 2026 07:37

vercel Bot temporarily deployed to Preview September 21, 2026 07:37 Inactive

cubic-dev-ai Bot reviewed Sep 21, 2026

View reviewed changes

Comment thread apps/sim/lib/knowledge/search/queries.ts

greptile-apps Bot reviewed Sep 21, 2026

View reviewed changes

Comment thread apps/sim/app/api/v1/knowledge/search/route.ts

Comment thread apps/sim/lib/knowledge/search/tin-keyword.ts Outdated

fix(knowledge): remember a saturated reach only once its count succee…

166a6d1

…ded, and read the Tin flag before the index

vercel Bot temporarily deployed to Preview September 21, 2026 07:51 Inactive

cubic-dev-ai Bot reviewed Sep 21, 2026

View reviewed changes

greptile-apps Bot reviewed Sep 21, 2026

View reviewed changes

Comment thread apps/sim/lib/knowledge/search/tin-keyword.ts

cubic-dev-ai Bot reviewed Sep 21, 2026

View reviewed changes

waleedlatif1 merged commit acc6cbd into staging Sep 21, 2026
34 checks passed

waleedlatif1 deleted the chore/search-final-sweep branch September 21, 2026 15:51

waleedlatif1 mentioned this pull request Sep 21, 2026

v0.8.49: knowledge search improvements, tables perf improvements #8108

Merged

Conversation

waleedlatif1 commented Sep 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Type of Change

Testing

Checklist

Uh oh!

vercel Bot commented Sep 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cubic-dev-ai Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

greptile-apps Bot commented Sep 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Confidence Score: 5/5

Summary

Uh oh!

Uh oh!

Uh oh!

waleedlatif1 commented Sep 21, 2026

Uh oh!

waleedlatif1 commented Sep 21, 2026

Uh oh!

cubic-dev-ai Bot commented Sep 21, 2026

Uh oh!

cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

waleedlatif1 commented Sep 21, 2026

Uh oh!

waleedlatif1 commented Sep 21, 2026

Uh oh!

cubic-dev-ai Bot commented Sep 21, 2026

Uh oh!

cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

This branch was previously deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

waleedlatif1 commented Sep 21, 2026 •

edited

Loading

vercel Bot commented Sep 21, 2026 •

edited

Loading

cubic-dev-ai Bot left a comment •

edited

Loading

greptile-apps Bot commented Sep 21, 2026 •

edited

Loading