fix(file-search): bound chunk inserts by trigram work

[icecrasher321]

Summary

• Bound each chunk insert by estimated trigram posting work as well as rows and bytes. Byte limits alone let dense text produce oversized GIN updates and hit the statement deadline.
• Share the batching policy with the storage guard. Preserve complete-file publication, chunk boundaries, exact/regex behavior, line numbers, and existing queue limits; a maximum-key chunk makes progress in its own batch.

Type of Change

• Bug fix

Testing

• 671 unit tests; 65 PostgreSQL integration tests passed on both PostgreSQL 16 and 17 (one optional benchmark skipped).
• Synthetic ten-writer comparison: byte-only batches exceeded 10 seconds; key-budget batches completed all input with a 0.87-second maximum. A larger index exceeding the configured shared cache also completed all input, with a 2.82-second p95 and 5.00-second maximum. These are local stress results, not production capacity guarantees.
• Full lint, app type check, all 47 audits, block-registry check, and docs-manifest check.

Checklist

• Code follows project style guidelines
• Self-reviewed my changes
• Tests added/updated and passing
• No new warnings introduced
• I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment

Project	Deployment	Actions	Updated
docs	Skipped		Sep 19, 2026 5:53pm UTC

[greptile-apps]

Confidence Score: 5/5

The PR appears safe to merge; no actionable correctness, security, or repository-rule violations were identified.

Summary

This PR adds trigram-work-aware batching to workspace file indexing while retaining the existing row and byte limits.

• Introduces a shared batch planner and storage-side budget validation.
• Preserves chunk boundaries, complete-file publication, line metadata, and cancellation behavior.
• Adds unit and PostgreSQL integration coverage for dense text, oversized batches, lookahead, and exact/regex results.
• Documents the new batching policy and its operational limitations.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Extract and plan file text] --> B[Generate bounded chunks]
  B --> C[Estimate distinct trigram keys per chunk]
  C --> D{Would next chunk exceed rows, bytes, or key target?}
  D -- Yes --> E[Insert current batch]
  D -- No --> F[Add chunk to current batch]
  F --> D
  E --> G[Storage guard recomputes all budgets]
  G --> H{Build token and lease valid?}
  H -- No --> I[Stop without publication]
  H -- Yes --> J[Insert chunk rows]
  J --> K{More chunks?}
  K -- Yes --> C
  K -- No --> L[Verify stored chunk count]
  L --> M[Publish complete build]

Loading

_{Reviews (1) · Last reviewed commit: "fix(file-search): bound chunk inserts by..."}

[cubic-dev-ai]

No issues found across 7 files

Confidence score: 5/5

• Automated review surfaced no issues in the provided summaries.
• No files require special attention.

_{Re-trigger cubic}