Skip to content

fix(knowledge): preserve live document processing during recovery - #7993

Merged
waleedlatif1 merged 4 commits into
stagingfrom
codex/processing-recovery-diagnostics
Sep 19, 2026
Merged

waleedlatif1 merged 4 commits into
stagingfrom
codex/processing-recovery-diagnostics

Conversation

@waleedlatif1

@waleedlatif1 waleedlatif1 commented Sep 18, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Extend the existing recovery guard with bounded database and cancellable Trigger checks, preserve work when status is unknown, and recheck the full observed generation before resetting it.
  • Apply the same protection to scheduled recovery, connector sweeps, manual retry, and legacy queue redelivery.
  • Add sanitized OCR response and Google token-transport diagnostics without changing provider retries, permission checks, or public errors.

Type of Change

  • Bug fix

Testing

  • 912 focused unit tests and 44 disposable PostgreSQL/Redis integration tests passed, including queue delays, generation races, legacy upload redelivery, durable delivery, lifecycle guards, SDK request cancellation, database acquisition deadlines, and diagnostic redaction.
  • Full monorepo type-check, lint and lint:check, 47 audits, API boundary validation, block registry and docs-manifest checks passed.
  • Cleanup and final independent reviews completed. No schema migration is required. Live verification after deployment remains necessary; this does not reset exhausted attempts or resolve provider access and database infrastructure failures.

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

@vercel

vercel Bot commented Sep 18, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated
Image docs Skipped Skipped Sep 19, 2026 12:25am UTC

Request Review

@greptile-apps

greptile-apps Bot commented Sep 18, 2026

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 5/5

The PR appears safe to merge; the previously reported regressions are resolved and no new actionable failure remains.

Summary

This PR protects live or indeterminate document-processing generations from being replaced during recovery and applies that protection consistently across scheduled, connector, manual retry, and legacy redelivery paths. It also makes worker outcomes explicit and adds sanitized provider diagnostics.

  • Rechecks durable outbox and Trigger.dev liveness before replacing a processing generation.
  • Uses snapshot guards and bounded inspection deadlines to prevent races with concurrent lifecycle transitions.
  • Preserves live work through connector sweeps, manual retries, and legacy outbox redelivery.
  • Distinguishes indexed documents from harmless unavailable, unclaimed, or superseded task outcomes.
  • Logs bounded, allowlisted OCR diagnostics without exposing provider messages or document contents.
Diagram
%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Recovery candidate] --> B{Active outbox carrier?}
  B -->|Yes| C[Preserve generation and defer recovery]
  B -->|No| D{Active Trigger.dev run?}
  D -->|Yes| C
  D -->|Lookup unavailable| E[Preserve generation and schedule short recheck]
  D -->|No active work| F[Recheck document snapshot]
  F -->|Snapshot changed| G[Leave winning generation untouched]
  F -->|Snapshot unchanged| H[Adopt or replace generation]
  H --> I[Enqueue durable processing event]
  I --> J[Process document]
  J -->|Index committed| K[Return indexed outcome]
  J -->|Unavailable, unclaimed, or superseded| L[Return harmless skipped outcome]
Loading

Reviews (4) · Last reviewed commit: "fix(knowledge): bound recovery database ..."

Comment thread apps/sim/lib/knowledge/documents/service.ts

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 16 files

Reply with feedback, questions, or to request a fix.

Fix all with cubic | Re-trigger cubic

Comment thread apps/sim/lib/knowledge/documents/service.ts
Comment thread apps/sim/lib/knowledge/documents/processing-liveness.ts Outdated
@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Sep 18, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

Comment thread apps/sim/lib/knowledge/documents/processing-liveness.test.ts Outdated

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 16 files

Reply with feedback, questions, or to request a fix.

Fix all with cubic | Re-trigger cubic

Comment thread apps/sim/lib/knowledge/documents/service.ts
@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Sep 19, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 17 files

Reply with feedback, questions, or to request a fix.

Fix all with cubic | Re-trigger cubic

Comment thread apps/sim/lib/knowledge/documents/processing-liveness.ts Outdated
@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Sep 19, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 17 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Re-trigger cubic

@waleedlatif1
waleedlatif1 merged commit 99d6a9f into staging Sep 19, 2026
35 checks passed
@waleedlatif1
waleedlatif1 deleted the codex/processing-recovery-diagnostics branch September 19, 2026 00:37

This branch was previously deployed

1 inactive deployment
Preview f211b908 Deployed Sep 19, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant