Skip to content

fix: sonic ast cache need clean after middleware#560

Merged
zijiren233 merged 5 commits into
labring:mainfrom
zijiren233:bodycache
May 13, 2026
Merged

fix: sonic ast cache need clean after middleware#560
zijiren233 merged 5 commits into
labring:mainfrom
zijiren233:bodycache

Conversation

@zijiren233

Copy link
Copy Markdown
Member

No description provided.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR addresses memory/cleanup concerns around Bytedance Sonic AST usage by switching to explicit GetWithOptions(..., ast.SearchOptions{}) parsing and by ensuring cached request-body AST nodes and request-body readers are properly released/cleared during relay handling.

Changes:

  • Replace multiple sonic.Get(...) calls with sonic.GetWithOptions(..., ast.SearchOptions{}) across plugins/adaptors to avoid unwanted AST caching behavior.
  • Ensure converted outbound request bodies are closed in prepareAndDoRequest, including early-error paths; add regression tests.
  • Change request-body caching to use a reusable request body wrapper and explicitly clear the per-request cached AST node key in middleware.

Reviewed changes

Copilot reviewed 15 out of 15 changed files in this pull request and generated no comments.

Show a summary per file
File Description
core/relay/plugin/web-search/search.go Use GetWithOptions when parsing response JSON for metadata injection.
core/relay/plugin/thinksplit/split.go Use GetWithOptions for safer streaming chunk parsing; add ast import.
core/relay/plugin/streamfake/fake.go Use GetWithOptions when parsing streaming chunks into AST nodes.
core/relay/plugin/cachefollow/cachefollow.go Use GetWithOptions for parsing retention fields from SSE/JSON.
core/relay/controller/dohelper.go Close converted request bodies reliably; refine body-detail truncation helpers.
core/relay/controller/dohelper_test.go Add tests asserting converted request bodies are closed on success/failure paths.
core/relay/adaptor/siliconflow/adaptor.go Use IsSuccessfulResponseStatus for mode-aware success codes.
core/relay/adaptor/openai/stt.go Use GetWithOptions when injecting usage into SSE JSON payloads.
core/relay/adaptor/openai/image.go Use GetWithOptions for parsing image stream SSE JSON payloads.
core/relay/adaptor/openai/chat.go Use GetWithOptions for parsing chat stream SSE JSON payloads.
core/relay/adaptor/anthropic/main.go Use GetWithOptions for parsing Claude stream SSE payloads.
core/middleware/distributor.go Clear cached request-body AST node before handing off to downstream handlers.
core/middleware/ctxkey.go Make request-body-node cache key internal (requestBodyNode).
core/common/body.go Rework request body caching via reusable body wrapper; use GetWithOptions for node extraction.
core/common/body_test.go Update expectations to match new request body caching behavior (no GetBody).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@zijiren233
zijiren233 merged commit b331a82 into labring:main May 13, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants