feat: OpenAI responses API: Add store and previous_response_id params

[Rana Singh (ranadeepsingh)]

What changes are proposed in this pull request?

This PR adds support for the store and previousResponseId parameters in the OpenAI Responses API for OpenAIResponses and OpenAIPrompt transformers.

New Parameters:

store (Boolean, default: false)

• When set to true, the generated model response is stored for later retrieval via API
• In OpenAIPrompt, when store=true and apiType="responses", the output becomes a struct containing both response and id fields

previousResponseId / previousResponseIdCol (String)

• Allows chaining responses together for multi-turn conversations
• setPreviousResponseId(String) - scalar value for all rows
• setPreviousResponseIdCol(String) - per-row values from a DataFrame column

Output Structure Changes for OpenAIPrompt:

Settings	Output Type
Default	string (just the response text)
store=true	struct{response, id}
returnUsage=true	struct{response, usage}
store=true + returnUsage=true	struct{response, usage, id}

Files Changed:

• cognitive/src/main/scala/.../openai/OpenAIResponses.scala - Added store and previousResponseId parameters to HasOpenAITextParamsResponses trait
• cognitive/src/main/scala/.../openai/OpenAIPrompt.scala - Added store and previousResponseId parameters, modified output structure to include id when store=true, added validation for Responses API parameters
• cognitive/src/test/scala/.../openai/OpenAIResponsesSuite.scala - Added tests for new parameters
• cognitive/src/test/scala/.../openai/OpenAIPromptSuite.scala - Added tests for new parameters and output structure
• docs/Explore Algorithms/OpenAI/OpenAI.ipynb - Added usage examples

How is this patch tested?

• I have written tests (not required for typo or doc fix) and confirmed the proposed feature/bug-fix/change works.

Tests added:

• OpenAIResponsesSuite:

  • store parameter should be included in optional params when set
  • store parameter defaults to false
  • previousResponseId parameter should be included in optional params when set
  • Basic Usage with store=true returns response with id

• OpenAIPromptSuite:

  • store parameter defaults to false
  • store parameter can be set
  • previousResponseId parameter can be set
  • previousResponseIdCol parameter can be set
  • store=true with Responses API returns struct with response and id
  • store=true with returnUsage=true returns struct with response, usage, and id
  • store=false does not change output structure
  • store parameter throws error when apiType is not responses
  • previousResponseId parameter throws error when apiType is not responses
  • previousResponseIdCol parameter throws error when apiType is not responses

Validation:

• Added validateResponsesApiParams() method to ensure store and previousResponseId parameters are only used with apiType="responses"
• Throws IllegalArgumentException with clear error messages when these parameters are used with other API types
• All tests pass (35 tests in OpenAIPromptSuite, 15 tests in OpenAIResponsesSuite)
• Scalastyle checks pass with 0 errors and 0 warnings

Does this PR change any dependencies?

• No. You can skip this section.

Does this PR add a new feature? If so, have you added samples on website?

• Yes. Make sure you have added samples following below steps.

Examples have been added to docs/Explore Algorithms/OpenAI/OpenAI.ipynb demonstrating:

1. OpenAIResponses with store=true and response chaining via previousResponseIdCol
2. OpenAIPrompt with store=true and response chaining via previousResponseIdCol

[github-actions]

Hey Rana Singh (@ranadeepsingh) 👋!
Thank you so much for contributing to our repository 🙌.
Someone from SynapseML Team will be reviewing this pull request soon.

We use semantic commit messages to streamline the release process.
Before your pull request can be merged, you should make sure your first commit and PR title start with a semantic prefix.
This helps us to create release messages and credit you for your hard work!

Examples of commit messages with semantic prefixes:

• fix: Fix LightGBM crashes with empty partitions
• feat: Make HTTP on Spark back-offs configurable
• docs: Update Spark Serving usage
• build: Add codecov support
• perf: improve LightGBM memory usage
• refactor: make python code generation rely on classes
• style: Remove nulls from CNTKModel
• test: Add test coverage for CNTKModel

To test your commit locally, please follow our guild on building from source.
Check out the developer guide for additional guidance on testing your change.

[Rana Singh (ranadeepsingh)]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[Copilot]

Pull request overview

This PR adds support for the store and previousResponseId parameters to enable response storage and multi-turn conversation chaining in the OpenAI Responses API. The implementation is backward-compatible with store defaulting to false, and the output structure dynamically adjusts based on parameter settings.

Key Changes:

• Added store (Boolean) and previousResponseId (String) parameters to both OpenAIResponses and OpenAIPrompt transformers
• Modified OpenAIPrompt output structure to conditionally include response id when store=true with the Responses API
• Extended output schema logic to handle four combinations: response-only, response+usage, response+id, and response+usage+id

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.

File	Description
cognitive/src/main/scala/.../openai/OpenAIResponses.scala	Added store and previousResponseId parameters to HasOpenAITextParamsResponses trait with appropriate getters/setters and included in shared params
cognitive/src/main/scala/.../openai/OpenAIPrompt.scala	Added store and previousResponseId parameters, refactored output column building to conditionally include response ID, and added ServiceParam import
cognitive/src/test/scala/.../openai/OpenAIResponsesSuite.scala	Added unit tests for store default, optional params inclusion, previousResponseId params, and integration test for store=true returning response with ID
cognitive/src/test/scala/.../openai/OpenAIPromptSuite.scala	Added comprehensive tests covering parameter defaults, setters, and three integration scenarios testing different output structure combinations

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Codecov Comments Bot (codecov-commenter)]

Codecov Report

❌ Patch coverage is 98.38710% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 84.37%. Comparing base (bcf251c) to head (8aeb16b).

Files with missing lines	Patch %	Lines
...e/synapse/ml/services/openai/OpenAIResponses.scala	94.44%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #2460      +/-   ##
==========================================
+ Coverage   84.36%   84.37%   +0.01%     
==========================================
  Files         335      335              
  Lines       17734    17782      +48     
  Branches     1602     1600       -2     
==========================================
+ Hits        14961    15004      +43     
- Misses       2773     2778       +5     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[Rana Singh (ranadeepsingh)]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[Rana Singh (ranadeepsingh)]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[Rana Singh (ranadeepsingh)]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[Rana Singh (ranadeepsingh)]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[Rana Singh (ranadeepsingh)]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[Rana Singh (ranadeepsingh)]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[Brendan Walsh (BrendanWalsh)]

Rana Singh (@ranadeepsingh) outputDataType?

[Rana Singh (ranadeepsingh)]

Rana Singh (@ranadeepsingh) outputDataType?

Doing in the next follow up PR