Skip to content

Conversation

@gargsaumya
Copy link
Contributor

@gargsaumya gargsaumya commented Sep 9, 2025

Work Item / Issue Reference

AB#33395

GitHub Issue: #<ISSUE_NUMBER>


Summary

This pull request adds support for streaming large binary parameters (bytes and bytearray) to SQL Server using VARBINARY(MAX), removing the previous limitation that prevented inserting binary data larger than 8192 bytes. The changes update both the Python and C++ layers to detect large binary parameters, use deferred execution (DAE) for streaming, and add new tests to verify correct handling of both small and large binary data.

Binary parameter streaming support:

  • Updated _map_sql_type in cursor.py to detect large bytes/bytearray parameters (>8000 bytes) and mark them for VARBINARY(MAX) streaming, while still handling small binaries directly.
  • Modified parameter binding in ddbc_bindings.cpp to use deferred execution for large binaries, and to allocate/bind buffers correctly for both small and large binary parameters.
  • Enhanced SQLExecute_wrap in ddbc_bindings.cpp to stream large binary data using SQLPutData in chunks during execution.

Testing improvements:

  • Refactored and expanded tests in test_004_cursor.py to separately verify small/medium binary inserts and to add a new test for inserting large binary data (>8000 bytes) using streaming. [1] [2]

Copilot AI review requested due to automatic review settings September 9, 2025 12:03
@github-actions github-actions bot added the pr-size: medium Moderate update size label Sep 9, 2025
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adds support for streaming large binary parameters (bytes and bytearray) to SQL Server using VARBINARY(MAX), removing the previous 8192-byte limitation. The implementation introduces deferred execution (DAE) for binary data larger than 8000 bytes while maintaining direct binding for smaller data.

Key changes:

  • Updated parameter type mapping to detect large binary parameters and use VARBINARY(MAX) with streaming
  • Modified C++ parameter binding to handle both small (direct) and large (streamed) binary data
  • Enhanced SQL execution to stream large binary data in chunks using SQLPutData

Reviewed Changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File Description
mssql_python/cursor.py Updated _map_sql_type to conditionally use VARBINARY(MAX) with DAE for binary data >8000 bytes
mssql_python/pybind/ddbc_bindings.cpp Enhanced parameter binding and execution to support streaming large binary data
tests/test_004_cursor.py Split binary data tests into separate small/large test cases

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

@github-actions github-actions bot added pr-size: medium Moderate update size and removed pr-size: medium Moderate update size labels Sep 9, 2025
@microsoft microsoft deleted a comment from Copilot AI Sep 9, 2025
Copy link
Contributor

@sumitmsft sumitmsft left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left a few comments

@github-actions github-actions bot added pr-size: medium Moderate update size and removed pr-size: medium Moderate update size labels Sep 15, 2025
@github-actions github-actions bot added pr-size: medium Moderate update size and removed pr-size: medium Moderate update size labels Sep 15, 2025
@gargsaumya gargsaumya merged commit 61ed764 into main Sep 15, 2025
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pr-size: medium Moderate update size

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants