-
Notifications
You must be signed in to change notification settings - Fork 31
FEAT: varbinarymax streaming support in execute() #231
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR adds support for streaming large binary parameters (bytes and bytearray) to SQL Server using VARBINARY(MAX), removing the previous 8192-byte limitation. The implementation introduces deferred execution (DAE) for binary data larger than 8000 bytes while maintaining direct binding for smaller data.
Key changes:
- Updated parameter type mapping to detect large binary parameters and use VARBINARY(MAX) with streaming
- Modified C++ parameter binding to handle both small (direct) and large (streamed) binary data
- Enhanced SQL execution to stream large binary data in chunks using SQLPutData
Reviewed Changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| mssql_python/cursor.py | Updated _map_sql_type to conditionally use VARBINARY(MAX) with DAE for binary data >8000 bytes |
| mssql_python/pybind/ddbc_bindings.cpp | Enhanced parameter binding and execution to support streaming large binary data |
| tests/test_004_cursor.py | Split binary data tests into separate small/large test cases |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
sumitmsft
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Left a few comments
Work Item / Issue Reference
Summary
This pull request adds support for streaming large binary parameters (bytes and bytearray) to SQL Server using VARBINARY(MAX), removing the previous limitation that prevented inserting binary data larger than 8192 bytes. The changes update both the Python and C++ layers to detect large binary parameters, use deferred execution (DAE) for streaming, and add new tests to verify correct handling of both small and large binary data.
Binary parameter streaming support:
_map_sql_typeincursor.pyto detect largebytes/bytearrayparameters (>8000 bytes) and mark them for VARBINARY(MAX) streaming, while still handling small binaries directly.ddbc_bindings.cppto use deferred execution for large binaries, and to allocate/bind buffers correctly for both small and large binary parameters.SQLExecute_wrapinddbc_bindings.cppto stream large binary data usingSQLPutDatain chunks during execution.Testing improvements:
test_004_cursor.pyto separately verify small/medium binary inserts and to add a new test for inserting large binary data (>8000 bytes) using streaming. [1] [2]