fix: do not spam the log with checksum related INFO messages when downloading using transfer_manager#1357
Conversation
|
🤖 I detect that the PR title and the commit message differ and there's only one commit. To use the PR title for the commit history, you can use Github's automerge feature with squashing, or use -- conventional-commit-lint bot |
cojenco
left a comment
There was a problem hiding this comment.
I think this is a reasonable patch to avoid passing in the default checksum md5 in the subsequent call blob._prep_and_do_download. @andrewsg thoughts/concerns to this change?
@rafalh Could you update the corresponding unit tests in https://github.com/googleapis/python-storage/blob/main/tests/unit/test_transfer_manager.py?
|
This makes sense to me. Thank you for your submission! |
…nloading using transfer_manager `download_chunks_concurrently` function does not allow to set `checksum` field in `download_kwargs`. It also does not set it on its own so it takes the default value of `"md5"` (see `Blob._prep_and_do_download`). Because ranged downloads do not return checksums it results in a lot of INFO messages (tens/hundreds): ``` INFO google.resumable_media._helpers - No MD5 checksum was returned from the service while downloading ... (which happens for composite objects), so client-side content integrity checking is not being performed. ``` To fix it set the `checksum` field to `None` which means no checksum checking for individual chunks. Note that `transfer_manager` has its own checksum checking logic (enabled by `crc32c_checksum` argument)
|
I fixed the tests and got them to pass locally (executed only |
cojenco
left a comment
There was a problem hiding this comment.
LGTM, thank you for working on this!
download_chunks_concurrentlyfunction does not allow to setchecksumfield indownload_kwargs. It also does not set it on its own so it takes the default value of"md5"(seeBlob._prep_and_do_download). Because ranged downloads do not return checksums it results in a lot of INFO messages (tens/hundreds):To fix it set the
checksumfield toNonewhich means no checksum checking for individual chunks. Note thattransfer_managerhas its own checksum checking logic (enabled bycrc32c_checksumargument)Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:
Fixes #1358 🦕