Skip to content

Conversation

@BrianMichell
Copy link
Collaborator

Resolves #626 by applying appropriate logic.

Tested that code below sets the amplitude Variable chunk size to (64, 64, 1501) where 1501 is discovered at runtime.

if __name__ == "__main__":
    from mdio import segy_to_mdio
    from segy.schema import HeaderField, ScalarType
    from segy.standards import get_segy_standard
    from mdio.builder.template_registry import get_template
    import os

    os.environ["MDIO__IMPORT__SAVE_SEGY_FILE_HEADER"] = "1"
    os.environ["MDIO__IMPORT__CPU_COUNT"] = "16"

    input_segy = "filt_mig.sgy"
    output_mdio = "filt_mig.mdio"

    input_spec = get_segy_standard(1)
    input_fields = [
        HeaderField(name="inline", byte=17, format=ScalarType.INT32),
        HeaderField(name="crossline", byte=13, format=ScalarType.INT32),
        HeaderField(name="cdp_x", byte=81, format=ScalarType.INT32),
        HeaderField(name="cdp_y", byte=85, format=ScalarType.INT32),
    ]
    input_spec = input_spec.customize(trace_header_fields=input_fields)

    template = get_template("PostStack3DTime")
    template.full_chunk_shape = (64, 64, -1)

    segy_to_mdio(
        segy_spec=input_spec,
        mdio_template=template,
        input_path=input_segy,
        output_path=output_mdio,
        overwrite=True,
    )

@BrianMichell BrianMichell added the enhancement New feature or request label Oct 17, 2025
@codecov
Copy link

codecov bot commented Oct 17, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 87.01%. Comparing base (c3ba558) to head (c1cfc89).
⚠️ Report is 205 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #723      +/-   ##
==========================================
+ Coverage   85.30%   87.01%   +1.70%     
==========================================
  Files          46       86      +40     
  Lines        2219     4812    +2593     
  Branches      306      320      +14     
==========================================
+ Hits         1893     4187    +2294     
- Misses        281      553     +272     
- Partials       45       72      +27     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@BrianMichell BrianMichell requested a review from tasansal October 20, 2025 13:10
@tasansal tasansal force-pushed the add_full_dim_chunking branch from 969cc01 to c96730a Compare October 20, 2025 15:57
@tasansal
Copy link
Contributor

tasansal commented Oct 20, 2025

@BrianMichell is this needed at all? What happens today on main if you specify chunks as (..., -1). Upstream libraries typically can handle -1 chunk size natively.

@BrianMichell
Copy link
Collaborator Author

@BrianMichell is this needed at all? What happens today on main if you specify chunks as (..., -1). Upstream libraries typically can handle -1 chunk size natively.

Traceback (most recent call last):
  File "test.py", line 24, in <module>
    template.full_chunk_shape = (64, 64, -1)
    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "site-packages/mdio/builder/templates/base.py", line 158, in full_chunk_shape
    raise ValueError(msg)
ValueError: Chunk shape (64, 64, -1) does not match dimension sizes ()

This PR fixes the error. Error generated with pypi version 1.0.8

@tasansal tasansal force-pushed the add_full_dim_chunking branch from c96730a to 58545bb Compare October 20, 2025 18:23
@tasansal tasansal force-pushed the add_full_dim_chunking branch from 58545bb to c1cfc89 Compare October 25, 2025 20:56
@tasansal tasansal changed the title Implement full-dimension chunking support Implement full-dimension chunking support in Templates Oct 25, 2025
@tasansal tasansal merged commit 8c40d61 into TGSAI:main Oct 25, 2025
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Allow chunk_size run-time override with template schemas

2 participants