GEMPAK performance updates #3431
Conversation
|
I'm ok with the change. Can you take a look and see if there is a major file type or scenario here that's not covered in our tests? It looks like 30% of the changes here don't get run by the tests. |
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
|
|
||
| def test_merged_sounding_no_packing(): | ||
| """Test loading a merged sounding without data packing.""" | ||
| gso = GempakSounding(get_test_data('gem_merged_nopack.snd')).snxarray( |
Check warning
Code scanning / CodeQL
File is not always closed
| gsped = gso[0].sped.values.squeeze() | ||
| ghght = gso[0].hght.values.squeeze() | ||
|
|
||
| gempak = pd.read_csv(get_test_data('gem_merged_nopack.csv', as_file_obj=False), |
Check warning
Code scanning / CodeQL
File is not always closed
|
|
||
| def test_climate_surface(): | ||
| """Test to read a cliamte surface file.""" | ||
| gsf = GempakSurface(get_test_data('gem_climate.sfc')) |
Check warning
Code scanning / CodeQL
File is not always closed
| gsf = GempakSurface(get_test_data('gem_climate.sfc')) | ||
| gstns = gsf.sfjson() | ||
|
|
||
| gempak = pd.read_csv(get_test_data('gem_climate.csv', as_file_obj=False)) |
Check warning
Code scanning / CodeQL
File is not always closed
| @pytest.mark.parametrize('order', ['little', 'big']) | ||
| def test_byte_swap(order): | ||
| """"Test byte swapping.""" | ||
| g = get_test_data(f'gem_{order}_endian.grd') |
Check warning
Code scanning / CodeQL
File is not always closed
Use `set` instead of `list` to hold possible elements within the GEMPAK file object. `set` has better performance when checking for element membership. This change also allowed for modifications to the looping done to unpack elements such that performace would be further increased. Adds climate surface files for testing. This also updates the surface file type detection that was producing incorrect results. String data was removed from unmerged sounding unpacking routine as this should not appear in those file types per GEMPAK documentation. Add additional tests to improve coverage.
dopplershift
left a comment
dopplershift
left a comment
There was a problem hiding this comment.
Thanks for the work to improve the test coverage!
dopplershift
added
Type: Enhancement
2 participants
Description Of Changes
Use
setinstead oflistto hold possible elements within the GEMPAK file object.sethas better performance when checking for element membership. This change also allowed for modifications to the looping done to unpack elements such that performance would be further increased.