DAS-2221: Add support for n-dimensional variables #41
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
joeyschultz
commented
Sep 17, 2025
| "/support_data/gas_profile", | ||
| "/support_data/scattering_weights", | ||
| "/support_data/temperature_profile" | ||
| "/support_data/cal_adjustment" |
There was a problem hiding this comment.
"/support_data/cal_adjustment" must remain in the excluded variables list because it is non-projectable due to its dimension shape not matching that of its coordinates.
float cal_adjustment(xtrack=2048, wavelength=12);
joeyschultz
commented
Sep 17, 2025
| "/support_data/ozone_profile_temperature", | ||
| "/qa_statistics/avg_residuals", | ||
| "/qa_statistics/fit_RMS" | ||
| "/support_data/other_gas_names" |
There was a problem hiding this comment.
/support_data/nongas_param_names, /support_data/nongas_param_units, and /support_data/other_gas_names must remain in the excluded variables list because they are non-projectable due to their dimension shape not matching that of the coordinates.
char nongas_param_names(non_gas_variables=9, param_strlen=4);
char nongas_param_units(non_gas_variables=9, param_unit_strlen=20);
char other_gas_names(gases=3, param_strlen=4);
flamingbear
requested changes
Sep 18, 2025
tests/unit/test_nc_single_band.py
Outdated
| 'latitude_longitude', | ||
| non_horizontal_dim = Mock(spec=Dimension) | ||
| non_horizontal_dim.name = "third_dim" | ||
| non_horizontal_dim.size = 2 |
There was a problem hiding this comment.
This would be better as 3 or 5 so that it doesn't get confused with the horizontal dimension with the same size.
There was a problem hiding this comment.
Updated in 6ae3f23.
| self.assertEqual(len(dataset.dimensions["time"]), 5) | ||
| self.assertEqual(len(dataset.dimensions["layer"]), 10) | ||
|
|
||
| with self.subTest("duplicate non-horizontal dimensions handled correctly"): |
There was a problem hiding this comment.
What causes duplicate dimensions that need to be dropped?
There was a problem hiding this comment.
It's not that duplicate dimensions need to be dropped, but rather that attempting to create a dimension that already exists would cause an error.
In write_non_horizontal_dimensions, I am preventing that error case by checking whether it exists before creating the dimension - this covers the duplicate dimension use case.
tests/unit/test_interpolation.py
Outdated
| {'values': source_layer, 'fill_value': fill_value}, | ||
| reprojection_information, | ||
| ) | ||
| np.testing.assert_array_equal(result, mocked_resample_result) |
There was a problem hiding this comment.
I don't think this is a great test. I think it's already implicitly covered in the test_resample_*** functions. It seems like it is just calling a function and testing that it puts the information into a dictionary and then calls a different function. I don't know it's probably not worth ripping out but not sure what we're learning.
There was a problem hiding this comment.
I agree with your point, I'm ok with removing it.
There was a problem hiding this comment.
Removed the test in a3033c0.
| ("time", "layer", "mirror_step", "xtrack"), | ||
| [time_dim, layer_dim], | ||
| ) | ||
| self.assertEqual(result, expected) |
There was a problem hiding this comment.
These are great and comprehensive. Nice!
vutrannasa
reviewed
Sep 24, 2025
swath_projector/interpolation.py
Outdated
| case of a 2-D layer representing a horizontal spatial slice. This slice | ||
| is resampled with the supplied resampler. | ||
| """ | ||
| if len(s_var.shape) > 2: |
There was a problem hiding this comment.
Up to you, this will return right away if the condition is not met.
if len(s_var.shape) <= 2:
return resample_layer(s_var[:], fill_value, reprojection_information, resampler)
for layer_index in range(s_var.shape[0]):
t_var[layer_index, ...] = resample_variable_data(
s_var[layer_index, ...],
t_var[layer_index, ...],
fill_value,
reprojection_information,
resampler,
)
return t_var
There was a problem hiding this comment.
Nice idea. I find that having the base case up top is more readable. Updated in a1f12a5.
flamingbear
approved these changes
Sep 25, 2025
|
Test: PASS Test Step:
|
vutrannasa
approved these changes
Sep 25, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This PR enables resampling of n-dimensional variables (3D and above).
Jira Issue ID
DAS-2221
Local Test Steps
Note: I am still working on resolving test failures in test_interpolation.py
swath-projector./bin/bootstrap-harmony/support_data/scattering_weightsto verify the reprojection of a 3D variableNote: If tester does not have permissions to the ASDC collections in the notebook, this url can be used to test the TEMPO_O3TOT_L2_V03 collection in the EEDTEST provider. Ensure the request suceeds and download the output to inspect the 3D variables were processed correctly:
http://localhost:3000/C1270257471-EEDTEST/ogc-api-coverages/1.0.0/collections/parameter_vars/coverage/rangeset?forceAsync=true&granuleId=G1270257477-EEDTEST&variable=all&serviceId=S1237974711-EEDTEST
PR Acceptance Checklist
CHANGELOG.mdupdated to include high level summary of PR changes.docker/service_version.txtupdated if publishing a release.