Skip to content

Conversation

joeyschultz
Copy link
Contributor

Description

This PR adds a metadata-override to properly annotate the units attribute on SMAP L3 timestamp variables.

Jira Issue ID

DAS-2415

Local Test Steps

Pull this branch, build the service and test images, and ensure the unit tests pass:
./bin/build-image && ./bin/build-test && ./bin/run-test

Ensure your HIAB configuration has:
LOCALLY_DEPLOYED_SERVICES=harmony-metadata-annotator,hoss,sds-maskfill

Restart harmony to pick up the new metadata-annotator image:
./bin/bootstrap-harmony

Run the DAS-2415.ipynb notebook attached to this ticket. These requests include a variable subset for the timestamp variables in each SMAP L3 collection that should get the units attribute override. Open the downloaded outputs and check that the units attribute was updated.

Run the applicable das-harmony-regression-tests notebooks to ensure no regressions. Make sure to set harmony_host_url = 'http://localhost:3000/'.

  • Metadata_Annotator.ipynb
  • HOSS_SPL2SMAP_S_tests.ipynb
  • HOSS_SPL3FTA_tests.ipynb
  • HOSS_SPL3FTP_E_tests.ipynb
  • HOSS_SPL3FTP_tests.ipynb
  • HOSS_SPL3SMA_tests.ipynb
  • HOSS_SPL3SMAP_tests.ipynb
  • HOSS_SPL3SMP_E_tests.ipynb
  • HOSS_SPL3SMP_tests.ipynb

PR Acceptance Checklist

  • Jira ticket acceptance criteria met.
  • CHANGELOG.md updated to include high level summary of PR changes.
  • docker/service_version.txt updated if publishing a release.
  • [N/A] Tests added/updated and passing. (N/A - simple configuration change)
  • Documentation updated (if needed).

"Attributes": [
{
"Name": "units",
"Value": "seconds since 2000-01-01 11:58:55.816Z"

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a question and I'm curious. How did you get 2000-01-01 11:58:55.816Z vs

time:units = "seconds since 1970-01-01 00:00:00"
time:units = "seconds since 2000-01-01 00:00:00"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From the granule metadata (this was taken from SPL3SMP), we can see that the epoch is J2000 and the UTCepoch matches what we are using. seconds since 2000-01-01 00:00:00 would be the J2000 epoch in TT. :

string :epochUTCDateTime = "2000-01-01T11:58:55.816Z" ;
:epochJulianDate = 2451545.f ;
string :timeVariableEpoch = "J2000" ;

Also, the value we chose matches what the SMAP L4 collections use for their timestamp variables.

Matt did a deeper dive exploring this in the WhatsUpWithTime.ipynb notebook attached to the ticket and confirmed this is the correct value for us to use.

@vutrannasa
Copy link

Test: PASS
ENV: http://localhost:3000

Test Step:

  • Follow instructions above
  • Ran blow test without issues
Metadata_Annotator.ipynb
HOSS_SPL2SMAP_S_tests.ipynb
HOSS_SPL3FTA_tests.ipynb
HOSS_SPL3FTP_E_tests.ipynb
HOSS_SPL3FTP_tests.ipynb
HOSS_SPL3SMA_tests.ipynb
HOSS_SPL3SMAP_tests.ipynb
HOSS_SPL3SMP_E_tests.ipynb
HOSS_SPL3SMP_tests.ipynb

1. variable_subset_request_status :  Success
2. bbox_subset_request_status :  Success
3. shape_file_subset_request_status :  Success
  • Ran DAS-2415.ipynb and verify variables below all show units "seconds since 2000-01-01 11:58:55.816Z"
            "Soil_Moisture_Retrieval_Data_1km/spacecraft_overpass_time_seconds_1km",
            "Soil_Moisture_Retrieval_Data_1km/spacecraft_overpass_time_seconds_apm_1km",
            "Soil_Moisture_Retrieval_Data_3km/spacecraft_overpass_time_seconds_3km",
            "Soil_Moisture_Retrieval_Data_3km/spacecraft_overpass_time_seconds_apm_3km",
            "Soil_Moisture_Retrieval_Data_AM/tb_time_seconds",
            "Soil_Moisture_Retrieval_Data_AM/tb_time_utc",
            "Soil_Moisture_Retrieval_Data_PM/tb_time_seconds_pm",
            "Soil_Moisture_Retrieval_Data_PM/tb_time_utc_pm"
            "Soil_Moisture_Retrieval_Data/spacecraft_overpass_time_seconds",
            "Soil_Moisture_Retrieval_Data/spacecraft_overpass_time_utc",
            "Freeze_Thaw_Retrieval_Data_Global/freeze_thaw_time_seconds",
            "Freeze_Thaw_Retrieval_Data_Global/freeze_thaw_time_utc",
            "Freeze_Thaw_Retrieval_Data_Polar/freeze_thaw_time_seconds",
            "Freeze_Thaw_Retrieval_Data_Polar/freeze_thaw_time_utc"
            "Freeze_Thaw_Retrieval_Data_Global/freeze_thaw_time_seconds",
            "Freeze_Thaw_Retrieval_Data_Global/freeze_thaw_time_utc",
            "Freeze_Thaw_Retrieval_Data_Polar/freeze_thaw_time_seconds",
            "Freeze_Thaw_Retrieval_Data_Polar/freeze_thaw_time_utc"
            "Soil_Moisture_Retrieval_Data_AM/tb_time_seconds",
            "Soil_Moisture_Retrieval_Data_AM/tb_time_utc",
            "Soil_Moisture_Retrieval_Data_PM/tb_time_seconds_pm",
            "Soil_Moisture_Retrieval_Data_PM/tb_time_utc_pm",
            "Soil_Moisture_Retrieval_Data_Polar_AM/tb_time_seconds",
            "Soil_Moisture_Retrieval_Data_Polar_AM/tb_time_utc",
            "Soil_Moisture_Retrieval_Data_Polar_PM/tb_time_seconds_pm",
            "Soil_Moisture_Retrieval_Data_Polar_PM/tb_time_utc_pm",
            "Freeze_Thaw_Retrieval_Data/freeze_thaw_time_seconds",
            "Soil_Moisture_Retrieval_Data/spacecraft_overpass_time_seconds",
            "Soil_Moisture_Retrieval_Data/spacecraft_overpass_time_utc",

Copy link

@vutrannasa vutrannasa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Thanks for answering my questions.

@joeyschultz joeyschultz merged commit f78c981 into main Oct 2, 2025
4 checks passed
@joeyschultz joeyschultz deleted the DAS-2415 branch October 2, 2025 12:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants