[8pt] PR: Fix Alaska HydroIDs

[GregoryPetrochenkov-NOAA]

This PR addresses #1629. Upon further review, even aside from the HydroIDs being more than 32766 in number, the gage watershed algorithm output normally creates HydroIDs starting with 8 digits. However, in some HUCs in Alaska there is a mix of 8 digit and 9 digit HydroIDs even though the amount of features are less than 32766 in cardinality. this addresses both cases as well as provides error for processing a catchment if Int16 is used and there are more than 32766 features and or a mix of 8 digit and 9 digit HydroIDs.

I also removed the overlapping_inundation script in favor of rasterio merge which will address the following issues:
#644, and partially addresses #1562.

Changes

• src/delineate_hydros_and_produce_HAND.sh: Add check for Alaska HUC2 identifier before running conversion to int16 data type.
• src/process_branch.sh: Add handling for int16 conversion failure error code.
• src/src_adjust_spatial_obs.py: Add check for Alaska HUC2 identifier to see if process should run in Int16 or Int32.
• src/utils/fim_enums.py: Add error code for int16 conversion error.
• tools/catfim/generate_categorical_fim_mapping.py: Process in either in Int16 for rem and HydroIDs, or int32 and float32 respectively in the case of an Alaska HUC.
• tools/inundation.py: Process in either in Int16 for rem and HydroIDs, or int32 and float32 respectively in the case of an Alaska HUC.
• tools/mosaic_inundation.py: Add rasterio merge and remove calls to removed script overlapping_inundation.py.
• tools/convert_to_int16.py: Add raise error for scenarios where catchment raster has too many HydroIDs or HydroIDs that have more than 8 digits.

Removals

• /tools/overlapping_inundation.py

---

Testing

Created FIM outputs with all of Alpha Test and every Alaska HUC in an output folder named alaska_fix.

I ran synthesize test cases and got identical results for all evaluation types aside from two NWS sites in Alaska HUC 19020302. I am still showing the difference even though the inundation was previously incorrect. While the CSI and PND regressed the MCC and FAR all increased for these two sites.

I also ran inundate nation for the alaska_fix output folder and remedied the over inundation exhibited in the original issue as seen here:

---

Deployment Plan (For FIM developers use)

• Does the change impact inputs, docker or python packages?

  • Yes
  • No (f no.. skip the rest of the Deployment Plan section)

• If you are not a FIM dev team member: Please let us know what you need and we can help with it.

• If you are a FIM Dev team member:

  • Please work with the DevOps team and do not just go ahead and do it without some co-ordination.

  • Copy where you can, assign where you can not, and it is your responsibility to ensure it is done. Please ensure it is completed before the PR is merged.

  • Has new or updated python packages, PipFile, Pipefile.lock or Dockerfile changes? DevOps can help or take care of it if you want. Just need to know if it is required.

    • Yes
    • No

  • Require new or adjusted data inputs? Does it have a way to version (folder or file dates)?

    • No
    • Yes
      • Require new pre-clip set or any other data reloads, such as DEMS, osm, etc. ie.. pre-requisite re-data upstream of your input changes.
        • Yes
        • No
      • Has the inputs been copied/exist in all five enviros:
        • FIM EFS
        • FIM S3
        • ESIP
        • Dev1
        • UCS2

• Please use caution in removing older version unless it is at least two versions ago. Confirm with DevOps if cleanup might be involved.

• If new or updated data sets, has the FIM code, including running fim_pipeline.sh, been updated and tested with the new/adjusted data? You can dev test against subsets if you like.

  • Yes

Notes to DevOps Team or others:

Please add any notes that are helpful for us to make sure it is all done correctly. Do not put actual server names or full true paths, just shortcut paths like 'efs..../inputs/, or 'dev1....inputs', etc.

---

Issuer Checklist (For developer use)

You may update this checklist before and/or after creating the PR. If you're unsure about any of them, please ask, we're here to help! These items are what we are going to look for before merging your code.

• Informative and human-readable title, using the format: [_pt] PR: <description>
• Links are provided if this PR resolves an issue, or depends on another other PR
• If submitting a PR to the dev branch (the default branch), you have a descriptive Feature Branch name using the format: dev-<description-of-change> (e.g. dev-revise-levee-masking)
• Changes are limited to a single goal (no scope creep)
• The feature branch you're submitting as a PR is up to date (merged) with the latest dev branch
• pre-commit hooks were run locally
• Any change in functionality is tested
• New functions are documented (with a description, list of inputs, and expected output)
• Placeholder code is flagged / future todos are captured in comments
• CHANGELOG updated with template version number, e.g. 4.x.x.x
• Add yourself as an assignee in the PR as well as the FIM Technical Lead

---

Merge Checklist (For Technical Lead use only)

• Update CHANGELOG with latest version number and merge date
• Update the Citation.cff file to reflect the latest version number in the CHANGELOG
• If applicable, update README with major alterations

[EmilyDeardorff]

So I ran both stage- and flow-based CatFIM on HUCs 19020402, 19020505, and 12040104.

Flow-based CatFIM (the main one we were worried about) looks pretty good. The overflooding bug appears to be gone.

---

Stage-based CatFIM, on the other hand, looks like it might have had a bug introduced. When you compare the results from this branch to what we currently have on FIAT, it looks like stage-based CatFIM in Alaska is heavily overflooding. Uncertain if that is directly connected to this PR but it is definitely necessary to look further into that before merging this PR.

[EmilyDeardorff]

I tested this PR for flow-based CatFIM and stage-based CatFIM and the over-inundation that was previously present in AK is no longer there. The performance if CatFIM in non-Alaska areas is also unchanged, which is a good thing.

[GregoryPetrochenkov-NOAA]

I ran the following commands testing the new precalb option (-p flag) and also without:

clear ; fim_version='alaska_fix' ; prev_metrics_file='hand_4_8_10_0' ; python3 foss_fim/tools/synthesize_test_cases.py -c DEV -l -o -v $fim_version -m /outputs/${fim_version}/${fim_version}_metrics2.csv -jh 30 -jb 2 -pcsv /data/previous_fim/${prev_metrics_file}/${prev_metrics_file}_metrics.csv

clear ; fim_version='alaska_fix' ; prev_metrics_file='hand_4_8_10_0' ; python3 foss_fim/tools/synthesize_test_cases.py -c DEV -l -p -o -v $fim_version -m /outputs/${fim_version}/${fim_version}_metrics3.csv -jh 30 -jb 2 -pcsv /data/previous_fim/${prev_metrics_file}/${prev_metrics_file}_metrics.csv

The first metrics are identical to testing before merging last round of PRs.

[EmilyDeardorff]

I reran stage and flow based CatFIM with the new code changes ad it looks like it still performed as expected.