DPA-1666: Support Virtual Stream IDs

[ro-tex]

DPA-1666

In order to support virtual stream IDs, we do a couple of things:

• include them in the job spec
• label the produced job with them, so we can handle the job by its stream id

Additionally, we moved all job-handling changesets under a dedicated submodule in order to reduce the mess.

[Copilot]

Pull Request Overview

This PR implements support for virtual stream IDs by refactoring identifier functions and updating job-related logic and tests.

• Refactored the DON identifier function from DonIdentifier to DonIDLabel.
• Introduced StreamIDLabel and StreamIDFromLabel to generate and parse virtual stream IDs.
• Updated tests and job proposal logic to use the new functions and adjusted package naming from changeset to jobs.

Reviewed Changes

Copilot reviewed 11 out of 13 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
deployment/data-streams/utils/identifiers.go	Renamed identifier function and added virtual stream ID utilities.
deployment/data-streams/changeset/testutil/test_helpers.go	Updated test helpers to use the new DON identifier function.
deployment/data-streams/changeset/jobs/jd_revoke_jobs_test.go	Updated test patterns and names for revoking jobs with virtual stream IDs.
deployment/data-streams/changeset/jobs/jd_revoke_jobs.go	Refactored job revocation logic with a new helper for IDs retrieval.
deployment/data-streams/changeset/jobs/jd_distribute_stream_jobs_test.go	Updated test expectations with new stream ID label syntax.
deployment/data-streams/changeset/jobs/jd_distribute_stream_jobs.go	Enhanced job spec creation with support for virtual stream ID extraction.
deployment/data-streams/changeset/jobs/jd_distribute_llo_jobs_test.go	Updated test usage of identifier function and label naming.
deployment/data-streams/changeset/jobs/jd_distribute_llo_jobs.go	Revised proposal labeling with the updated DON identifier.
deployment/data-streams/changeset/jobs/helpers_test.go	Introduced a random stream ID generator to ensure test uniqueness.
deployment/data-streams/changeset/jobs/common.go	Package naming updated from changeset to jobs.
deployment/data-streams/changeset/jd_register_nodes.go	Updated node registration to use the new DON identifier function.

Files not reviewed (2)

• deployment/data-streams/jobs/templates/osrc_mercury_v1_median.go.tmpl: Language not supported
• deployment/data-streams/jobs/templates/osrc_mercury_v1_quote.go.tmpl: Language not supported

[Copilot]

Pull Request Overview

This PR implements support for virtual stream IDs by introducing new helper functions and refactoring job-handling logic into a dedicated submodule. Key changes include:

• Refactoring identifier functions (e.g. renaming DonIdentifier to DonIDLabel, adding StreamIDLabel and StreamIDFromLabel).
• Updating job templates and tests to incorporate streamID parameters for both traditional and virtual streams.
• Adjusting job-related changesets and test utilities to support the new virtual stream ID logic.

Reviewed Changes

Copilot reviewed 13 out of 13 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
deployment/data-streams/utils/identifiers.go	Renamed DonIdentifier to DonIDLabel and added functions for managing stream IDs.
osrc_mercury_v1_quote.go.tmpl osrc_mercury_v1_median.go.tmpl	Updated templates to include an optional streamID from report fields.
changeset/testutil/test_helpers.go	Updated label usage from DonIdentifier to DonIDLabel in test helpers.
jobs/jd_revoke_jobs_test.go	Modified regex patterns to extract streamID and added tests for virtual stream IDs.
jobs/jd_revoke_jobs.go	Refactored job revocation logic, extracting job lookup into helper functions.
jobs/jd_distribute_stream_jobs_test.go	Updated job spec tests to include streamID support and adjusted labels.
jobs/jd_distribute_stream_jobs.go	Refactored job proposal creation to use updated labels and extract virtual stream IDs.
jobs/jd_distribute_llo_jobs_test.go jobs/jd_distribute_llo_jobs.go	Replaced DonIdentifier with DonIDLabel and updated node labeling logic accordingly.
jobs/helpers_test.go	Updated test helpers to dynamically generate stream IDs and use the new ID functions.
jobs/common.go	Package renaming from changeset to jobs for consistency.
jd_register_nodes.go	Updated node registration to use DonIDLabel.

[cl-sonarqube-production]

Quality Gate passed

Issues
1 New issue
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube

[Copilot]

Pull Request Overview

This PR introduces support for virtual stream IDs while consolidating job-handling changesets under a dedicated submodule. Key changes include:

• Renaming the DonIdentifier function to DonIDLabel and updating its documentation accordingly.
• Adding new utility functions (StreamIDLabel and StreamIDFromLabel) to generate and parse stream ID labels.
• Updating job spec templates and tests to accommodate virtual stream IDs.

Reviewed Changes

Copilot reviewed 13 out of 13 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
deployment/data-streams/utils/identifiers.go	Renamed DonIdentifier to DonIDLabel and added stream ID utility functions for virtual stream IDs.
deployment/data-streams/jobs/templates/osrc_mercury_v1_quote.go.tmpl	Updated job spec templates to include an optional streamID parameter for benchmark, bid, and ask prices.
deployment/data-streams/jobs/templates/osrc_mercury_v1_median.go.tmpl	Updated the median job spec template with the new streamID parameter.
deployment/data-streams/changeset/testutil/test_helpers.go	Replaced usage of DonIdentifier with DonIDLabel in helper functions.
deployment/data-streams/changeset/jobs/jd_revoke_jobs_test.go	Updated regex patterns in tests to extract stream IDs and included virtual stream ID tests.
deployment/data-streams/changeset/jobs/jd_revoke_jobs.go	Refactored job revocation logic by delegating to a helper (findJobsForIDs) for both UUIDs and stream IDs.
deployment/data-streams/changeset/jobs/jd_distribute_stream_jobs_test.go	Modified test templates to include streamID labels.
deployment/data-streams/changeset/jobs/jd_distribute_stream_jobs.go	Added support for virtual stream IDs and introduced helper methods to derive stream ID labels from report fields.
deployment/data-streams/changeset/jobs/jd_distribute_llo_jobs_test.go	Updated bootstrap spec to reflect the changes in DON labeling.
deployment/data-streams/changeset/jobs/jd_distribute_llo_jobs.go	Replaced DonIdentifier with DonIDLabel for consistency in labeling nodes and job proposals.
deployment/data-streams/changeset/jobs/helpers_test.go	Introduced a helper to generate random stream IDs for tests.
deployment/data-streams/changeset/jobs/common.go	Updated package name to reflect the new submodule structure.
deployment/data-streams/changeset/jd_register_nodes.go	Updated DON label usage from DonIdentifier to DonIDLabel.

Comments suppressed due to low confidence (2)

deployment/data-streams/utils/identifiers.go:13

• The renaming of DonIdentifier to DonIDLabel is clear; ensure that all corresponding documentation and comments across the codebase reflect this change consistently.

// DonIDLabel generates a unique identifier for a DON based on its ID and name.

deployment/data-streams/changeset/jobs/jd_revoke_jobs_test.go:83

• [nitpick] The updated regex pattern relies on precise newline characters; consider using a more flexible pattern that accounts for varying whitespace to improve robustness.

matches := regexp.MustCompile("\nstreamID\s*=\s*([0-9]+)\s*\n").FindStringSubmatch(jobSpec)