HWM is not working correctly on multisource nodes.

[jabibo]

The HWM that is implemented is not working the way it is intended on multisourced nodes like Hubs and Links.
The SQL determines the last HWM on the highest LDTS in the node no matter where it comes from.
I think it should at least honor the record-source while determine the highest LDTS.
Then again ... I think it would be worth noting in the WIKI that if two business-keys are being read from the same source (e.g. parent-child) they should be read (and written) with different record-sources (for the same reason as stated above)

[tkirschke]

Hi @jabibo and thanks for reaching out!

For this reason, the HWM should be disabled right now for multi-source entities like Hubs and Links.

The config for a Hub looks like this in the data.yml:

` - groupName: Data Vault
items:

      - displayName: Hub Hash Key column
        type: columnSelector
        attributeName: is_Hub_hk
        isRequired: true
        enableIf: 'true'
        
      - displayName: Hub LoadDateTimestamp Column
        enableIf: 'false'
        type: columnSelector
        attributeName: is_Hub_ldts
        isRequired: false

      - displayName: Disable High-Water-Mark?
        type: toggleButton
        isRequired: false
        enableIf: 'false'
        attributeName: disable_hwm   
        default: true`

Since the HWM is set to "enableIf: false" it is not shown in the GUI, and the default of this config item is set to "true". Therefore the code in the RUN template that generates the HWM is not activated.

Can you confirm this on your side?

Best regards,
Tim

[jabibo]

..actually I liked the implementation of the HWM..one has to be careful in the case of the case stated above and even more if used to source one hub twice from the same source (parent-child or source splitted on row-type) .. in that case I use different RSCR tweaked with an identifier...but Yes I can confirm that the HWM is disabled by default on Links and Hubs