Skip to content

Update cloudwatch query to include records that were accidently being… #12207

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

sb-2011
Copy link
Contributor

@sb-2011 sb-2011 commented May 23, 2025

To fix the stale data check bug that is causing mismatches between idp log group and redshift tables.

Related MRs:

🎫 Ticket

issue-1005

🛠 Summary of changes

Update the log column extractor with the following changes:

  • Remove explicit table locking and truncate query to avoid dropping missed records and deadlock exceptions
  • Update deduplication logic to address bug where uuids with duplicates are being dropped

📜 Testing Plan

  • Step 1: Deploy to personal env
  • Step 2: Run the entire stale data check workflow and confirm that the cloudwatch row count check passes

Copy link
Contributor

@colter-nattrass colter-nattrass left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Simple enough. Looks like you need a changelog message to merge it

… dropped from DW

changelog: Internal, DataWarehouse Alerts, Update the TableSummaryStatsExportJob cloudwatch query to include set of previously excluded records (Issue 1005)
@sb-2011 sb-2011 force-pushed the 1005-update-cw-query branch from 8e01f67 to d7c8a2b Compare May 23, 2025 23:45
Copy link
Contributor

@samathad2023 samathad2023 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants