Skip to content

Commit b0263c2

Browse files
authored
chore(tests): Address flakiness on eco-related tests (#94869)
Resolves #93975, #94793 **Source Code Integrations** ``` tests/sentry/integrations/github/test_webhooks.py::PullRequestEventWebhook::test_multiple_orgs_creates_missing_repo ``` - The flake seems to have been resolved in #91347 ``` tests/sentry/integrations/github/tasks/test_open_pr_comment.py::TestOpenPRCommentWorkflow::test_comment_workflow tests/sentry/integrations/gitlab/tasks/test_pr_comment.py::TestCommentWorkflow::test_comment_workflow_updates_comment tests/sentry/integrations/github/tasks/test_pr_comment.py::TestCommentWorkflow::test_comment_workflow_updates_comment ``` - These I never couldn't repro, but I think it was inconsistent with the Group.objects.all() call for the created groups, so sometimes the wrong group was prioritized. Made these consistent to hopefully avoid that in future runs. --- **Plugins** ``` tests/sentry_plugins/twilio/test_plugin.py::test_entry_point tests/sentry_plugins/victorops/test_plugin.py::test_entry_point ``` - The failures dont seem directly related to the entry point tests, but regardless, they aren't testing anything important, removing them. ``` tests/acceptance/sentry_plugins/test_amazon_sqs.py::AmazonSQSTest::test_simple ``` - These acceptance tests are not useful, I'm deleting the entire directory --- **Slack** ``` tests/sentry/integrations/slack/webhooks/actions/test_status.py::StatusActionTest::test_permission_denied tests/sentry/integrations/slack/webhooks/actions/test_status.py::StatusActionTest::test_permission_denied_through_unfurl tests/sentry/integrations/slack/webhooks/actions/test_status.py::StatusActionTest::test_archive_issue_until_escalating ``` - The URLs built from these webhooks have signed parameters which use TimestampSigner. If the timestamp changes between between test and implementation, a flake is produced. `freeze_time` prevented this. ``` tests/acceptance/test_link_team.py::SlackLinkTeamTest::test_link_team tests/acceptance/test_link_team.py::SlackLinkTeamTest::test_link_team_as_team_admin ``` - The flakiness was a race, addressed by waiting for the next page to load --- **Charts** ``` tests/acceptance/chartcuterie/test_chart_renderer.py::TestChartRenderer::test_debug_renders tests/acceptance/chartcuterie/test_image_block_builder.py::TestSlackImageBlockBuilder::test_image_block_for_function_regression tests/acceptance/chartcuterie/test_image_block_builder.py::TestSlackImageBlockBuilder::test_image_not_generated_for_unsupported_issues ``` - Couldn't reproduce any errors here, tried slow network and looping the test. I just refactored them a bit but leaving them untouched mostly --- **Integration Utils** ``` tests/sentry/integrations/utils/test_scope.py::BindOrgContextFromIntegrationTest::test_binds_org_context_with_multiple_orgs tests/sentry/integrations/utils/test_scope.py::BindOrgContextFromIntegrationTest__InRegionMode::test_binds_org_context_with_multiple_orgs tests/sentry/integrations/utils/test_scope.py::BindOrgContextFromIntegrationTest__InControlMode::test_binds_org_context_with_multiple_orgs ``` - The list of orgs passed to the function could be random in how django retrieved them. Switched to asserting on the sorted lists for consistency. --- **Misc** ``` tests/sentry_plugins/test_anonymizeip.py::test_ipv6 ``` - I don't see how this test can flake, its simple and seems pure to me. ran it 10M times with no errors, if it is flaking there is some unique pattern to it being run in CI, not within the test itself. ``` tests/sentry/tasks/test_daily_summary.py::DailySummaryTest::test_no_performance_issues tests/sentry/integrations/opsgenie/test_integration.py::OpsgenieMigrationIntegrationTest::test_migrate_plugin tests/jobs/ ``` - List sorting flakiness I suspect, but couldn't reproduce the flakes.
1 parent 1b3f9b3 commit b0263c2

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

46 files changed

+84
-460
lines changed

src/sentry/integrations/slack/views/__init__.py

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,6 @@
1111

1212

1313
def build_linking_url(endpoint: str, **kwargs: Any) -> str:
14-
"""TODO(mgaeta): Remove cast once sentry/utils/http.py is typed."""
1514
url: str = absolute_uri(reverse(endpoint, kwargs={"signed_params": sign(salt=SALT, **kwargs)}))
1615
return url
1716

src/sentry/templates/sentry/integrations/slack/post-linked-team.html

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@
1111
{% trans body_text %}
1212
</p>
1313
<p>
14-
<a href="slack://channel?id={{ channel_id }}&team={{ team_id }}" class="btn btn-default btn-login-slack">
14+
<a href="slack://channel?id={{ channel_id }}&team={{ team_id }}" class="btn btn-default btn-login-slack" data-test-id="back-to-slack">
1515
<span class="provider-logo slack"></span> Go back to Slack
1616
</a>
1717
</p>

src/sentry/testutils/helpers/plugins.py

Lines changed: 0 additions & 20 deletions
This file was deleted.

tests/acceptance/chartcuterie/test_chart_renderer.py

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,8 @@ def test_debug_renders(self):
1212
with self.options(options):
1313
self.browser.get("debug/chart-renderer/")
1414

15+
self.wait_for_loading()
16+
1517
images = self.browser.elements(selector="img")
1618
assert len(images) > 0
1719

tests/acceptance/chartcuterie/test_image_block_builder.py

Lines changed: 1 addition & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -12,11 +12,7 @@
1212
ProfileFunctionRegressionType,
1313
)
1414
from sentry.models.group import Group
15-
from sentry.testutils.cases import (
16-
AcceptanceTestCase,
17-
MetricsEnhancedPerformanceTestCase,
18-
ProfilesSnubaTestCase,
19-
)
15+
from sentry.testutils.cases import MetricsEnhancedPerformanceTestCase, ProfilesSnubaTestCase
2016
from sentry.testutils.helpers.datetime import before_now
2117
from sentry.testutils.helpers.features import with_feature
2218
from tests.sentry.issues.test_utils import OccurrenceTestMixin
@@ -25,7 +21,6 @@
2521

2622

2723
class TestSlackImageBlockBuilder(
28-
AcceptanceTestCase,
2924
MetricsEnhancedPerformanceTestCase,
3025
ProfilesSnubaTestCase,
3126
OccurrenceTestMixin,

tests/acceptance/sentry_plugins/test_amazon_sqs.py

Lines changed: 0 additions & 20 deletions
This file was deleted.

tests/acceptance/sentry_plugins/test_asana.py

Lines changed: 0 additions & 20 deletions
This file was deleted.

tests/acceptance/sentry_plugins/test_bitbucket.py

Lines changed: 0 additions & 20 deletions
This file was deleted.

tests/acceptance/sentry_plugins/test_github.py

Lines changed: 0 additions & 20 deletions
This file was deleted.

0 commit comments

Comments
 (0)