feat(deletions): Task for deleting nodestore events #96795
Conversation
github-actions
bot
added
the
Scope: Backend
❌ 1 Tests Failed:
View the top 1 failed test(s) by shortest run time
To view more test analytics, go to the Test Analytics Dashboard |
|
bugbot run |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
| @@ -114,6 +114,24 @@ def test_simple_multiple_groups(self) -> None: | |||
| assert Group.objects.filter(id=self.keep_event.group_id).exists() | |||
| assert nodestore.backend.get(self.keep_node_id) | |||
|
|
|||
| def test_simple_multiple_groups_parallel(self) -> None: | |||
| with self.options({"deletions.nodestore.parallelization-task-enabled": True}): | |||
| group_ids = [self.event.group.id, self.event_id2.group.id] | |||
There was a problem hiding this comment.
Bug: Event ID String Access Fails
Accessing self.event_id2.group.id causes an AttributeError because self.event_id2 is an event ID string, not an Event object. This is due to self.event_id2 being assigned self.store_event(...).event_id in the setUp method, which extracts only the string ID.
Locations (1)
| ) | ||
|
|
||
| assert not Group.objects.filter(id__in=group_ids).exists() | ||
| assert not nodestore.backend.get_multi([self.event_node_id, self.node_id2]) |
There was a problem hiding this comment.
Bug: Undefined Node ID Causes Test Failure
The newly added test test_simple_multiple_groups_parallel references self.event_node_id, which is not defined in the test setup. This causes a NameError at runtime. It should likely be self.node_id, which is the correct node ID for self.event.
Locations (1)
|
|
||
| def _delete_events_from_nodestore(events: Sequence[Event], extra: dict[str, Any]) -> None: | ||
| node_ids = [Event.generate_node_id(event.project_id, event.event_id) for event in events] | ||
| nodestore.backend.delete_multi(node_ids) |
There was a problem hiding this comment.
Bug: Issue Platform Events Deletion Fails
The new parallel nodestore deletion task, specifically the _delete_events_from_nodestore function, incorrectly generates node IDs for Issue Platform events. It consistently uses event.event_id, whereas for Dataset.IssuePlatform events, the node ID should be derived from event._snuba_data["occurrence_id"]. This discrepancy prevents Issue Platform events from being properly deleted from nodestore.
Locations (1)
Groups with millions of events fail to be deleted because the nodestore deletions fail.
This creates a new task to handle the deletions in parallel.
An option is to be added to control switching to this new task.
Fixes ID-821