Fixed operator eventTrigger not capturing all ERC20 transfer problem

[chrisli30]

This pull request introduces several enhancements and fixes to the task engine, focusing on improved execution state handling, deduplication of events, and more robust event processing. The key changes include adding execution state tracking, ensuring event deduplication, and refining event-trigger processing logic.

Execution State Enhancements:

• Introduced AggregateChecksResultWithState to return detailed execution state, including task activity status, remaining executions, and error messages. Updated NotifyTriggers to leverage this new method. (aggregator/rpc_server.go, core/taskengine/engine.go) [1] [2] [3] [4]

Event Deduplication:

• Added a deduplication mechanism to track processed events using a unique key (blockNumber-txHash-logIndex) and avoid duplicate processing across multiple subscriptions. (core/taskengine/trigger/event.go) [1] [2]

Event Processing Improvements:

• Enhanced buildExecutionStepOutputData to handle nil trigger outputs gracefully by creating empty structures based on trigger type. (core/taskengine/engine.go) [1] [2]
• Updated buildTriggerDataMapFromProtobuf to align field names with the runTrigger format for consistency. (core/taskengine/engine.go)

Event Trigger Logic Refinements:

• Modified event trigger subscriptions to initialize even when no queries are available initially, allowing dynamic task additions. Introduced periodic cleanup of old processed events and event counts to prevent memory leaks. (core/taskengine/trigger/event.go) [1] [2]

Event Search Optimization:

• Adjusted event search logic to comprehensively scan recent blocks instead of stopping early, ensuring the most recent events are captured. (core/taskengine/run_node_immediately.go) [1] [2] [3]

[Copilot]

Pull Request Overview

This PR enhances event processing and execution state management for the task engine, addressing duplicate event handling, improved logging, and more robust trigger output processing. Key changes include:

• Updates to protobuf messages and generated code to include execution state fields.
• Modifications in operator and trigger processing logic to leverage new execution state and enhanced deduplication.
• Improvements in event query optimization and cleanup routines.

Reviewed Changes

Copilot reviewed 13 out of 13 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
protobuf/node.proto & protobuf/node.pb.go	Added execution state fields in NotifyTriggersResp to support detailed task status.
operator/worker_loop.go	Updated logging and task removal logic based on new execution state fields.
core/taskengine/vm*.go	Injected enhanced trigger configuration data to support downstream processing.
core/taskengine/trigger/event.go	Improved event deduplication, optimized filter query grouping, and added cleanup for processed events.
core/taskengine/run_node_immediately.go	Adjusted event search logic to perform comprehensive query searches.
core/taskengine/engine.go & aggregator/rpc_server.go	Modified aggregator logic to leverage AggregateChecksResultWithState for robust trigger handling.
core/taskengine/vm_trigger_config_test.go	Expanded tests to verify proper exposure of trigger configuration data.

[Copilot]

[nitpick] The duplicate deduplication checks in the main event loop and in processLog create redundancy. Consider centralizing the deduplication logic to a single function to reduce code duplication and potential inconsistency.

[Copilot]

[nitpick] The current cleanup method removes arbitrary entries since the keys lack timestamps. To improve accuracy and avoid processing duplicates that may reappear soon, consider augmenting the deduplication keys with timestamps or another expiry mechanism.