Skip to content

[BUG] Content Manager flaky integration tests #408

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
AlexRuiz7 opened this issue May 8, 2025 · 3 comments · Fixed by #413
Closed

[BUG] Content Manager flaky integration tests #408

AlexRuiz7 opened this issue May 8, 2025 · 3 comments · Fixed by #413
Assignees
Labels
level/task Task issue type/bug Bug issue

Comments

@AlexRuiz7
Copy link
Member

Describe the bug
The integration tests from the Content Manager often fail on GitHub Runners caused by Java Heap out of memory errors.

These errors are caused by the content manager downloading and indexing the snapshot on each test run.

We need to improve these tests to do not initialize from a snapshot.

Expected behavior
The integration tests set up and mock the environment correctly, skipping or mocking the initialization from snapshots on each test.

Plugins
Content Manager.

Additional context
Example in https://github.com/wazuh/wazuh-indexer-plugins/actions/runs/14858193917/job/41716397598

@AlexRuiz7
Copy link
Member Author

Needs the changes from #401

Initial approach in the attached diff.

patch.txt

@abbonno abbonno self-assigned this May 12, 2025
@wazuhci wazuhci moved this from Backlog to In progress in XDR+SIEM/Release 6.0.0 May 12, 2025
@AlexRuiz7 AlexRuiz7 marked this as a duplicate of wazuh/wazuh-indexer#829 May 12, 2025
@Jorgesnchz Jorgesnchz assigned Jorgesnchz and unassigned Jorgesnchz May 12, 2025
@Jorgesnchz
Copy link
Member

I've been investigating the issue, to fix it I attempted to mock the initialize method of the SnapshotManager class. However, after reviewing information from OpenSearch, it appears that mocking this method is impossible because it executes within the onNodeStarted method, which runs prior to the integration tests. Therefore, I believe the most viable solution is the implementation of the JobScheduler (#366) which should fix the bug.

@Jorgesnchz
Copy link
Member

Testing the bug In a development environment, I was able to reproduce it by following the steps from this comment:

[2025-05-13T14:37:54,232][INFO ][c.w.c.c.CTIClient        ] [integTest-0] Starting snapshot download from [https://cti.wazuh.com/store/contexts/vd_1.0.0/consumers/vd_4.8.0/1931858_1747057658.zip]
»  java.lang.OutOfMemoryError: Java heap space
»  Dumping heap to logs/java_pid146593.hprof ...
»  Heap dump file created [370766792 bytes in 0.719 secs]
» [2025-05-13T12:37:58.243189593Z] [BUILD] Stopping node

I think that we thought It was not happening in a development environment because we forgot to do the steps from the comment previously mentioned.

@wazuhci wazuhci moved this from In progress to In review in XDR+SIEM/Release 6.0.0 May 13, 2025
@wazuhci wazuhci moved this from In review to Pending final review in XDR+SIEM/Release 6.0.0 May 13, 2025
@wazuhci wazuhci moved this from Pending final review to In final review in XDR+SIEM/Release 6.0.0 May 14, 2025
@wazuhci wazuhci moved this from In final review to Done in XDR+SIEM/Release 6.0.0 May 14, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
level/task Task issue type/bug Bug issue
Projects
Status: Done
Development

Successfully merging a pull request may close this issue.

3 participants