SCA Testing and Validation - Phase 2

[TomasTurina]

Description

This issue tracks the testing and validation of the Security Compliance Assessment (SCA) module developed in Phase 1. The objective of this phase is to ensure correctness, performance, and integration of the module within the new agent architecture. This includes unit testing, integration testing, and end-to-end validation across supported operating systems.

Functional Requirements

• The module must be tested for correctness across all Tier 1 supported operating systems.
• Integration tests must validate the module’s behavior within the agent framework.
• Performance tests must be conducted to assess execution efficiency.
• The module must correctly report compliance results to the Indexer.
• The module must support graceful shutdown during an ongoing scan.
• End-to-end (E2E) tests must validate:
  • Initial scan execution and correct inventory visualization.
  • Subsequent scan execution and correct change alerts visualization.

Non-Functional Requirements

• The testing process must ensure that policy execution behavior remains consistent with 4.x.
• Performance tests should evaluate the impact of scan scheduling and execution time.
• Test results must be documented for future reference and improvements.

Implementation Restrictions

• Testing must be conducted using real-case scenario policy files.
• The validation process must ensure compliance with ECS schema mappings.
• The testing environment must replicate real deployment conditions where feasible.

Plan

Implement Integration Tests

• Perform manual integration tests to compare policy execution against 4.x.
• Validate stateful and stateless message transmission correctness.
• Test DBsync integration for state persistence and synchronization.

Conduct Performance Testing

• Evaluate the execution time of policy scans under different configurations.
• Test resource usage (CPU, memory) under various workloads.
• Identify and address any performance bottlenecks.

Validate Graceful Shutdown Behavior

• Execute scans and verify that shutdown is handled correctly.
• Ensure no data corruption or inconsistency occurs during shutdown.

Conduct End-to-End (E2E) Testing

• Develop an E2E test case to execute an initial scan and visualize the inventory in the dashboard.
• Execute a subsequent scan and verify that change alerts are correctly displayed.
• Document test results and identified issues.

Document Module Behavior

• Record detailed documentation of all testing scenarios and results.
• Provide clear guidelines for debugging and troubleshooting.
• Outline potential areas for further improvement based on test outcomes.