You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
PR #60 introduced a new Performance report to measure sniff performance during a scan.
This new report will never give reliable/accurate results if the scan is run with caching turned on as the caching feature will lead to unchanged files being skipped in the scan in favour of re-using the cached results.
Describe the solution you'd like
The cache feature should automatically be turned off when the Performance report is requested (or is one of the requested reports).
Additional context (optional)
I've had a quick look at turning caching off automatically when the Performance report is requested, but this is not as straight-forward as it sounds as it is not clear-cut where the change should be made/how it should be implemented.
There are a number of things to take into account:
Config::$overriddenDefaults should still be accurate.
CLI arguments can be passed in any order from the command-line, which means that a check would be needed both when setting the cache option, as well as when setting the reports option in Config::processLongArgument(). Or a separate check would need to be done at the end of the method Config::setCommandLineValues() to rectify.
CLI arguments can be passed from a custom ruleset, but in that case, the Config::setConfigData() method is used.
The Config class uses magic methods which include some logic, which could possible be used for this change, however, those methods can be bypassed via the Config::setSettings() method, which could lead to the Config::$overriddenDefaults no longer being accurate (potential issue anyway, independently of the change proposed here)
Outside of the Config class, it could be an option to make the change in the Runner class instead. This would likely make it more reliable, but would be a hard-overrule.
The change would likely be untestable as the cache feature is prevented from being turn on when the PHP_CODESNIFFER_IN_TESTS constant is defined.
I intend to create a pull request to implement this feature.
The text was updated successfully, but these errors were encountered:
jrfnl
changed the title
Config: automatically turn result caching off when Performance report is used
Automatically turn result caching off when Performance report is used
Jan 27, 2024
Is your feature request related to a problem?
PR #60 introduced a new
Performance
report to measure sniff performance during a scan.This new report will never give reliable/accurate results if the scan is run with caching turned on as the caching feature will lead to unchanged files being skipped in the scan in favour of re-using the cached results.
Describe the solution you'd like
The cache feature should automatically be turned off when the Performance report is requested (or is one of the requested reports).
Additional context (optional)
I've had a quick look at turning caching off automatically when the
Performance
report is requested, but this is not as straight-forward as it sounds as it is not clear-cut where the change should be made/how it should be implemented.There are a number of things to take into account:
Config::$overriddenDefaults
should still be accurate.cache
option, as well as when setting thereports
option inConfig::processLongArgument()
. Or a separate check would need to be done at the end of the methodConfig::setCommandLineValues()
to rectify.Config::setConfigData()
method is used.Config
class uses magic methods which include some logic, which could possible be used for this change, however, those methods can be bypassed via theConfig::setSettings()
method, which could lead to theConfig::$overriddenDefaults
no longer being accurate (potential issue anyway, independently of the change proposed here)Config
class, it could be an option to make the change in theRunner
class instead. This would likely make it more reliable, but would be a hard-overrule.cache
feature is prevented from being turn on when thePHP_CODESNIFFER_IN_TESTS
constant is defined.The text was updated successfully, but these errors were encountered: