skip-path provided multiple times has unexpected effects

[ohookins]

We have a repository containing all of our Infrastructure code (Terraform mainly) which consists of several directories for different kinds of infrastructure (out of scope of this issue, but they are logical groupings for organisational purposes) as well as a directory for shared modules.

At least one of these directories isn't checked with Checkov for various reasons. We also don't want to check the modules directory as it is checked implicitly by checking the other directories containing our deployments/workspaces.

As per issue #7327 we cannot yet specify multiple directories to Checkov, so a workaround of specifying the root directory and skipping paths was suggested. This has odd effects that are not easy to understand. For example, consider the layout as follows:

$ tree -L 1
.
├── bin
├── deploys1
├── deploys2
├── deploys3
├── docs
├── images
├── lib
├── modules
├── policy
└── template

We want to check deploys2 and deploys3 but not deploys1 or modules (or any of the other directories, but they don't contain Terraform so they are less of a concern.

Running Checkov by skipping the undesirable directories, we get the following:

$ checkov -d . --skip-path deploys1 --skip-path modules --quiet
terraform scan results:

Passed checks: 2128, Failed checks: 0, Skipped checks: 150

However, this is not the same check or skipped check count as the deploys2 or deploys3 directories:

$ checkov -d deploys2 --quiet
terraform scan results:

Passed checks: 158, Failed checks: 0, Skipped checks: 9

$ checkov -d deploys3 --quiet
terraform scan results:

Passed checks: 3343, Failed checks: 0, Skipped checks: 431

Generally deploys1 and deploys2 are approximately the same structure. deploys3 has a few subdirectories with an additional level of directory structure, but within that all of the module references are correct. And it can be checked independently without any errors. So it's a bit confusing.

Running checkov 3.2.477 on MacOS.