Qualys: do not use access_path as endpoints

[nv-pipo]

Description

Couple of bugfixes:

• fix(qualys_was_parser): do not use access_path as endpoints, because they aren't
• fix(qualys_was_parser): update import paths for settings: to reflect current module structure

Test results

Passed all unittests for qualys_webapp_parser

[dryrunsecurity]

No security concerns detected in this pull request.

---

All finding details can be found in the DryRun Security Dashboard.

[Maffooch]

I can understand why putting the access paths in the endpoints can be a bit controversial, but take this example:

<URL>https://store.example.com/js/xdr/core/lang/xdr_en.json.gz</URL>
<ACCESS_PATH>
    <URL>https://store.example.com/</URL>
    <URL>https://store.example.com/index.html</URL>
</ACCESS_PATH>

The URL shows the exact file that is vulnerable, but this could be a file from a CDN or something that we cannot change directly. The ACCESS_PATH list shows where this vulnerability was found to exist by Qualys. I think there is a lot of value in knowing the surface area of the vulnerability. If the root (and likely every other path) of the site impacted, I would liekly prioritize that higher than other things. Without the ACCESS_PATH list, we will not have that info in DefectDojo to make that decision

[nv-pipo]

@Maffooch ,
I agree that having the information in the ACCESS_PATH is of great value, but that doesn't make it an endpoint.
Maybe a more permanent solution would be for DD to have a field for ACCESS_PATH?

[Maffooch]

I agree that having the information in the ACCESS_PATH is of great value, but that doesn't make it an endpoint.

I'm not sure I follow this one. What disqualifies a host+path where a vulnerability is found from being an endpoint?

[nv-pipo]

I agree that having the information in the ACCESS_PATH is of great value, but that doesn't make it an endpoint.

I'm not sure I follow this one. What disqualifies a host+path where a vulnerability is found from being an endpoint?

Maybe there is a misunderstanding with the definitions we are using.

For me:
ACCESS_PATH: is a page that contains a html anchor that points to a vulnerable Endpoint
Endpoint: is a web page where the vulnerability can be exploited

Qualys also believes that endpoints and access_path are 2 different things: they have a field for each one.

For example a Qualys WAS XML report I'm trying to import has:
ACCESS_PATH: http://example.com/index.html (no vulnerability, it's just an index)
Endpoint: http://example.com/vulnerable_page.html (page with vulnerability)

Currently the DD Qualys_webapp parser is putting both URLs in the Endpoint field, I think this is a mistake/bug. The URL http://example.com/index.html is not exploitable so it shouldn't be an endpoint. Again, I'm not saying that we should ignore the ACCESS_PATH, there is value in this field, but maybe it needs it's own field in DD

[nv-pipo]

To prevent this issue from becoming stale, I will rephrase my previous message with a question:

Will the current version of this PR be merged into defect dojo?

If not, for future references to other contributors:
@Maffooch, could you provide comments on why DD chooses not to have a dedicated field for access_path (a not exploitable URL) but instead expects parsers to store these access_paths in the endpoints field?

[Maffooch]

@Maffooch, could you provide comments on why DD chooses not to have a dedicated field for access_path (a not exploitable URL) but instead expects parsers to store these access_paths in the endpoints field?

I am unsure of the number of tools that could leverage a new field for access paths. Qulays WAS is the only tool I have seen do this tbh. Having a new field for a very small percentage of parsers (possibly only Qualys WAS) does not seem worth taking the ownership and maintenance of the feature.

Will the current version of this PR be merged into defect dojo?

Since the Qualys WAS parser uses the default dedupe algorithm, endpoints will be part of the equation in determining the hash code of a given finding. If we remove endpoints from the finding by removing the access paths, deduplication/reimport will not able to identify existing findings, close them, and create new findings with fresh SLAs. This is not ideal. For the sake of avoiding churn there, I don't think we should accept this change.

We have a mantra here that goes something like: "A no today is just for today, but a yes today is yes forever"

[nv-pipo]

@Maffooch

Thank you for responding my previous message.

To clarify, the current version of this PR does not remove any vulnerable endpoints.

I am unclear why removing ACCESS_PATH from the endpoints would result in issues with the HASH computation/deduplication process. Vulnerable endpoints will still be present. From the tests we have run exporting Qualys WAS reports, 'vulnerabilities' always have a VULNERABLE_ENDPOINT (URL) and sometimes have 1 or more additional ACCESS_PATH.

The primary issue that motivated the creation of this PR is that we have reports and KPIs that track the count of vulnerable endpoints. However, the current parser in DD is also adding non-vulnerable endpoints (noise). Currently, after an endpoint is created in DD, the information of whether it was from a VULNERABLE_ENDPOINT or an ACCESS_PATH is lost.

I hope you reconsider accepting the current version of this PR.

[valentijnscholten]

Sounds like the ACCESS_PATH is more something to include as information in description or impact field as it is useful to know in some (rare) cases.