The Severity table in the Vulnerability UI would benefit from clarification and normalization

[DennisClark]

Please see https://nvd.nist.gov/vuln-metrics/cvss where NIST explains (at great length) how a Severity (CVSS) is calculated.

While looking at https://public.vulnerablecode.io/vulnerabilities/8683
I could not help but wonder about the bewildering range of Severity scores, including totally different terminology.
This is not the fault of VulnerableCode, which is simply returning what it found in the original sources, but the results are still rather difficult for the user to interpret.

Perhaps a link to the NIST page, right after the "Severity" label, would be appropriate here.

[DennisClark]

or maybe the link should be part of floatover help for the Score column in the table. See attached image.

[DennisClark]

I think we have two closely-related issues then:

• what does Severity mean and how should we explain/present it, and
• a bug, I think, that we do not return consistently formatted values in Score

In the example above for VULCOID-6P7 the Score values include (1) labels, (2) calculated CVSS numbers, and (3) some string that is probably used to support the scoring, but I don't quite understand it. I think we should always avoid returning the third thing; note that in the case of this row in the sample Severity table (which impacts Celery, btw) VulnerableCode could have actually returned a Score, becuase there is one on the page of the provided nvd.nist link, and that score is Base Score: [7.5 HIGH] but instead we return this string of stuff as the Score:

cvssv3_vector | CVSS:3.1/AV:N/AC:H/PR:L/UI:N/S:U/C:H/I:H/A:H | https://nvd.nist.gov/vuln/detail/CVE-2021-23727

So I think we have both a backend server problem and a presentation problem. Sounds like a job for @johnmhoran perhaps in collaboration with @TG1999

An interesting and important issue to resolve I think.

[DennisClark]

Discussion item: it is probably time to focus on CVSS v3 scoring and to drop (completely) CVSS v2 scoring.

[DennisClark]

More comments. Following the various links in the VULCOID-6P7 example, it seems to me that we should always try to return the calculated CVSS3 number, followed by its Label, and we should NOT return those "vector" strings.

Note that if you follow the links, such as for the last row GHSA-q4xr-rc97-m4xx you will always (at least in my experience) find the "Score" number (0.0 - 10.0), and the Severity "Label" (None, Low, Medium, High, Critical) can be derived from that number. See attached official NIST image.

[DennisClark]

Here is some pertinent input from @mjherzog

CVSS is owned and managed by FIRST.Org, Inc. (FIRST), a US-based non-profit organization, whose mission is to help computer security incident response teams across the world. The official CVSS documentation can be found at https://www.first.org/cvss/.

and more important - https://nvd.nist.gov/general/news/retire-cvss-v2 so we can ignore cvss v2 imho

[ziadhany]

CVSS calculation is quite complicated ,so
we use a package called cvss. it supports cvss-v2 , cvss-v3, and they made a release for us. RedHatProductSecurity/cvss#33
Have a look at #747

[pombredanne]

Let me reformulate where we are and where we want to go:

1. Remove duplicate rows for CVSS - We have a PR to rationalize the way we store severities add support for calculating CVSS score from the CVSS vector #747 and this will help avoid having duplicate rows for a CVSS vector and its score

2. Design a universal way to compare scores with mappings - We have multiple severity sources and not all are CVSS-based. We should have a mapping table for all scoring systems to a scale such as this The Severity table in the Vulnerability UI would benefit from clarification and normalization #889 (comment) and that would be "universal". Once we have this we can display primarily this VulnerableCode severity value (which can then be sorted) for each Vulnerability (see proposed change below)

3. Streamline schema to relate Severity directly to Vulnerability - Beyond this we need to rethink the schema: I suggest

• Today we have this relationship: Vulnerability -> VulnerabilityReference -> VulnerabilitySeverity
• Tomorrow:
  • either Vulnerability -> VulnerabilitySeverity and it would have a URL, no need for extra relationship for reference
  • or Vulnerability -> VulnerabilitySeverity -> VulnerabilityReference and the reference would show also as a plain reference
• The benefits are simpler model traversal, query and sorting and a more natural relationship

[mjherzog]

Two questions:

1. Are we talking about some composite Severity score?
2. What will we display in the Package-Vulnerability view? Severity would be very relevant, especially when there are multiple vulnerabilities for a Package.

[DennisClark]

An emerging issue among consumers of VulnerabilityDB data is how best to respond to incoming vulnerability reports (level of urgency, which ones need to be addressed immediately, etc.). The current severity scoring available is not all that consistent. For example, the vulnerabilities for urllib3 (at https://public.vulnerablecode.io/vulnerabilities/VCID-6rq6-hxp1-aaae ) show a bunch of different severity evaluations, including:

7.5
Medium
medium
5.0
HIGH
Moderate

So I think that VulnerableCode needs a "severity normalizer" feature or, as @mjherzog suggested, some kind of compositie severity score.

[DennisClark]

@TG1999 this issue might make a nice discussion topic at our next VC weekly meeting.

[DennisClark]

per @mjherzog "I suspect the best we can do in the short term is to normalize the available security scores by source - in the example above there are 3 sources: NIST/CVE, Red Hat and GitHub where we have one CVE reference from Red Hat repeating the CVE score and others with severity=medium. Since the real world impact of a vulnerability is likely to vary by software context we are best off to narrow the number of scores to one per source. I think that we can also safely exclude csvss2 because it has been deprecated."

I think we should construct some kind of mapping to convert the descriptive severity scores to numeric values, something like:
low ==>> 3
medium, moderate ==>> 5
high ==>> 9
(I am not sure what all possible descriptive scores might be, but our mapping should follow this general pattern)
and then...
as Michael suggested, we "narrow the number of scores to one per source, and we exclude csvss2", and then
compute an average score from the mapped numeric values and call it something like
computed_severity or normalized_severity or vcio_severity_score or similar.

[mjherzog]

I am a bit concerned that numbers will convey more precision than there is in the underlying data. So at a minimum we need to avoid fractions.

[DennisClark]

alternate approach would be to compute a Range rather than an average.

[ziadhany]

I am a bit concerned that numbers will convey more precision than there is in the underlying data. So at a minimum we need to avoid fractions.

I think we shouldn't avoid fractions because if we avoid fractions the severity range will be useless,
and should we exclude cvssv2?

for example :

[DennisClark]

please keep the fractions and display them in the severity range

[mjherzog]

We should exclude cvssv2 and document why - because these are the deprecated side of redundant data.

[TG1999]

We should exclude cvssv2 and document why - because these are the deprecated side of redundant data.

Added issue for same here #1187

[DennisClark]

See issue #1187 . closing this one.