Duplicate components are valid with sbom in json format but not in xml

[Frazerus]

Intro

Validating a duplicate component (one instance in the metadata and one in the components section) in the json SBOM does not create a validation error. The same duplicate does create a validation error in the xml SBOM.

Used version: 0.27.2

The big question

Is the xml behaviour correct, or the json behaviour? This also creates a problem with #326 (comment), as the metadata component is duplicated, reproducing this exact issue in xml, while not creating a problem in json.

Repro

When an sbom is in xml and has a duplicate component (sambo) between the metadata component and a normal one, the validation fails:

<bom xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" serialNumber="urn:uuid:15ff9d24-9079-45dc-8afc-33b61a0498b5" version="1" xmlns="http://cyclonedx.org/schema/bom/1.6">
  <metadata>
    <component type="application" bom-ref="@sambo@4.4.4">
      <name>sambo</name>
      <version>4.4.4</version>
      <purl>pkg:npm/%40sambo@4.4.4</purl>
      <properties>
        <property name="cdx:npm:package:path" />
      </properties>
    </component>
  </metadata>
  <components>
    <component type="library" bom-ref="axe-core@4.8.2">
      <name>axe-core</name>
      <version>4.8.2</version>
      <description>Accessibility engine for automated Web UI testing</description>
      <licenses>
        <license>
          <id>MPL-2.0</id>
        </license>
      </licenses>
      <purl>pkg:npm/axe-core@4.8.2</purl>
      <properties>
        <property name="cdx:npm:package:path">node_modules/axe-core</property>
      </properties>
    </component>
    <component type="application" bom-ref="@sambo@4.4.4">
      <name>sambo</name>
      <version>4.4.4</version>
      <purl>pkg:npm/%40sambo@4.4.4</purl>
      <properties>
        <property name="cdx:npm:package:path" />
      </properties>
    </component>
  </components>
  <dependencies>
    <dependency ref="@sambo@4.4.4">
      <dependency ref="axe-core@4.8.2" />
    </dependency>
    <dependency ref="axe-core@4.8.2" />
  </dependencies>
</bom>

 .\cyclonedx-cli.exe validate --input-file test.xml --input-version v1_6
Validating XML BOM...
Validation failed at line number 82 and position 7: There is a duplicate key sequence '@sambo@4.4.4' for the 'http://cyclonedx.org/schema/bom/1.6:bom-ref' key or unique identity constraint.
BOM is not valid.

When I convert that sbom to a json one:

.\cyclonedx-cli.exe convert --input-file test.xml --output-file test.json

{
  "bomFormat": "CycloneDX",
  "specVersion": "1.6",
  "serialNumber": "urn:uuid:15ff9d24-9079-45dc-8afc-33b61a0498b5",
  "version": 1,
  "metadata": {
    "component": {
      "type": "application",
      "bom-ref": "@sambo@4.4.4",
      "authors": [],
      "name": "sambo",
      "version": "4.4.4",
      "purl": "pkg:npm/%40sambo@4.4.4",
      "properties": [
        {
          "name": "cdx:npm:package:path"
        }
      ],
      "tags": [],
      "omniborId": [],
      "swhid": []
    },
    "lifecycles": []
  },
  "components": [
    {
      "type": "library",
      "bom-ref": "axe-core@4.8.2",
      "authors": [],
      "name": "axe-core",
      "version": "4.8.2",
      "description": "Accessibility engine for automated Web UI testing",
      "licenses": [
        {
          "license": {
            "id": "MPL-2.0",
            "properties": []
          }
        }
      ],
      "purl": "pkg:npm/axe-core@4.8.2",
      "properties": [
        {
          "name": "cdx:npm:package:path",
          "value": "node_modules/axe-core"
        }
      ],
      "tags": [],
      "omniborId": [],
      "swhid": []
    },
    {
      "type": "application",
      "bom-ref": "@sambo@4.4.4",
      "authors": [],
      "name": "sambo",
      "version": "4.4.4",
      "purl": "pkg:npm/%40sambo@4.4.4",
      "properties": [
        {
          "name": "cdx:npm:package:path"
        }
      ],
      "tags": [],
      "omniborId": [],
      "swhid": []
    }
  ],
  "dependencies": [
    {
      "ref": "@sambo@4.4.4",
      "dependsOn": [
        "axe-core@4.8.2"
      ],
      "provides": []
    },
    {
      "ref": "axe-core@4.8.2",
      "provides": []
    }
  ],
  "vulnerabilities": [],
  "annotations": [],
  "properties": [],
  "formulation": []
}

and validate that, the bom validates successfully, even though the sambo component exists in the metadata and the components list:

.\cyclonedx-cli.exe validate --input-file test.json --input-version v1_6
Validating JSON BOM...
BOM validated successfully.

[Frazerus]

The workaround of adding an extra name and version as a metadata component does not really work. This is because when later merging the sbom in a downstream project, the merge adds those metadata components which do not represent parts of the application.

[andreas-hilti]

Is the xml behaviour correct, or the json behaviour?

As far as I'm aware, the xml behavior is correct. All bom-refs must be unique in the entire BOM:
https://cyclonedx.org/docs/1.6/json/#components_items_bom-ref

Every bom-ref must be unique within the BOM.

The xml schema allows to enforce this directly in the schema, json schema doesn't have an equivalent concept. At the moment, all the validation does is to check against the schema. There is a PR to detect duplicated bom-refs, but it hasn't yet made it into a release:
CycloneDX/cyclonedx-dotnet-library#240

[andreas-hilti]

In addition, the fact that flat merges can cause issues with duplicated bom-refs is known (there are several open issues). The behavior has been partially improved, but there isn't a complete agreement on how to approach it (and there are several open PRs that try to improve it).