feat(mitre): add new datasource: mitre #392

MaineK00n · 2024-06-27T09:21:39Z

If this Pull Request is work in progress, Add a prefix of “[WIP]” in the title.

What did you implement:

Type of change

New feature (non-breaking change which adds functionality)
This change requires a documentation update

How Has This Been Tested?

$ go-cve-dictionary fetch mitre
$ go-cve-dictionary search cve
[
  "CVE-2021-34564",
  "CVE-2023-30720",
  "CVE-2015-0434",
...
$ go-cve-dictionary search cve CVE-2024-5732
{
  "CveID": "CVE-2024-5732",
  "Nvds": [],
  "Jvns": [],
  "Fortinets": [],
  "Mitres": [
    {
      "DataType": "",
      "DataVersion": "",
      "CVEMetadata": {
        "CVEID": "CVE-2024-5732",
        "AssignerOrgID": "1af790b2-7ee1-4545-860a-a788eba489b5",
        "AssignerShortName": "VulDB",
        "RequesterUserID": null,
        "Serial": null,
        "State": "PUBLISHED",
        "DatePublished": "2024-06-07T10:00:04.02Z",
        "DateUpdated": "2024-06-07T14:50:46.944Z",
        "DateReserved": "2024-06-07T05:12:19.233Z",
        "DateRejected": null
      },
      "Containers": [
        {
          "ContainerType": "CNA",
          "ProviderMetadata": {
            "OrgID": "1af790b2-7ee1-4545-860a-a788eba489b5",
            "ShortName": "VulDB",
            "DateUpdated": "2024-06-07T10:00:04.02Z"
          },
          "Title": "Clash Proxy Port improper authentication",
          "Descriptions": [
            {
              "Lang": "en",
              "Value": "A vulnerability was found in Clash up to 0.20.1 on Windows. It has been declared as critical. This vulnerability affects unknown code of the component Proxy Port. The manipulation leads to improper authentication. The attack can be initiated remotely. The exploit has been disclosed to the public and may be used. It is recommended to change the configuration settings. VDB-267406 is the identifier assigned to this vulnerability.",
              "SupportingMedia": []
            },
            {
              "Lang": "de",
              "Value": "In Clash bis 0.20.1 für Windows wurde eine Schwachstelle ausgemacht. Sie wurde als kritisch eingestuft. Das betrifft eine unbekannte Funktionalität der Komponente Proxy Port. Durch das Manipulieren mit unbekannten Daten kann eine improper authentication-Schwachstelle ausgenutzt werden. Der Angriff kann über das Netzwerk angegangen werden. Der Exploit steht zur öffentlichen Verfügung. Als bestmögliche Massnahme werden Anpassungen an der Konfiguration empfohlen.",
              "SupportingMedia": []
            }
          ],
          "ProblemTypes": [
            {
              "Descriptions": [
                {
                  "Type": "CWE",
                  "Lang": "en",
                  "Description": "CWE-287 Improper Authentication",
                  "CweID": "CWE-287",
                  "References": []
                }
              ]
            }
          ],
          "Impacts": [],
          "Metrics": [
            {
              "Format": "CVSS",
              "Scenarios": [],
              "CVSSv2": null,
              "CVSSv30": null,
              "CVSSv31": null,
              "CVSSv40": {
                "VectorString": "CVSS:4.0/AV:N/AC:L/AT:N/PR:N/UI:N/VC:L/VI:L/VA:L/SC:N/SI:N/SA:N",
                "baseScore": 6.9,
                "BaseSeverity": "MEDIUM",
                "ThreatScore": null,
                "ThreatSeverity": null,
                "EnvironmentalScore": null,
                "EnvironmentalSeverity": null
              },
              "SSVC": null,
              "KEV": null,
              "Other": null
            },
            {
              "Format": "CVSS",
              "Scenarios": [],
              "CVSSv2": null,
              "CVSSv30": null,
              "CVSSv31": {
                "VectorString": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:L/I:L/A:L",
                "AttackVector": "",
                "AttackComplexity": "",
                "PrivilegesRequired": "",
                "UserInteraction": "",
                "Scope": "",
                "ConfidentialityImpact": "",
                "IntegrityImpact": "",
                "AvailabilityImpact": "",
                "BaseScore": 7.3,
                "BaseSeverity": "HIGH",
                "ExploitabilityScore": 0,
                "ImpactScore": 0
              },
              "CVSSv40": null,
              "SSVC": null,
              "KEV": null,
              "Other": null
            },
            {
              "Format": "CVSS",
              "Scenarios": [],
              "CVSSv2": null,
              "CVSSv30": {
                "VectorString": "CVSS:3.0/AV:N/AC:L/PR:N/UI:N/S:U/C:L/I:L/A:L",
                "AttackVector": "",
                "AttackComplexity": "",
                "PrivilegesRequired": "",
                "UserInteraction": "",
                "Scope": "",
                "ConfidentialityImpact": "",
                "IntegrityImpact": "",
                "AvailabilityImpact": "",
                "BaseScore": 7.3,
                "BaseSeverity": "HIGH",
                "ExploitabilityScore": 0,
                "ImpactScore": 0
              },
              "CVSSv31": null,
              "CVSSv40": null,
              "SSVC": null,
              "KEV": null,
              "Other": null
            },
            {
              "Format": "CVSS",
              "Scenarios": [],
              "CVSSv2": {
                "VectorString": "AV:N/AC:L/Au:N/C:P/I:P/A:P",
                "AccessVector": "",
                "AccessComplexity": "",
                "Authentication": "",
                "ConfidentialityImpact": "",
                "IntegrityImpact": "",
                "AvailabilityImpact": "",
                "BaseScore": 7.5,
                "Severity": ""
              },
              "CVSSv30": null,
              "CVSSv31": null,
              "CVSSv40": null,
              "SSVC": null,
              "KEV": null,
              "Other": null
            }
          ],
          "Workarounds": [],
          "Solutions": [],
          "Exploits": [],
          "Configurations": [],
          "References": [
            {
              "Link": "https://vuldb.com/?id.267406",
              "Source": "",
              "Tags": "vdb-entry",
              "Name": "VDB-267406 | Clash Proxy Port improper authentication"
            },
            {
              "Link": "https://vuldb.com/?ctiid.267406",
              "Source": "",
              "Tags": "signature,permissions-required",
              "Name": "VDB-267406 | CTI Indicators (IOB, IOC)"
            },
            {
              "Link": "https://vuldb.com/?submit.345469",
              "Source": "",
              "Tags": "third-party-advisory",
              "Name": "Submit #345469 | clash for windows 0.10-0.0.20.1 Unverified Ownership"
            },
            {
              "Link": "https://github.com/GTA12138/vul/blob/main/clash%20for%20windows.md",
              "Source": "",
              "Tags": "exploit",
              "Name": ""
            }
          ],
          "Timeline": [
            {
              "Time": "2024-06-07T00:00:00Z",
              "Lang": "en",
              "Value": "Advisory disclosed"
            },
            {
              "Time": "2024-06-07T02:00:00Z",
              "Lang": "en",
              "Value": "VulDB entry created"
            },
            {
              "Time": "2024-06-07T07:17:38Z",
              "Lang": "en",
              "Value": "VulDB entry last update"
            }
          ],
          "Credits": [
            {
              "Type": "reporter",
              "Lang": "en",
              "User": null,
              "Value": "rollingchair (VulDB User)"
            }
          ],
          "Source": "",
          "Tags": [],
          "TaxonomyMappings": [],
          "DateAssigned": null,
          "DatePublic": null
        },
        {
          "ContainerType": "ADP",
          "ProviderMetadata": {
            "OrgID": "134c704f-9b21-4f2e-91b3-4a467353bcc0",
            "ShortName": "CISA-ADP",
            "DateUpdated": "2024-06-07T14:50:46.944Z"
          },
          "Title": "CISA ADP Vulnrichment",
          "Descriptions": [],
          "ProblemTypes": [],
          "Impacts": [],
          "Metrics": [
            {
              "Format": "SSVC",
              "Scenarios": [],
              "CVSSv2": null,
              "CVSSv30": null,
              "CVSSv31": null,
              "CVSSv40": null,
              "SSVC": {
                "Role": "CISA Coordinator",
                "Version": "2.0.3",
                "Timestamp": "2024-06-07T14:50:35.35155Z",
                "Exploitation": "none",
                "Automatable": "no",
                "TechnicalImpact": "partial"
              },
              "KEV": null,
              "Other": null
            }
          ],
          "Workarounds": [],
          "Solutions": [],
          "Exploits": [],
          "Configurations": [],
          "References": [],
          "Timeline": [],
          "Credits": [],
          "Source": "",
          "Tags": [],
          "TaxonomyMappings": [],
          "DateAssigned": null,
          "DatePublic": null
        }
      ]
    }
  ]
}

Checklist:

You don't have to satisfy all of the following.

Write tests
Write documentation
Check that there aren't other open pull requests for the same issue/feature
Format your source code by make fmt
Pass the test by make test
Provide verification config / commands
Enable "Allow edits from maintainers" for this PR
Update the messages below

Is this ready for review?: YES

Reference

commands/fetchmitre.go

fetcher/mitre/mitre.go

shino

Fantastic!

jbmaillet · 2024-06-28T13:39:13Z

It will probably breaks my workflow at first, but it looks really interesting! Thanks. 👍

jbmaillet · 2024-07-01T14:20:32Z

I gave it a try, some feedback using CVE-2024-36971 as an example. It is a Linux kernel CVE.

Here it is, as from the CNA itself on the linux-cve-announce.vger.kernel.org mailing list, with lots of details:
https://lore.kernel.org/linux-cve-announce/[email protected]/
Here it is on CVE.org:
https://www.cve.org/CVERecord?id=CVE-2024-36971
...and from CVE.org as well, as JSON:
https://cveawg.mitre.org/api/cve/CVE-2024-36971
With its details such as impacted ProgramFiles and the git sha1 that introduced and fixed the bug. Needless to say, we do not have this on the NVD, and in my use cases this is where CVE.org would be really useful.

Using go-cve-dictionary master 73f1570, I have some info from Mitre, but not these details. I can think of a few possible reasons for this:

This is a misunderstanding from me, as this could be a first needed step before possible future development. For example, as of today, CVE.org announced the availability of CVE Record Format Version 5.1.0 (https://www.cve.org/Media/News/item/blog/2024/05/09/CVE-Record-Format-CVE-Services-Updated), while this go-cve-dictionary development is linked to #287, for format 5.0

To add to the confusion, https://cveawg.mitre.org/api/cve/CVE-2024-0564 clearly indicates dataType CVE_RECORD dataVersion 5.0, while my https://cveawg.mitre.org/api/cve/CVE-2024-36971 example does not indicate a dataVersion at all. This is not correct accordingly to the note at the top of https://github.com/CVEProject/cvelistV5/ README. I opened an issue about that: CVEProject/cvelistV5#57. While on this matter, I could not find explicit references to the schema v4 or v5 in go-cve-dictionary code, am I correct?

(And of course, the NVD and CVE ecosystem chaos that does not help, for sure.)

MaineK00n · 2024-07-02T03:17:05Z

@jbmaillet
Currently, the information that is fetched with go-cve-dictionary fetch mitre does not include affected information.
go-cve-dictionary has a function to detect with CPE, but it was difficult to match the affected information to go-cve-dictionary.

jbmaillet · 2024-07-02T12:53:30Z

When I mention version 5.0 or version 5.1, I mean the version of the CVE record format, also known as "the CVE JSON format". It has no relationship with the affected product version, which is very difficult to do (if possible as of today) from MITRE / CVE.org information since they do not use formal CPE.

On CVE-2024-36971 in my example above I taught it was missing, but you noticed it is at the bottom of the JSON here:
https://github.com/CVEProject/cvelistV5/blob/main/cves/2024/36xxx/CVE-2024-36971.json#L198

But I can see that these data are not present in a go-cve-dictionary database:

...the list of affected files:
https://github.com/CVEProject/cvelistV5/blob/main/cves/2024/36xxx/CVE-2024-36971.json#L31

...the list of affected versions, though we do not have CPE, we have much better data quality than a CPE:
https://github.com/CVEProject/cvelistV5/blob/main/cves/2024/36xxx/CVE-2024-36971.json#L38
https://github.com/CVEProject/cvelistV5/blob/main/cves/2024/36xxx/CVE-2024-36971.json#L101

...the DataType is missing as well while it should be "CVE_RECORD":
https://github.com/CVEProject/cvelistV5/blob/main/cves/2024/36xxx/CVE-2024-36971.json#L2
...and the DataVersion too, while it should be "5.1":
https://github.com/CVEProject/cvelistV5/blob/main/cves/2024/36xxx/CVE-2024-36971.json#L198C20-L198C25

All this is missing from my go-cve-dictionary database using NVD + JVN + MITRE.

My question: are these data missing because your work is about CVE Record format version 5.0, not yet 5.1?

jbmaillet · 2024-07-04T17:32:48Z

Clarification regarding my question above in #395
Question closed.

MaineK00n self-assigned this Jun 27, 2024

MaineK00n linked an issue Jun 27, 2024 that may be closed by this pull request

add support for CVSS 4.0 #351

Closed

MaineK00n requested a review from shino June 27, 2024 09:22

MaineK00n force-pushed the MaineK00n/vulnrichment branch 3 times, most recently from 56ec478 to 9226db0 Compare June 27, 2024 14:11

feat(mitre): add new datasource: mitre

867f0ed

MaineK00n force-pushed the MaineK00n/vulnrichment branch from 9226db0 to 867f0ed Compare June 27, 2024 17:51

This was referenced Jun 28, 2024

feat(cve/mitre): support go-cve-dictionary:mitre future-architect/vuls#1978

Merged

feat(cvedb/mitre): support for fetching mitre vulsio/vulsctl#81

Merged

shino reviewed Jun 28, 2024

View reviewed changes

commands/fetchmitre.go Outdated Show resolved Hide resolved

shino reviewed Jun 28, 2024

View reviewed changes

fetcher/mitre/mitre.go Outdated Show resolved Hide resolved

shino reviewed Jun 28, 2024

View reviewed changes

fetcher/mitre/mitre.go Outdated Show resolved Hide resolved

chore: adopt reviewer comments

638ba0c

MaineK00n force-pushed the MaineK00n/vulnrichment branch from 46146ad to 638ba0c Compare June 28, 2024 06:39

shino approved these changes Jun 28, 2024

View reviewed changes

MaineK00n merged commit 73f1570 into master Jun 28, 2024
6 checks passed

MaineK00n deleted the MaineK00n/vulnrichment branch June 28, 2024 07:26

jbmaillet mentioned this pull request Jul 4, 2024

feat(mitre): fill DataType, DataVersion, affected #395

Merged

9 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(mitre): add new datasource: mitre #392

feat(mitre): add new datasource: mitre #392

MaineK00n commented Jun 27, 2024

shino left a comment

jbmaillet commented Jun 28, 2024 •

edited

Loading

jbmaillet commented Jul 1, 2024

MaineK00n commented Jul 2, 2024

jbmaillet commented Jul 2, 2024 •

edited

Loading

jbmaillet commented Jul 4, 2024

feat(mitre): add new datasource: mitre #392

feat(mitre): add new datasource: mitre #392

Conversation

MaineK00n commented Jun 27, 2024

What did you implement:

Type of change

How Has This Been Tested?

Checklist:

Reference

shino left a comment

Choose a reason for hiding this comment

jbmaillet commented Jun 28, 2024 • edited Loading

jbmaillet commented Jul 1, 2024

MaineK00n commented Jul 2, 2024

jbmaillet commented Jul 2, 2024 • edited Loading

jbmaillet commented Jul 4, 2024

jbmaillet commented Jun 28, 2024 •

edited

Loading

jbmaillet commented Jul 2, 2024 •

edited

Loading