Skip to content

Enhance CNVM index#1081

Merged
jeniawhite merged 13 commits intoelastic:mainfrom
jeniawhite:evgb-VulnIndexChanges
Jul 16, 2023
Merged

Enhance CNVM index#1081
jeniawhite merged 13 commits intoelastic:mainfrom
jeniawhite:evgb-VulnIndexChanges

Conversation

@jeniawhite
Copy link
Copy Markdown
Contributor

@jeniawhite jeniawhite commented Jun 26, 2023

Summary of your changes

Enhancing the CNVM index with additional data from the instance that is being scanned.
Added cloud, network, host, and security group information to the index.
Note that the cloud and host data is regarding scanned instances and not the scanner instance.
This is why we did not use cloud and host processors because they collect information for the scanner machine.

Example of event:

{
  "_index": ".ds-logs-cloud_security_posture.vulnerabilities-default-2023.07.16-000001",
  "_id": "K7c5YYkBz2hMpetn5RA-",
  "_version": 1,
  "_score": 0,
  "_ignored": [
    "vulnerability.description"
  ],
  "_source": {
    "agent": {
      "name": "ip-172-31-32-84.eu-west-1.compute.internal",
      "id": "1f859814-361a-4693-8717-fdbd2b9c45c7",
      "type": "cloudbeat",
      "ephemeral_id": "39524697-5b09-4a5e-bfdc-abdd1bc0983f",
      "version": "8.9.0"
    },
    "package": {
      "path": "Python",
      "fixed_version": "39.0.1",
      "name": "cryptography",
      "type": "python-pkg",
      "version": "3.4.8"
    },
    "resource": {
      "name": "amir-env-imV",
      "id": "0214c95f3c81ea3bc"
    },
    "elastic_agent": {
      "id": "1f859814-361a-4693-8717-fdbd2b9c45c7",
      "version": "8.9.0",
      "snapshot": false
    },
    "vulnerability": {
      "severity": "HIGH",
      "package": {
        "fixed_version": "39.0.1",
        "name": "cryptography",
        "version": "3.4.8"
      },
      "description": "There is a type confusion vulnerability relating to X.400 address processing inside an X.509 GeneralName. X.400 addresses were parsed as an ASN1_STRING but the public structure definition for GENERAL_NAME incorrectly specified the type of the x400Address field as ASN1_TYPE. This field is subsequently interpreted by the OpenSSL function GENERAL_NAME_cmp as an ASN1_TYPE rather than an ASN1_STRING. When CRL checking is enabled (i.e. the application sets the X509_V_FLAG_CRL_CHECK flag), this vulnerability may allow an attacker to pass arbitrary pointers to a memcmp call, enabling them to read memory contents or enact a denial of service. In most cases, the attack requires the attacker to provide both the certificate chain and CRL, neither of which need to have a valid signature. If the attacker only controls one of these inputs, the other input must already contain an X.400 address as a CRL distribution point, which is uncommon. As such, this vulnerability is most likely to only affect applications which have implemented their own functionality for retrieving CRLs over a network.",
      "title": "X.400 address type confusion in X.509 GeneralName",
      "classification": "CVSS",
      "data_source": {
        "ID": "ghsa",
        "URL": "https://github.com/advisories?query=type%3Areviewed+ecosystem%3Apip",
        "Name": "GitHub Security Advisory Pip"
      },
      "cwe": [
        "CWE-843"
      ],
      "reference": "https://avd.aquasec.com/nvd/cve-2023-0286",
      "score": {
        "version": "3.1",
        "base": 7.4
      },
      "report_id": 1689519799,
      "scanner": {
        "vendor": "Trivy",
        "version": "v0.35.0"
      },
      "id": "CVE-2023-0286",
      "enumeration": "CVE",
      "category": "lang-pkgs",
      "class": "lang-pkgs",
      "cvss": {
        "redhat": {
          "V3Vector": "CVSS:3.1/AV:N/AC:H/PR:N/UI:N/S:U/C:H/I:N/A:H",
          "V3Score": 7.4
        },
        "nvd": {
          "V3Vector": "CVSS:3.1/AV:N/AC:H/PR:N/UI:N/S:U/C:H/I:N/A:H",
          "V3Score": 7.4
        },
        "ghsa": {
          "V3Vector": "CVSS:3.1/AV:N/AC:H/PR:N/UI:N/S:U/C:H/I:N/A:H",
          "V3Score": 7.4
        }
      },
      "published_date": "2023-02-08T20:15:00Z"
    },
    "network": {
      "public_ip": "54.154.24.188",
      "private_ip": "172.31.13.27",
      "mac_addresses": [
        "02:bf:d7:76:6b:4d"
      ]
    },
    "cloud": {
      "availability_zone": "eu-west-1a",
      "instance": {
        "name": "amir-env-imV",
        "id": "0214c95f3c81ea3bc"
      },
      "provider": "aws",
      "service": {
        "name": "AWS EC2"
      },
      "machine": {
        "launch_time": "2023-06-28T14:23:33Z",
        "image": "ami-0a5b3305c37e58e04",
        "type": "c5.4xlarge",
        "authentication": {
          "key": "cloudbeat-generated-4c687128"
        }
      },
      "region": "eu-west-1",
      "account": {
        "name": "elastic-security-cloud-security-dev",
        "id": "704479110758"
      },
      "tags": {
        "division": "engineering",
        "provisioner": "terraform",
        "org": "security",
        "ec2_type": "kspm",
        "project": "test-environments",
        "team": "cloud-security-posture",
        "id": "4c687128",
        "Name": "amir-env-imV"
      }
    },
    "security": {
      "security_groups": [
        {
          "group_id": "sg-040c153bec4b15dc2",
          "group_name": "terraform-20230628142329353700000001"
        }
      ]
    },
    "@timestamp": "2023-07-17T00:22:03.491Z",
    "ecs": {
      "version": "8.6.0"
    },
    "cloudbeat": {
      "commit_sha": "2a9a2de4d2cac154d4bb7fddf453abd7add47c17",
      "commit_time": "2023-07-16T13:35:59Z",
      "version": "8.9.0"
    },
    "data_stream": {
      "namespace": "default",
      "type": "logs",
      "dataset": "cloud_security_posture.vulnerabilities"
    },
    "host": {
      "os": {
        "platform": "Linux/UNIX"
      },
      "name": "amir-env-imV",
      "architecture": "x86_64"
    },
    "event": {
      "agent_id_status": "verified",
      "sequence": 1689519799,
      "ingested": "2023-07-17T00:22:11Z",
      "created": "2023-07-17T00:22:03.491025338Z",
      "kind": "state",
      "id": "0048cfc1-ed60-489c-8558-76291d1ec259",
      "category": [
        "vulnerability"
      ],
      "type": [
        "info"
      ],
      "dataset": "cloud_security_posture.vulnerabilities",
      "outcome": "success"
    }
  },
  "fields": {
    "vulnerability.package.version": [
      "3.4.8"
    ],
    "event.category": [
      "vulnerability"
    ],
    "elastic_agent.version": [
      "8.9.0"
    ],
    "vulnerability.cvss.nvd.V3Vector": [
      "CVSS:3.1/AV:N/AC:H/PR:N/UI:N/S:U/C:H/I:N/A:H"
    ],
    "vulnerability.cvss.ghsa.V3Vector": [
      "CVSS:3.1/AV:N/AC:H/PR:N/UI:N/S:U/C:H/I:N/A:H"
    ],
    "vulnerability.scanner.vendor": [
      "Trivy"
    ],
    "vulnerability.data_source.ID": [
      "ghsa"
    ],
    "cloud.availability_zone": [
      "eu-west-1a"
    ],
    "package.type": [
      "python-pkg"
    ],
    "cloud.tags.id": [
      "4c687128"
    ],
    "vulnerability.data_source.URL": [
      "https://github.com/advisories?query=type%3Areviewed+ecosystem%3Apip"
    ],
    "agent.name": [
      "ip-172-31-32-84.eu-west-1.compute.internal"
    ],
    "host.name": [
      "amir-env-imV"
    ],
    "event.agent_id_status": [
      "verified"
    ],
    "event.kind": [
      "state"
    ],
    "cloud.tags.project": [
      "test-environments"
    ],
    "event.outcome": [
      "success"
    ],
    "package.version": [
      "3.4.8"
    ],
    "cloud.region": [
      "eu-west-1"
    ],
    "package.path": [
      "Python"
    ],
    "vulnerability.title": [
      "X.400 address type confusion in X.509 GeneralName"
    ],
    "cloud.machine.image": [
      "ami-0a5b3305c37e58e04"
    ],
    "data_stream.type": [
      "logs"
    ],
    "cloud.tags.org": [
      "security"
    ],
    "host.architecture": [
      "x86_64"
    ],
    "cloud.provider": [
      "aws"
    ],
    "cloud.machine.type": [
      "c5.4xlarge"
    ],
    "cloudbeat.version": [
      "8.9.0"
    ],
    "cloud.tags.Name": [
      "amir-env-imV"
    ],
    "agent.id": [
      "1f859814-361a-4693-8717-fdbd2b9c45c7"
    ],
    "cloud.service.name": [
      "AWS EC2"
    ],
    "ecs.version": [
      "8.6.0"
    ],
    "event.created": [
      "2023-07-17T00:22:03.491Z"
    ],
    "cloud.machine.launch_time": [
      "2023-06-28T14:23:33Z"
    ],
    "agent.version": [
      "8.9.0"
    ],
    "vulnerability.reference": [
      "https://avd.aquasec.com/nvd/cve-2023-0286"
    ],
    "vulnerability.cvss.nvd.V3Score": [
      7.4
    ],
    "cloudbeat.commit_sha": [
      "2a9a2de4d2cac154d4bb7fddf453abd7add47c17"
    ],
    "vulnerability.package.fixed_version": [
      "39.0.1"
    ],
    "vulnerability.enumeration": [
      "CVE"
    ],
    "vulnerability.cvss.redhat.V3Vector": [
      "CVSS:3.1/AV:N/AC:H/PR:N/UI:N/S:U/C:H/I:N/A:H"
    ],
    "vulnerability.published_date": [
      "2023-02-08T20:15:00Z"
    ],
    "resource.name": [
      "amir-env-imV"
    ],
    "vulnerability.package.name": [
      "cryptography"
    ],
    "event.sequence": [
      1689519799
    ],
    "cloud.instance.id": [
      "0214c95f3c81ea3bc"
    ],
    "agent.type": [
      "cloudbeat"
    ],
    "vulnerability.id": [
      "CVE-2023-0286"
    ],
    "vulnerability.score.version": [
      "3.1"
    ],
    "package.name": [
      "cryptography"
    ],
    "vulnerability.score.base": [
      7.4
    ],
    "elastic_agent.snapshot": [
      false
    ],
    "network.mac_addresses": [
      "02:bf:d7:76:6b:4d"
    ],
    "vulnerability.report_id": [
      "1.689519799E9"
    ],
    "cloud.tags.division": [
      "engineering"
    ],
    "cloud.account.name": [
      "elastic-security-cloud-security-dev"
    ],
    "vulnerability.classification": [
      "CVSS"
    ],
    "elastic_agent.id": [
      "1f859814-361a-4693-8717-fdbd2b9c45c7"
    ],
    "data_stream.namespace": [
      "default"
    ],
    "cloud.machine.authentication.key": [
      "cloudbeat-generated-4c687128"
    ],
    "vulnerability.class": [
      "lang-pkgs"
    ],
    "security.security_groups.group_id": [
      "sg-040c153bec4b15dc2"
    ],
    "vulnerability.cwe": [
      "CWE-843"
    ],
    "vulnerability.scanner.version": [
      "v0.35.0"
    ],
    "cloud.tags.provisioner": [
      "terraform"
    ],
    "resource.id": [
      "0214c95f3c81ea3bc"
    ],
    "vulnerability.category": [
      "lang-pkgs"
    ],
    "network.private_ip": [
      "172.31.13.27"
    ],
    "cloud.tags.ec2_type": [
      "kspm"
    ],
    "event.ingested": [
      "2023-07-17T00:22:11.000Z"
    ],
    "@timestamp": [
      "2023-07-17T00:22:03.491Z"
    ],
    "vulnerability.severity": [
      "HIGH"
    ],
    "security.security_groups.group_name": [
      "terraform-20230628142329353700000001"
    ],
    "cloud.account.id": [
      "704479110758"
    ],
    "host.os.platform": [
      "Linux/UNIX"
    ],
    "event.type": [
      "info"
    ],
    "data_stream.dataset": [
      "cloud_security_posture.vulnerabilities"
    ],
    "package.fixed_version": [
      "39.0.1"
    ],
    "vulnerability.data_source.Name": [
      "GitHub Security Advisory Pip"
    ],
    "agent.ephemeral_id": [
      "39524697-5b09-4a5e-bfdc-abdd1bc0983f"
    ],
    "vulnerability.cvss.redhat.V3Score": [
      7.4
    ],
    "vulnerability.cvss.ghsa.V3Score": [
      7.4
    ],
    "vulnerability.description.text": [
      "There is a type confusion vulnerability relating to X.400 address processing inside an X.509 GeneralName. X.400 addresses were parsed as an ASN1_STRING but the public structure definition for GENERAL_NAME incorrectly specified the type of the x400Address field as ASN1_TYPE. This field is subsequently interpreted by the OpenSSL function GENERAL_NAME_cmp as an ASN1_TYPE rather than an ASN1_STRING. When CRL checking is enabled (i.e. the application sets the X509_V_FLAG_CRL_CHECK flag), this vulnerability may allow an attacker to pass arbitrary pointers to a memcmp call, enabling them to read memory contents or enact a denial of service. In most cases, the attack requires the attacker to provide both the certificate chain and CRL, neither of which need to have a valid signature. If the attacker only controls one of these inputs, the other input must already contain an X.400 address as a CRL distribution point, which is uncommon. As such, this vulnerability is most likely to only affect applications which have implemented their own functionality for retrieving CRLs over a network."
    ],
    "event.id": [
      "0048cfc1-ed60-489c-8558-76291d1ec259"
    ],
    "cloudbeat.commit_time": [
      "2023-07-16T13:35:59.000Z"
    ],
    "network.public_ip": [
      "54.154.24.188"
    ],
    "cloud.tags.team": [
      "cloud-security-posture"
    ],
    "event.dataset": [
      "cloud_security_posture.vulnerabilities"
    ],
    "cloud.instance.name": [
      "amir-env-imV"
    ]
  },
  "ignored_field_values": {
    "vulnerability.description": [
      "There is a type confusion vulnerability relating to X.400 address processing inside an X.509 GeneralName. X.400 addresses were parsed as an ASN1_STRING but the public structure definition for GENERAL_NAME incorrectly specified the type of the x400Address field as ASN1_TYPE. This field is subsequently interpreted by the OpenSSL function GENERAL_NAME_cmp as an ASN1_TYPE rather than an ASN1_STRING. When CRL checking is enabled (i.e. the application sets the X509_V_FLAG_CRL_CHECK flag), this vulnerability may allow an attacker to pass arbitrary pointers to a memcmp call, enabling them to read memory contents or enact a denial of service. In most cases, the attack requires the attacker to provide both the certificate chain and CRL, neither of which need to have a valid signature. If the attacker only controls one of these inputs, the other input must already contain an X.400 address as a CRL distribution point, which is uncommon. As such, this vulnerability is most likely to only affect applications which have implemented their own functionality for retrieving CRLs over a network."
    ]
  }
}

Related Issues

Checklist

  • I have added tests that prove my fix is effective or that my feature works
  • I have added the necessary README/documentation (if appropriate)

@jeniawhite jeniawhite requested a review from a team as a code owner June 26, 2023 01:31
@mergify
Copy link
Copy Markdown

mergify bot commented Jun 26, 2023

This pull request does not have a backport label. Could you fix it @jeniawhite? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

  • backport-v./d./d./d is the label to automatically backport to the 8./d branch. /d is the digit
    NOTE: backport-skip has been added to this pull request.

@github-actions
Copy link
Copy Markdown

github-actions bot commented Jun 26, 2023

📊 Allure Report - 💚 No failures were reported.

Result Count
🟥 Failed 0
🟩 Passed 33
⬜ Skipped 1

@mergify
Copy link
Copy Markdown

mergify bot commented Jun 26, 2023

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b evgb-VulnIndexChanges upstream/evgb-VulnIndexChanges
git merge upstream/main
git push upstream evgb-VulnIndexChanges

Copy link
Copy Markdown
Contributor

@amirbenun amirbenun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, can you please print a single vulnerability?


type Machine struct {
Type string `json:"type,omitempty"`
Key AuthInfo `json:"key,omitempty"`
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will create cloud.machine.key.key, please rename to authentication

@jeniawhite jeniawhite force-pushed the evgb-VulnIndexChanges branch from d4299ff to a1ff8bc Compare July 4, 2023 07:25
Comment on lines +313 to +339
func getMacAddresses(interfaces []ec2Types.InstanceNetworkInterface) []string {
macAddresses := make([]string, len(interfaces))
for i, iface := range interfaces {
macAddresses[i] = *iface.MacAddress
}
return macAddresses
}

func getInstanceTags(tags []ec2Types.Tag) map[string]string {
instanceTags := make(map[string]string, len(tags))
for _, tag := range tags {
instanceTags[*tag.Key] = *tag.Value
}
return instanceTags
}

func getSecurityGroups(groups []ec2Types.GroupIdentifier) []SecurityGroupInfo {
securityGroups := make([]SecurityGroupInfo, len(groups))
for i, group := range groups {
securityGroups[i] = SecurityGroupInfo{
GroupId: group.GroupId,
GroupName: group.GroupName,
}
}
return securityGroups
}

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just wondering if maybe those methods should reside under the ec2 provider.

@jeniawhite jeniawhite force-pushed the evgb-VulnIndexChanges branch from c01cc7e to 81c1376 Compare July 5, 2023 11:35
@jeniawhite
Copy link
Copy Markdown
Contributor Author

jeniawhite commented Jul 5, 2023

@amirbenun
Please review fields added/removed due to this document.
Did not map anything in ECS due to this PR:

The current PR is reliant on the above PR to merge and map the ECS fields.

Copy link
Copy Markdown
Contributor

@amirbenun amirbenun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@opauloh, we want to deprecate a few fields vulnerability.class, resource.id, resource.name.
Can you please check our FE is using any of them?

@opauloh
Copy link
Copy Markdown
Contributor

opauloh commented Jul 5, 2023

@opauloh, we want to deprecate a few fields vulnerability.class, resource.id, resource.name. Can you please check our FE is using any of them?

@amirbenun the vulnerability.class field is currently not being used.

The fields resource.id, resource.name are widely used across all the CNVM pages, so we need to coordinate the changes on the Kibana side and also make sure to consider backward compatibility.

image

image

image

@nick-alayil, in that case, are we going to review the naming in Kibana according to the new fields?

cc @kfirpeled for awareness and prioritization

@amirbenun
Copy link
Copy Markdown
Contributor

Thanks @opauloh, so as a first step we will just add the new fields and use them in kibana instead.
Since we are in beta, I think we can remove completely the deprecated fields.
This will affect customer in the following way.

  • Users who upgraded their stack version but not the agent will suffer the most as the UI will show blanks resource.id and resource.name.
  • Users who upgraded their stack and agents will get that behavior for old findings only.

@nick-alayil what do you think? Do real users exploring CNVM? If not I would make this change before CNVM goes GA.

@oren-zohar
Copy link
Copy Markdown
Collaborator

  • Users who upgraded their stack version but not the agent will suffer the most as the UI will show blanks resource.id and resource.name.

In this case, we can create a migration flow like we already had in other cases, that will map these fields to the right place. I'm not sure if it was an ingest pipeline or something else in Kibana itself but I do remember we had something like that already @orouz @kfirpeled

@opauloh
Copy link
Copy Markdown
Contributor

opauloh commented Jul 10, 2023

Hey @amirbenun / @oren-zohar

I opened a Kibana ticket for us to track the required changes on Kibana to deprecate the resource fields to the new fields. I believe it's not a blocker for this PR since new fields were added without removing the resource fields, so this PR can be merged.

@jeniawhite jeniawhite merged commit faa7fbf into elastic:main Jul 16, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[CNVM] Index vulnerable resource fields

5 participants