Integrate Asset Inventory with backend

[albertoblaz]

Summary

Closes https://github.com/elastic/security-team/issues/11592.

Fetch and render backend data upon opening the Asset Inventory page.

Depends on

• https://github.com/elastic/security-team/issues/11270
• Implement Header and Toolbar with Search Bar #201709
• Implement Filters Section #201710
• https://github.com/elastic/security-team/issues/11687

Screenshots

No applied filters

Filtering through search bar with KQL

Filtering through filter dropdowns

Filtering through both search bar and filter dropdowns - no results found in this case

Default empty state (no rows fetched)

Definition of done

• Asset Inventory page fetches data prepared by the data-view that comes pre-installed with the "Cloud Asset Inventory" integration
  • Search bar
  • Filters
  • Data Grid
  • Empty state when number of fetched rows is zero

How to test

1. Prepare cloud user
   • Go to users page on Elastic Cloud
   • Create a new user with a custom username and password
   • Copy the same roles from other users
2. Start local env running these commands
   • Run ES with node scripts/es snapshot --license trial -E path.data=../default -E reindex.remote.whitelist=<REMOTE_URL>:<REMOTE_PORT> -E xpack.security.authc.api_key.enabled=true
   • Run Kibana with yarn start --no-base-path
3. Go to Integrations page, switch on the "Display beta integrations" control, then add the Cloud Asset Inventory integration on your local environment. Postpone Elastic Agent addition.
4. Go to Dev Tools page, click on the "config" tab and add the following environment variables:
   Use the dev tools config tab to save your as follows:
   • ${ES_REMOTE_HOST}: (the URL to the cloud instance)
   • ${ES_REMOTE_USER}: (the username you set for your user on step 0)
   • ${ES_REMOTE_PASS}: (the pass you set for your user on step 0)
5. Run the following script:

Script

POST _reindex?wait_for_completion=false
{
  "conflicts": "proceed", 
  "source": {
    "remote": {
      "host": "${ES_REMOTE_HOST}",
      "username": "${ES_REMOTE_USER}",
      "password": "${ES_REMOTE_PASS}"
    },
    "index": "logs-cloud_asset_inventory*",
    "query": {
      "bool": {
        "must": [
          {
            "range": {
              "@timestamp": {
                "gte": "now-1d"
              }
            }
          }
        ]
      }
    }
  },
  "dest": {
    "op_type": "create",
    "index": "logs-cloud_asset_inventory.asset_inventory-default"
  },
  "script": {
    "source": """
      ctx._source['entity.category'] = ctx._source.asset.category;
      ctx._source['entity.name'] = ctx._source.asset.name;
      ctx._source['entity.type'] = ctx._source.asset.type;
      ctx._source['entity.sub_type'] = ctx._source.asset.sub_type;
      ctx._source['entity.sub_category'] = ctx._source.asset.sub_category;
    """
  }
}

Finally, open Discover page and set the DataView filter on the top-right corner to logs-cloud_asset_inventory.asset_inventory-*, as in the screenshot below. If the grid is populated, you've got data and the whole setup worked!

Discover page


Checklist

• Unit or functional tests were updated or added to match the most common scenarios
• This was checked for breaking HTTP API changes, and any breaking changes have been approved by the breaking-change committee. The release_note:breaking label should be applied in these situations.
• The PR description includes the appropriate Release Notes section, and the correct release_note:* label is applied per the guidelines

Identify risks

No risks at all.

[elasticmachine]

Pinging @elastic/kibana-cloud-security-posture (Team:Cloud Security)

[albertoblaz]

Added some notes for reviewers down below

[albertoblaz]

Note: I needed to hard-code this value to populate the filters. However, in the future it should be ASSET_INVENTORY_INDEX_PATTERN, whose value is "logs-cloud_asset_inventory.asset_inventory-*"

[albertoblaz]

This tiny change happens because .title is a deprecated field

[albertoblaz]

Previously I propagated showFilterBar={true} onto the <SearchBar> component. Using these props here broke the display value of the selected filter tags, rendering " - " instead.

Then, I switched showFilterBar={true} in <SearchBar> and this was not needed, so I removed it. Please let me know if this might cause an issue

[albertoblaz]

@opauloh It seems the best way to solve the time range issue is simply not passing in any timeRange value. The filter dropdowns get populated correctly.

[opauloh]

Great, thanks for letting me know!

[albertoblaz]

@opauloh Addressed your comments. Thanks for your review! 🙏

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: b556ab4

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
securitySolution	7092	7095	+3

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
securitySolution	8.9MB	8.9MB	+4.4KB

Unknown metric groups

References to deprecated APIs

id	before	after	diff
securitySolution	353	352	-1

History

• 💚 Build #277212 succeeded 403a362

cc @albertoblaz

[opauloh]

LGTM