[LIBS PLUGINS] Accessors to libs state tables fields

[incertum]

Motivation

Opening this issue first with the intention of seeking clarifications. Based on the answers received, there may or may not be a need to request an expansion or improvement of the plugin API.

While onboarding falcosecurity/plugins#419 to the plugin framework for syscall event analysis augmentation, I encountered a few hiccups:

• Whenever the thread entry field was not defined using define_static_field in libs, it seemed inaccessible. For example, the following didn't work: m_exe_from_memfd = m_thread_table.get_field(t.fields(), "exe_from_memfd", st::SS_PLUGIN_ST_BOOL);
• Additionally, for non-standard types, I'm unsure how to access them. For instance, the k8smeta plugin fetched the cgroups natively from the event arguments. Is this intentional? It would be beneficial to access these fields after the library has completed its work, regardless. For example, I would prefer to easily access the following thread table entries:

std::vector<std::string> m_args; ///< Command line arguments (e.g. "-d1")
std::vector<std::string> m_env; ///< Environment variables
std::unique_ptr<cgroups_t> m_cgroups; ///< subsystem-cgroup pairs
...

• Lastly, I wasn't yet sure how to access the fd table entries, and it seemed the documentation didn't provide examples. However, I might have overlooked them. I would appreciate some initial examples, if possible.
• I haven't checked yet how to access container cache entries, but I anticipate this may also be needed down the road, however less urgently for a v1

[jasondellaluce]

exe_from_memfd

I think it's safe to add this one to the ones exported by libsinsp.

std::vectorstd::string m_args; std::vectorstd::string m_env; std::unique_ptr<cgroups_t> m_cgroups

Non-trivial types (e.g. vectors, maps) are something we currently don't support in the state API. This is something I have on my backlog since a while and is something I'm trying to figure out the best solution for. My guess is that the k8smeta plugin used the workaround of getting that data from the event payloads themselves, which is also an option for the short term.

Lastly, I wasn't yet sure how to access the fd table entries, and it seemed the documentation didn't provide examples. However, I might have overlooked them. I would appreciate some initial examples, if possible.
I haven't checked yet how to access container cache entries, but I anticipate this may also be needed down the road, however less urgently for a v1

Both the fd tables and the containers table are not exported yet. There are technical implications behind this that are similar to the ones of non-trivial types. Again, this is something I'm working on. How much of a road blocker is this currently for developing your plugin? Is this something you can attain from inspecting event payloads for the short term?

[incertum]

exe_from_memfd

I think it's safe to add this one to the ones exported by libsinsp.

std::vectorstd::string m_args; std::vectorstd::string m_env; std::unique_ptr<cgroups_t> m_cgroups

Non-trivial types (e.g. vectors, maps) are something we currently don't support in the state API. This is something I have on my backlog since a while and is something I'm trying to figure out the best solution for. My guess is that the k8smeta plugin used the workaround of getting that data from the event payloads themselves, which is also an option for the short term.

It would also be fair to find an (interim) way to support a set of currently known vector types etc, most notably to expose the cmd args as this is part of the backbone of the anomalydetection plugin.

Lastly, I wasn't yet sure how to access the fd table entries, and it seemed the documentation didn't provide examples. However, I might have overlooked them. I would appreciate some initial examples, if possible.
I haven't checked yet how to access container cache entries, but I anticipate this may also be needed down the road, however less urgently for a v1

Both the fd tables and the containers table are not exported yet. There are technical implications behind this that are similar to the ones of non-trivial types. Again, this is something I'm working on. How much of a road blocker is this currently for developing your plugin? Is this something you can attain from inspecting event payloads for the short term?

Thanks @jasondellaluce, basically right now I cannot implement the planned v1 of the anomalydetection plugin, most notably because the fd table is not accessible and there wouldn't even be a workaround. Defining behavior profiles based on fd.name is the top number 1 use case of the planned anomalydetection plugin as I also highlighted during my KubeCon NA 23 talk 🙃.

How can I help to make this happen?

[incertum]

@jasondellaluce: @leogr shared that you are already looking into it 🎉 ❤️ .

In case a prioritization is useful:

Top 1: Access to fd.name and fd.nameraw and proc.cmdline (using the filtercheck fields for simplicity, you know which members I am referring to).

Actually Top 1 is all that is needed for a v1, but full access to the entire state would of course be ideal.

---

Timelines:

If we get this in for Falco 0.38.0 I can release an experimental anomalydetection plugin in June/July 2024, the originally planned timeline.

[jasondellaluce]

@incertum yes, this is on my plate, so the Falco 0.38 deadline seems very reasonable.

Access to fd.name and fd.nameraw and proc.cmdline (using the filtercheck fields for simplicity, you know which members I am referring to).

Good call about the cmdline, we currently don't share that. As for fd.*, the goal would be to open up all fd-related information!

[jasondellaluce]

Just want to update that I'm working on this together with @mrgian and I'm gonna open up a tracking issue for the workstream in few days

[jasondellaluce]

Opened the first PR implementing the feature. The contribution plan is expected to consists of 4 pull requests:

• Ground work on libsinsp state API
  • refactor(userspace/libsinsp): minor changes in state API, expose new threadinfo fields #1838
• Definition of subtables in plugin API and sinsp state API
  • update(userspace): define subtable field type in plugin API and sinsp state API #1849
• Exposing file descriptor tables from threads table and extensive unit testing:
  • update(userspace/libsinsp): expose file descriptors as state subtable fields compatible with plugins #1866
• Exposing complex data-type fields (e.g. vector of strings env and args) from threads table: TBD
  • update(userspace/libsinsp): support array-like types as state fields in thread infos, compatible with plugin API #1868

/milestone 0.17.0

Tentative ☝️

[incertum]

Amazing thanks a bunch @jasondellaluce!

[jasondellaluce]

@incertum the changes tracked in #1712 (comment) are now all merged, and I think this should suit most of the needs for this tickets. The only thing left out is the containers table for now.

[incertum]

Amazing, I'll get back to the plugin soon (after Falco 0.38.0 is out) and will check it out. Thanks so much ❤️ !

Should we close this one and open a new one for what's left for future use cases?