Logs to support nested attributes without flatten

[sdzhong]

Problem Statement

Feature request to support 'object' attribute in the same way as the 'sdk' attribute without needing to flatten the object that you add as an attribute (e.g. with https://www.npmjs.com/package/flat).

Solution Brainstorm

No response

Product Area

Unknown

[linear]

ENG-5324 Logs to support nested attributes without flatten

[getsantry]

Assigning to @getsentry/support for routing ⏲️

[sdzhong]

Internal case https://www.notion.so/sentry/2548b10e4b5d804c8b04fa06c0184c72

[Dhrumil-Sentry]

This is very valid feedback and we'd like to address this - We don't have concrete timelines yet but we will keep this issue updated

[Dhrumil-Sentry]

@sdzhong How nested can these attributes be 1 level, 2? The depth of unnesting we need to support would influence how we build this

[garethlowriehive]

We would love this functionality as we have a lot of attributes on our logs that contain nested json which we are used to searching against.

We're trying to work around this for now with our fluentbit solution by using this function

      processors:
        logs:
          # Flatten as Sentry doesn't yet support nested objects, so we'll flatten
          # the properties using dot notation instead using a lua script
          - name: lua
            match: "*"
            script: /etc/fluentbit/scripts/flatten.lua
            call: flatten_record

function flatten(tag, timestamp, record)
    local function flatten_table(tbl, prefix, result)
        prefix = prefix or ""
        result = result or {}

        for k, v in pairs(tbl) do
            local new_key = prefix == "" and k or prefix .. "." .. k

            if type(v) == "table" then
                flatten_table(v, new_key, result)
            else
                result[new_key] = v
            end
        end

        return result
    end

    local flattened = flatten_table(record)
    return 1, timestamp, flattened
end