Simple stack trace concat

[mayurcherukuri]

I'm trying to write a simple test of concat for merging multiline stack trace into a single record and write it to file but the records are still separate.

<filter kubernetes.var.log.containers.**.log>
  @type concat
  key log
  multiline_start_regexp /^\s*at/
  continuous_line_regexp /^\s*at/  
</filter>

<match kubernetes.var.log.containers.**.log>
  @type file
  path /var/log/fluent/myapp
  <buffer>
    timekey 1m
    timekey_use_utc true
    timekey_wait 1s
  </buffer>
</match>

1970-01-01T00:33:40+00:00       kubernetes.var.log.containers.frontdoor-deployment-779c9cc45d-d5mkx_sparta_frontdoor-50e83cc7811bc17c6f95072a42d308d22ada34c55298c5df65826003754e0b1a.log       {"log":"         at Azure.Identity.ManagedIdentityCredential.GetTokenImplAsync(TokenRequestContext requestContext, CancellationToken cancellationToken)\n","stream":"stdout","ContainerName":"frontdoor","NamespaceName":"sparta","PodName":"frontdoor-deployment-779c9cc45d-d5mkx","Node":"aks-agentpool-43717940-vmss000005","MasterUrl":"https://10.101.0.1:443/api"}
1970-01-01T00:33:40+00:00       kubernetes.var.log.containers.frontdoor-deployment-779c9cc45d-d5mkx_sparta_frontdoor-50e83cc7811bc17c6f95072a42d308d22ada34c55298c5df65826003754e0b1a.log       {"log":"         at Azure.Identity.DefaultAzureCredential.GetTokenAsync(Boolean isAsync, TokenRequestContext requestContext, CancellationToken cancellationToken)\n","stream":"stdout","ContainerName":"frontdoor","NamespaceName":"sparta","PodName":"frontdoor-deployment-779c9cc45d-d5mkx","Node":"aks-agentpool-43717940-vmss000005","MasterUrl":"https://10.101.0.1:443/api"}

Expected Output:

1970-01-01T00:33:40+00:00       kubernetes.var.log.containers.frontdoor-deployment-779c9cc45d-d5mkx_sparta_frontdoor-50e83cc7811bc17c6f95072a42d308d22ada34c55298c5df65826003754e0b1a.log       {"log":"         at Azure.Identity.ManagedIdentityCredential.GetTokenImplAsync(TokenRequestContext requestContext, CancellationToken cancellationToken)\n         at Azure.Identity.DefaultAzureCredential.GetTokenAsync(Boolean isAsync, TokenRequestContext requestContext, CancellationToken cancellationToken)\n","stream":"stdout","ContainerName":"frontdoor","NamespaceName":"sparta","PodName":"frontdoor-deployment-779c9cc45d-d5mkx","Node":"aks-agentpool-43717940-vmss000005","MasterUrl":"https://10.101.0.1:443/api"}

Output:

Same as input

Environment

• td-agent 1.3.3
• fluent-plugin-concat (2.4.0)

Docker Container details :

NAME="Ubuntu"
VERSION="16.04.5 LTS (Xenial Xerus)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 16.04.5 LTS"
VERSION_ID="16.04"
HOME_URL="http://www.ubuntu.com/"
SUPPORT_URL="http://help.ubuntu.com/"
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"
VERSION_CODENAME=xenial
UBUNTU_CODENAME=xenial

[okkez]

Please search k8s in README, and try the sample configuration for k8s.