Produce JSON-line output when JSON logging is configured

[ammar92]

About this feature

Detailed description

This feature introduces pure JSON-lines log output for OpenKAT log records when the LOGGING_FORMAT option is set to json. This change allows all log entries to be formatted as single-line JSON objects, which should lead to easier parsing and integration with log aggregation tools. Currently the JSON log record is also printed, but with a fixed preamble that in most cases isn't relevant for machine processing.

Feature benefit/User story

As a developer/ administrator, I want to configure OpenKAT to enable pure JSON line logging so that log aggregation and analysis become more efficient and easier.

Specifications

The implementation should include:

• When json is set for the [*_]LOGGING_FORMAT variables, OpenKAT should log JSON lines without preamble that contains the complete log record (event message, log level, timestamp, context variables)

Additional information

Certain log lines that come from external dependencies (or plugins) may introduce some noise in the logging because of different log record formats. Failures due to parsing errors or parsing specific external packages logging should be handled by the log aggregator. This feature will focus only on the log records produced by our own applications for now.

Implementation

Implement the correct log format for JSON lines for each service.

Possible solution

There are a few ways to deal with this:

• Configure the default formatter for console loggin to use this format "%(message)s"
• See this for more ideas using structlog
  • Most of the suggestions involve overriding the root logger with structlog's JSON formatter
• Use python-json-logger
  • This also involves overriding the root logger's format
  • Can also be used in conjunction with structlog

Alternatives considered

An alternative would be to not implement this feature and instead rely on a log aggregator to parse the logs. Since the structure of the preamble is fixed, extracting the JSON log record should be doable. However, implementing this on the application side would improve overall performance and reduce reliance on potentially complex configurations in log aggregators.