How to use rich logger without enforced wrapping?

[ssbarnea]

While replacing the standard python logger with a rich one I discovered that longer lines get wrapped and that normal lines do get extra padding at the end, both being undesired actions, especially as same logging result can be sent to files or other apps for further processing.

I tried adding width=0 to Console instance used for logger but that basically broke it and produced no output. None seems to be reserved to auto-detection. I also tried putting a very big value but that produced very long spam lines, even when called with console=False

LOGGING_CONSOLE = Console(
    file=sys.stderr,
    force_terminal=False,
    width=300)

How can I make the rich logging not add extra padding and avoid any wrapping?

[willmcgugan]

If you are using RichHandler, then you can't change that behaviour. It's designed to format it for the terminal and wrapping is necessary for neat columns.

If you want to also log to a file, you can configure a FileHandler (or any other logging handler(s)) in conjunction with the Rich handler.

[ssbarnea]

As "final" console width cannot be always be known at the time logging happens (CI is best example, where you want to produce nice output, but you never known the width, as this will endup being rendered in a browser).

This means that we either need a new handler that does not wrap or another option to disable wrapping in current one. Considering the "0" value is not currently used, it could make sense to use it to disable wrapping. Still, as did not look on how is implemented yet, I will likely wait for you to say if is doable or not.

I do love the richhandler, it would a pity not to be able to make use of it.

[willmcgugan]

It's not possible to disable hard wrapping in the Rich logging handler because it would break the columns. If this is a requirement for you I would suggest implementing your own handler based on rich/logging.py.

[ssbarnea]

I was already able to created a custom FluidLogRender class that does not make use of Table(Columns) and had partial success with it.

Still, I am still facing some issues related to some wrapping coming from other places. For example I found that Text() has an implicit no_wrap=None (same as False) which is called from various places. I tried to hack Rich to temporary make no_wrap=True but it was not enough.

Maybe you could give me a hint on how to render the text without any wrapping to occur.

[Toreno96]

Thank you very much @ssbarnea!

While the rich as a whole is amazing library, I wasn't really fond of the RichHandler formatting (I simply wanted some coloring, that's all), and I gave up after trying for definitely too long to hack the original implementation.

You did amazing work at providing the solution 💪

[ssbarnea]

Thanks @Toreno96 for the kind word. To be honest I still hope I will be able to persuade @willmcgugan to get this fluidlogger into rich itself. The reason I want this is because packaging another library just for these few lines of code is not a joy.

It seems that these 85 lines did not need any maintenance in more than an year and I would be more than happy to be the one assigned to address issues if reported, just to avoid having to ask other projects to add an extra library. It is hard for me not to remember nightmares from npm registry where people created packages for even a 3 line function.

[Toreno96]

@willmcgugan Could you share what are the arguments against integrating this into rich? 🙏

[willmcgugan]

I'm not apposed to a fluid logger in principle. But frankly its not for me to say why it shouldn't go in, you will need to make the case why it should.

As soon as I add that, I will get issue after issue for requests for flags and options, because everyones needs differ.

@ssbarnea 's implementation is unnecessarily convoluted. You don't need to derive from RichHandler or create a log render object, when you can create a logging Handler of your own and print a Text instance . In a few lines of code you can have a custom handler that does exactly what you need. Start from the Python docs -- don't hack RichHandler, and it will be much easier.

Ultimately, its an uphill battle to expect me to maintain a FluidLogHandler, when rolling your own is trivial.

[davidparsson]

This is how i managed to print log lines that are colored by another formatter:

import logging

from rich.console import Console
from rich.text import Text


class FluidRichHandler(logging.Handler):
    def __init__(self, console: Console) -> None:
        super().__init__()
        self._console = console

    def emit(self, record: logging.LogRecord) -> None:
        log_line = self.format(record)
        self._console.print(Text.from_ansi(log_line))

[anvaari]

Maybe simple workaround is this use custome handler like this. You can modify format to format other things to

import logging
from rich.console import Console


rprint = Console(soft_wrap=True).print
class MyRichLogHandler(logging.Handler):
    LEVEL_MAPPING = {
        logging.DEBUG: "[blue]DEBUG[/blue]",
        logging.INFO: "[green]INFO[/green]",
        logging.WARNING: "[yellow]WARNING[/yellow]",
        logging.ERROR: "[red]ERROR[/red]",
        logging.CRITICAL: "[bold red]CRITICAL[/bold red]",
    }
    def emit(self, record):
        msg = self.format(record)
        rprint(msg)

    def format(self, record):
        levelname = self.LEVEL_MAPPING.get(record.levelno, str(record.levelno))
        
        file_name_line = (
            f"[link file://{record.filename}#{record.lineno}]"
            f"{record.filename}:{record.lineno}"
            f"[/link file://{record.filename}#{record.lineno}]")
        
        record.levelname = levelname
        record.filename = file_name_line
        return super().format(record)

You can modify format method to format other things too.

in this example if you set base_log_format = %(asctime)s - %(levelname)s in %(filename)s: %(message)s", the `filename wil be link to that number of error in the file which long belong to.

[demizer]

For those that want pretty output in the terminal AND Gitlab CI.

Can probably be cut down further but it works.

Set CI=true to limit table column width.

Edit: for python 3.12.

Warning: I haven't fully tested this with all the renderables, only with what my project is currently using (tables).

import logging
import os
from collections.abc import Callable
from collections.abc import Iterable
from datetime import datetime
from logging import Handler
from logging import LogRecord
from types import ModuleType
from typing import Any
from typing import ClassVar

from rich.console import Console as RichConsole
from rich.console import ConsoleRenderable
from rich.console import JustifyMethod
from rich.console import NewLine
from rich.highlighter import Highlighter
from rich.highlighter import ReprHighlighter
from rich.panel import Panel
from rich.scope import render_scope
from rich.segment import Segment
from rich.style import Style
from rich.styled import Styled
from rich.text import Text
from rich.traceback import Traceback

FormatTimeCallable = Callable[[datetime], Text]

"""An example for monkey patching Rich to allow fluid output in terminal and CI and also keep pretty highlighting.

File links are removed since those break in Gitlab.

Gitlab issue that will never get fixed: https://gitlab.com/gitlab-org/gitlab-runner/-/issues/29365
"""

class Console(RichConsole):
    """A modified version of Rich Console to remove file links in output for Gitlab.

    From https://github.com/Textualize/rich/blob/master/rich/console.py#L593

    """

    def log(
        self,
        *objects: Any,
        sep: str = " ",
        end: str = "\n",
        style: str | Style | None = None,
        justify: JustifyMethod | None = None,
        emoji: bool | None = None,
        markup: bool | None = None,
        highlight: bool | None = None,
        log_locals: bool = False,
        _stack_offset: int = 1,
    ) -> None:
        """Log rich content to the terminal.

        Args:
            objects: Objects to log to the terminal.
            sep: String to write between print data. Defaults to " ".
            end: String to write at end of print data. Defaults to "\\\\n".
            style: A style to apply to output. Defaults to None.
            justify: One of "left", "right", "center", or "full". Defaults to ``None``.
            emoji: Enable emoji code, or ``None`` to use console default. Defaults to None.
            markup: Enable markup, or ``None`` to use console default. Defaults to None.
            highlight: Enable automatic highlighting, or ``None`` to use console default. Defaults to None.
            log_locals: Boolean to enable logging of locals where ``log()``
                was called. Defaults to False.
            _stack_offset: Offset of caller from end of call stack. Defaults to 1.
        """
        if not objects:
            objects = (NewLine(),)

        render_hooks = self._render_hooks[:]

        with self:
            renderables: list[ConsoleRenderable] = self._collect_renderables(
                objects,
                sep,
                end,
                justify=justify,
                emoji=emoji,
                markup=markup,
                highlight=highlight,
            )
            if style is not None:
                renderables = [Styled(renderable, style) for renderable in renderables]

            filename, line_no, locals = self._caller_frame_info(_stack_offset)
            # link_path = None if filename.startswith("<") else os.path.abspath(filename)
            path = filename.rpartition(os.sep)[-1]
            if log_locals:
                locals_map = {key: value for key, value in locals.items() if not key.startswith("__")}
                renderables.append(render_scope(locals_map, title="[i]locals"))

            renderables = [
                self._log_render(
                    self,
                    renderables,
                    log_time=self.get_datetime(),
                    path=path,
                    line_no=line_no,
                    # all because of this
                    # link_path=link_path,
                )
            ]
            for hook in render_hooks:
                renderables = hook.process_renderables(renderables)
            new_segments: list[Segment] = []
            extend = new_segments.extend
            render = self.render
            render_options = self.options
            for renderable in renderables:
                extend(render(renderable, render_options))
            buffer_extend = self._buffer.extend
            for line in Segment.split_and_crop_lines(new_segments, self.width, pad=False):
                buffer_extend(line)


class FluidRichHandler(Handler):
    LEVEL_MAPPING = {
        logging.DEBUG: "[blue]DBUG[/blue]",
        logging.INFO: "[green]INFO[/green]",
        logging.WARNING: "[yellow]WARN[/yellow]",
        logging.ERROR: "[red]ERRR[/red]",
        logging.CRITICAL: "[bold red]CRIT[/bold red]",
    }

    HIGHLIGHTER_CLASS: ClassVar[type[Highlighter]] = ReprHighlighter

    KEYWORDS: ClassVar[list[str] | None] = [
        "GET",
        "POST",
        "HEAD",
        "PUT",
        "DELETE",
        "OPTIONS",
        "TRACE",
        "PATCH",
    ]

    def __init__(
        self,
        console: Console,
        level: int | str = logging.NOTSET,
        *,
        enable_link_path: bool = True,
        highlighter: Highlighter | None = None,
        markup: bool = False,
        rich_tracebacks: bool = False,
        tracebacks_width: int | None = None,
        tracebacks_code_width: int = 88,
        tracebacks_extra_lines: int = 3,
        tracebacks_theme: str | None = None,
        tracebacks_word_wrap: bool = True,
        tracebacks_show_locals: bool = False,
        tracebacks_suppress: Iterable[str | ModuleType] = (),
        tracebacks_max_frames: int = 100,
        locals_max_length: int = 10,
        locals_max_string: int = 80,
        columns: int = -1,
        keywords: list[str] | None = None,
    ) -> None:
        super().__init__(level=level)
        self.console = console
        self.highlighter = highlighter or self.HIGHLIGHTER_CLASS()
        self.markup = markup
        self.enable_link_path = enable_link_path
        self.markup = markup
        self.rich_tracebacks = rich_tracebacks
        self.tracebacks_width = tracebacks_width
        self.tracebacks_extra_lines = tracebacks_extra_lines
        self.tracebacks_theme = tracebacks_theme
        self.tracebacks_word_wrap = tracebacks_word_wrap
        self.tracebacks_show_locals = tracebacks_show_locals
        self.tracebacks_suppress = tracebacks_suppress
        self.tracebacks_max_frames = tracebacks_max_frames
        self.tracebacks_code_width = tracebacks_code_width
        self.locals_max_length = locals_max_length
        self.locals_max_string = locals_max_string
        self.keywords = keywords
        if columns > 0:
            self.columns(columns)

    def columns(self, size: int) -> None:
        self.columns_size = size
        # rich will default to 80 columns if there isn't a tty
        # https://github.com/Textualize/rich/blob/master/docs/source/console.rst#terminal-detection
        os.environ["COLUMNS"] = str(size)

    def emit(self, record):
        message = self.format(record)
        traceback = None
        if self.rich_tracebacks and record.exc_info and record.exc_info != (None, None, None):
            exc_type, exc_value, exc_traceback = record.exc_info
            assert exc_type is not None
            assert exc_value is not None
            traceback = Traceback.from_exception(
                exc_type,
                exc_value,
                exc_traceback,
                width=self.tracebacks_width,
                code_width=self.tracebacks_code_width,
                extra_lines=self.tracebacks_extra_lines,
                theme=self.tracebacks_theme,
                word_wrap=self.tracebacks_word_wrap,
                show_locals=self.tracebacks_show_locals,
                locals_max_length=self.locals_max_length,
                locals_max_string=self.locals_max_string,
                suppress=self.tracebacks_suppress,
                max_frames=self.tracebacks_max_frames,
            )
            message = record.getMessage()

        message_renderable = self.render_message(
            record,
            [message] if not traceback else [message, traceback],
        )
        try:
            self.console.print(message_renderable, soft_wrap=True)
        except Exception:
            self.handleError(record)

    def render_message(self, record: LogRecord, message: list[str | Traceback]) -> "ConsoleRenderable":
        """Render message text in to Text.

        Args:
            record (LogRecord): logging Record.
            message (str): String containing log message.

        Returns:
            ConsoleRenderable: Renderable to display log message.
        """
        message_text = Text()
        for elem in message:
            if isinstance(elem, str):
                use_markup = getattr(record, "markup", self.markup)
                message_text = Text.from_markup(elem) if use_markup else Text(record.msg)
            else:
                message_text += elem

        highlighter = getattr(record, "highlighter", self.highlighter)
        if highlighter:
            message_text = highlighter(message_text)

        if self.keywords is None:
            self.keywords = self.KEYWORDS

        if self.keywords:
            message_text.highlight_words(self.keywords, "logging.keyword")

        return message_text

    def format(self, record):
        record.levelname = self.LEVEL_MAPPING[getattr(logging, str(record.levelname))]
        return super().format(record)


def banner(text: str) -> None:
    """Print a banner"""
    console.log(Panel(text), _stack_offset=2)


if __name__ == "__main__":
    console = Console(
        soft_wrap=True,
        color_system="256",
        force_terminal=True,
    )
    format = "%(asctime)s %(levelname)s \\[%(filename)s:%(lineno)s] :: %(message)s"
    sink = FluidRichHandler(
        console=console,
        level=logging.DEBUG,
        markup=True,
        rich_tracebacks=True,
        enable_link_path=False,
    )
    if os.getenv("CI") == "true":
        sink.columns(150)
    logging.basicConfig(level=logging.INFO, format=format, datefmt="[%X]", handlers=[sink])

    banner("link to https://alexandrite.app/")
    logging.info("A really long line with a long link that should be clickable (alexandrite.app): http://www.reallylong.link/rll/4QMpSmNQFPAzm6xssWDkgcED_nBklJunV2HwJKvBSjLOOIq3Pn/RvmOrS6lZmzNonqJ0X4Y_P4iBuLoMX3et/F9lebVVSnufNSssnWAiZUL1Q7aj9mnII9aQWQ2mhuPidyTYaOunxFGgKO8/NEzPfVjNDbNsZ61xhx0tQGfEerdwFtRagcaM0vI0XDLD2VwrRw/OOGxnPFH9HL_c5oe3w15WfFfALrqAP58/hI6Pm82wTzucsF08ak5Eh5A/6kt3q3FXPetygBQqxkVHqBt52dK0fYz4AShrsn4Rfj9Qc6KKN/HzJsUOV7nU6CSlo5BALkgUt5Uh0YhC_G5OQzGyNWC/qSZbTfL9mulULs8FM0s__JbLb949SP9nevrdMrJeznIPhPw9IHvbuJJtPNuaAcIVklk9r2mrf/8WoE3H8l_UKh175r_hvQaP4mOG2u91trDDlWiC5HeO2gemnpqhLX3GDYQuWUO8imD9hY3GtiqUMQ65STCluNolWkpJEO/6vXisSkC7RpoBqTSGBc3a9AeuK/F/NvcxZdZPLdkpcMnihcZjgFE31oed5Le25YxBd5EFEm_aXTXsjIJuUMuTZgcpiumYDnDosI9xxAVYeKt5z0AGeVr2tl3HH27Oi451qAgbMOgszdMT8Bpe0smUGXiKsPPKtwOh8RNqjX7XeiCDwlS_YN0MikEkqoB42hK87rOfQHm7ubPbVVIk8uv9d6sz66e/7lSsNg7ySfPMyhFAqSDvLdT5V2DWD7MAPyBJpqmcPyZPc3J9882fYlREgelmuKQRTGMzc36tV4WtOCnlH/Gq86Dc9Gq2ogZp2gzR9gNqiK1HaSYMiVkNdIpPYRGoPeUMAhJzaCfGFpGqvD6rl3bkOFv1q4DaAIynDJkUpgcrevR2Ds77rl9GeHbyHDRn4cqXI5PpVZKh5lVSiZ_PViXKQLW_e_w2UwCVxu5bSHiM28c/dRoZfU3VHBgYCp2PPgIYuMfS2reGhdn8ppHgSpXppn15iGw/_zVaLWJ2F0XbzoOBA4pRI5wmRxvIbNHEj3prswk4Ya_uXe3urInAaBK5oy5XmuO_hqr/3uyqb8v9ZmvnT_h_fxL3er_IVfsEb0Nx1ramqKZmkACbbnQ3nqibFmzs9EflzPvEudNPyim8QbIGDJ9ZflPSCFjOuqp_Na7SJKbKRMVOP1xsV_K_xMC6me6g1s0A79TXZPfNKJ1jH5VbmlwgnIoRGoFtLp10ecLuCfeYR2/yhsqlXz3kM7xNv09LyPOA5EmV9_64wlDjRCgxhZOZNvS4P5QP4vbnSU8PC6JKS/TVgywW/UZyyfPoUsi9ePeDXzRrvfKFpZnrPFySLz6VNiYjnEV09Zv9CXMIJaNbWGY98oQcDXwx3kHmDiYRaTF4SPgfyeJDl5hjt8LOQWZYMz_x07KW/5DZo1Dexf/3k4wNRxTytEsWlLVDJ20eN/lRDCckix_6CToi8oQA7jZP9j5elkjkPXDwm231_SSjgRZe52vWI1vErr8KgpZ5GKJLSVyUkPOjVK07jj/7A0mDGUdI_x7lyeUjIwTlZvtEf0YWpsZJpOoZ7Kb2kKApAVRV2YdZspMRmjP6suWnThSgQaLO9m5dFH/7i2oaLRBAK8158k0JhPMMUweI8UNq/8UwzHqBYw6IZ1eRKgQMyFsTaLi44SYeZiGIMDBPksKNJAuv_qNqklD9wNIYpouYQxz3_cHPzEdzyUHc7ub82oBw9joACFrnLR1IJZaG3QsULwsNqAcU7Vpz/ZPyaYgwim9H3lAGnjLI8uSzFoaNUt49g_fSaHX/ozqAPjyeX8g1LFXPJHwOXPQSwRHBLh4Av_qaeL7Xd/ateWXnp_WtrXzPW0Bf8UhMi9y9Y75cTM2AzT2Qfa91NXka/vg1Uf2a2BWCGcADTgHUl8yDYrUvcpgXFWHpr2ZEm49SwM24H91GYYqVbTUMcvoPvMq")
    banner("end https://alexandrite.app/")

[ssbarnea]

The reality is that we should not have to rely on such hacks to be able to make it work. There are two main usages for rich-colors apps: TUI apps (rigid terminal size, with optional resize support) and second being improving output of console applications that are not really TUI ones (logging fluid). At this moment rich seems highly focused only on the first use case.

I only have one TUI app I contribute to and funny that one happens not to use rich. All the other apps use rich only to improve their console output and none of them wants to perform wrapping of long lines, as we assume that the terminal will do that (thus they would work well on CI as that would use a web browser with variable width).

I hope that this might change in the future and that second use case will be supported officially. My take is that a LogHandler should be fluid anyway, not assuming any COLUMNS.