Discussion: a builtin pager for 'bat'

[sharkdp]

Let's use this ticket to discuss some ideas and possible upsides/downsides of writing our own custom pager library that we could use instead of calling an external $PAGER process (or, essentially, less).

A few advantages if we would use a builtin pager in bat:

• Have one consistent pager experience across all operating systems. No need to call external programs that might not be available. No special handling / bug fixing for different versions of less. Consistent "quit if one screen" handling, and much more (Default pager breaks colours in Windows Terminal #860, Garbage unicode chars on wide screen display #1000, Can't open Dockerfile or .dockerfile  #717, Bat does not work as file viewer for Midnight Commander for small files #710, pager="less -RF" breaks on macOS for single-page, but default pager breaks pgup/pgdn #681, File content not printed #520, Bat quickly exits #415, Bat with BAT_PAGER set doesn't work on small files #312, Always displays escape characters #271, Horizontal line wraps if less --status-column is used as pager #519, Width is wrong when using less as pager with -J option #376, Since recent update, short files that do not require paging are displayed and then lost #887, Bat opens a file and close it itself in a second on Windows #943, Can't view some files #838, …)
• Handling of terminal resize events. This would be my personal main motivation. (iTerm2 issue with fullscreen #294, Support terminal resize. #322, bat doesn't redraw content when size of terminal changes when using i3 #921, SIGWINCH support #1009)
• Open editor with a key press (could you add edit mode? #358, editor mode #464, v command does not work with a file as arg #528, Cannot edit standard input (Press RETURN) #697, Have the ability to open a vim session from bat (like less) #1051, Open file with v, like with less #1607)
• Possibility to always draw the header at the top of the screen, even when scrolling down (Option to add header to every page #908)
• Builtin --follow support (Add follow option #457, Pass in "Ctrl-C" to pager? #808)
• Special handling for multiple files? (press 'n' to go to the next file)
• Error handling: we could print errors/warnings in the footer of the pager, instead of in the background (Error message printed in the middle of the output for another file #946)
• Better 'tab' handling (--tabs should be 0 by default instead of 4 #980, --tabs 0 is realy 8 #894)
• Handling of ANSI hyperlinks (bat breaks hyperlinks in command output #961)
• Reload the displayed file with a keypress.
• Disable style components via keypress. For example: switch off the side bar with a keypress - for easy copy&paste.
• Properly handle search with wrapping enabled. Search is currently broken for words being wrapped (Bat search is slow and not accurate sometime #1475)

General advantages of a "pager" as a library (not just for bat):

• Make everything easily configurable: custom header and footer messages, keybindings
• Bidirectional communication: send events like "terminal resize" or key presses back to the main application to trigger a redraw or other actions.

The obvious downsides are:

• this will be a lot of work 😄
• I'm sure there will be TONS of bug reports regarding obscure terminal emulators and operating systems where things are not working as expected - which results in even more work. less comes with 36 years(!!) of experience.

One way to circumvent the second point would be to make this an opt-in feature, at least in the beginning.

I have actually started to implement a really simple cross-platform pager-as-a-library like this in a private repository. The first results look promising 😄

I'd be happy to hear about your thoughts and ideas!

[eth-p]

I'm 100% in support for all the advantages this would provide.

The obvious downsides are:

• this will be a lot of work 😄

It would be, but the payoff would be amazing. All those useful features would make bat even better than it currently is.

One way to circumvent the [volume of bug reports] would be to make this an opt-in feature, at least in the beginning.

I did some brainstorming on my own about this a while back, and I came to the same conclusion. Making it an opt-in flag (--pager=pager-lib-name) would be a better user experience compared to suddenly changing the pager they use. It should also help reduce the potential for duplicate issues, since not everybody would be experiencing them at the same time.

I have actually started to implement a really simple cross-platform pager-as-a-library like this in a private repository. The first results look promising 😄

That looks great! I'd be happy to help contribute to it when you feel that it's ready for more contributors :)

I'd be happy to hear about your thoughts and ideas!

Scroll anchoring. When you resize the terminal, having your scroll position jump up or down because of soft-wrapped lines is a frustrating experience. Passing metadata along with each line that defines an anchor could help the pager automatically adjust the scroll position to the new location of the previous anchor.

[rr0ss0rr]

Wow. I just came across bat the other day and fell in love with the beauty of it. Everything mentioned would make bat a leader in the paging category. Thanks for your passion and hard work

[rr0ss0rr]

The addition of vim color schemes would be great

[mouse07410]

I'd also like to see this becoming a printer-friendly solution - not only the on-screen anchor, but per-printed-page anchors (header and footer).

[oconnor663]

"pager" as a library

A lot of tools could be better with a good library for this. man (and things like git help that invoke man) doesn't support terminal resizing properly either.

[metov]

Is it an option to simply make the pager a separate program, and have bat recommend using it? That way Unix philosophy is followed more closely.

[ssokolow]

@metov I'm not very picky and as long as it obeys my export PAGER='/usr/bin/less -iR' so I don't have to manually shove | less -iR after every invocation or wrap it in a shell function, I'd like that. but the proposal does make good points for why to have a built-in one.

[thecaralice]

Found minus and wanted to share

[sharkdp]

Found minus and wanted to share

Thanks - sounds promising!

[pickfire]

One more thing, I found more more useful compared to less since it provide a feature to stop between page breaks (I saw ^L) for RFC texts, maybe it would be good to support it?

more /usr/share/doc/rfc/txt/rfc6902.txt vs less -z56 /usr/share/doc/rfc/txt/rfc6902.txt (I need to manually specify window scrolling size to make it scroll better).

[thecaralice]

Found minus and wanted to share

It has a C-l shortcut for togging line numbers, but it shows them quite differently.

Command	Line numbers off	Line numbers on
bat -f less-rs.rs | less-rs /dev/stdin
bat --pager="less-rs /dev/stdin" -f less-rs.rs
bat --pager="less-rs /dev/stdin" -f less-rs.rs --terminal-width=$(($COLUMNS - 4))
bat --pager="less-rs /dev/stdin" -fp less-rs.rs
less-rs less-rs.rs

minus has a setting for never/always displaying line numbers (disabling the shortcut), but that's more like a workaround. There might be a way to change the line number display format.

[stdedos]

As the list keeps growing - would you mind publishing your effort #1053 (comment), behind an experimental flag?

Unless it raises the binary size too much for "just a wish" that is fully experimentally working.

[sharkdp]

As the list keeps growing - would you mind publishing your effort #1053 (comment), behind an experimental flag?

Sure - why not. I just "polished" it a little bit, but note that this is not much more than a quick & dirty proof of concept:

https://github.com/sharkdp/riffle

Instead of properly integrating this into bat as a library, the demo executable currently runs bat internally (on each terminal resize event) and captures its output.

What it can do:

• Implements most basic input events (up, down, page up, page down, mouse scrolling, quit)
• keeps header and footer fixed at top/bottom of the screen (I noticed that this only makes sense if we display a single file)
• properly redraw on terminal resize (has problems when not at the top of the file, due to wrapping of lines)
• open $EDITOR by pressing e
• toggle sidebar by pressing n
• toggle wrapping by pressing w

What it can not do:

• Display multiple files (easy to integrate, but clashes with the 'fix header at the top' feature)
• Properly handle terminal resize events when wrapping is enabled (see above)
• every other feature listed in the top post
• All other typical pager features:
  • searching
  • "quit if one screen"

If someone on Linux/amd64 wants to try this out without compiling: bat-riffle.zip

Screencast: https://asciinema.org/a/YhN35zGJxbnODvbDeXxTEDPG6 (note that I am resizing my terminal window in the last few seconds. Asciinema doesn't really show the full effect here.

Before we go further with this, we should probably come up with a list of requirements and also a draft for the design of this new pager library. I'm not sure if the current callback-based implementation is the best way to proceed. We should also carefully check if one of the already available libraries (e.g. https://crates.io/crates/minus) would be suitable for our needs.

[stavros-mobal]

If someone on Linux/amd64 wants to try this out without compiling: bat-riffle.zip

I am not sure what did I do wrong

$ bat-riffle ~/Documents/ISO_Next_Group-after-Shift.sh 
bat-riffle: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.33' not found (required by bat)
bat-riffle: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.32' not found (required by bat)
$ uname -a
Linux h 5.13.0-28-generic #31~20.04.1-Ubuntu SMP Wed Jan 19 14:08:10 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

... but maybe:

$ ldd --version
ldd (Ubuntu GLIBC 2.31-0ubuntu9.2) 2.31
Copyright (C) 2020 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Written by Roland McGrath and Ulrich Drepper.

That's from Ubuntu 20.04

[hacker-DOM]

While we're at it, the possibility to use two columns would be sweet ;-P

e.g.:

1. one column shows lines 1-30
2. second column shows lines 30-59
3. when you scroll, both scroll

😍

[hacker-DOM]

Fwiw, nu community is also working on a (structured) pager - nushell/nushell#6984

Honestly I wish they did more to generalize it, afaict it should work for any structured data/tree data, not necessarily nu objects