CTRL-R: Show timestamp and syntax highlighted command

[junegunn]

Close #1049
Close #3836

This is a proof-of-concept implementation of enhanced CTRL-R binding for bash.

Screenshot

Technicals

HISTTIMEFORMAT performance

$ HISTTIMEFORMAT='%F %T: '
$ time history | wc -l
   18295

real    0m1.132s
user    0m0.190s
sys     0m0.975s

$ time history > /dev/null

real    0m0.034s
user    0m0.024s
sys     0m0.009s

For some reason, $HISTTIMEFORMAT makes piping history extremely slow. Strangely, there is no slowdown when redirecting the output to another file.

So we can work around this performance problem by writing it to a temp file and then reading it. But...

Syntax-highlighting of each command

We need to syntax-highlight each command, however, it's not viable to execute an external process such as bat for each command.

# This is extremely slow
while read line; do
  bat --style=plain --no-pager --language=bash <<< "$line"
done < ~/.bash_history

It's much faster to pass the whole file to bat at once,

bat --style=plain --no-pager --language=bash ~/.bash_history

However, syntax-highlighting quickly breaks down due to malformed commands in history.

Complexity of the code

The Perl and Awk scripts for de-duplicating the list and handling multi-line commands are already quite complex and not easy to read and maintain. Adding timestamp and syntax-highlighting support to them would bring too much complexity.

Solution

I used a Ruby script to implement the enhanced CTRL-R binding. rouge gem was used for syntax-highlighting. The code isn't short, but I'd argue that it's still more readable.

Drawbacks

• Unavailability of Ruby on some systems.
• Yet another dependency: rouge gem required for syntax-highlighting.
• Performance. Slow to syntax-highlight many commands.
  • It takes 4 seconds to process my 11MB command history
  • Without syntax highlighting, 1 second. But this is still quite slower than the current implementation

Alternatives

• Should we write an external tool for formatting the history file? It would yield better performance, but it's a hard dependency users have to install.
  • Doesn't help much. See below.
• Embed the functionality in fzf. This avoids an extra dependency, but this is a bad design choice.

TODO

• Other shells

[junegunn]

Formatter program in compiled languages

Golang

I wrote a Go version of the Ruby code using Chroma library. However, the performance is worse than the Ruby version.

$ wc -l foo
  441988 foo

$ du -h foo
 11M    foo

$ time ./fmt.rb foo > /dev/null

real    0m3.259s
user    0m2.885s
sys     0m0.094s

$ time ./fmt.go foo > /dev/null

real    0m8.108s
user    0m7.682s
sys     0m0.604s

$ time ./fmt.go.nohl foo  > /dev/null

real    0m0.218s
user    0m0.175s
sys     0m0.056s

Rust

I'm not familiar with Rust ecosystem, but trishume/syntect seems to be the most popular choice for syntax highlighting. However, according to the doc,

Highlighting 9200 lines/247kb of jQuery 2.1 takes 600ms

but the file I tested with is 11MBs, so we can't expect stellar performance. bat is known to use the library, so let's see.

time bat --style=plain --language=bash --color=always foo > /dev/null

real    0m3.881s
user    0m3.697s
sys     0m0.153s

So, nope.