backscanner

Ever needed or wondered how to efficiently search for something in a log file, but starting at the end and going backward? Here's your solution.

Package backscanner provides a scanner similar to bufio.Scanner, but it reads and returns lines in reverse order, starting at a given position (which may be the end of the input) and going backward.

This library only uses the standard library, but the test uses an external package. To install this library (along with the test dependency), simply run:

go get -t github.com/icza/backscanner

Advancing and accessing lines of the input is done by calling Scanner.Line(), which returns the next line (previous in the source) as a string.

For maximum efficiency there is Scanner.LineBytes(). It returns the next line as a byte slice, which shares its backing array with the internal buffer of Scanner. This is because no copy is made from the line data; but this also means you can only inspect or search in the slice before calling Line() or LineBytes() again, as the content of the internal buffer–and thus slices returned by LineBytes()–may be overwritten. If you need to retain the line data, make a copy of it or use Line().

Example using it:

input := "Line1\nLine2\nLine3"
scanner := backscanner.New(strings.NewReader(input), len(input))
for {
	line, pos, err := scanner.Line()
	if err != nil {
		fmt.Println("Error:", err)
		break
	}
	fmt.Printf("Line position: %2d, line: %q\n", pos, line)
}

Output:

Line position: 12, line: "Line3"
Line position:  6, line: "Line2"
Line position:  0, line: "Line1"
Error: EOF

Using it to efficiently scan a file, finding last occurrence of a string ("error"):

file, err := os.Open("mylog.txt")
if err != nil {
	panic(err)
}
fileStatus, err := file.Stat()
if err != nil {
	panic(err)
}
defer file.Close()

scanner := backscanner.New(file, int(fileStatus.Size()))
what := []byte("error")
for {
	line, pos, err := scanner.LineBytes()
	if err != nil {
		if err == io.EOF {
			fmt.Printf("%q is not found in file.\n", what)
		} else {
			fmt.Println("Error:", err)
		}
		break
	}
	if bytes.Contains(line, what) {
		fmt.Printf("Found %q at line position: %d, line: %q\n", what, pos, line)
		break
	}
}