Architecture

rga

the rga binary calls rg with most arguments passed through but a few arguments added based on the configured adapters. The most important one is --pre rga-preproc, which instructs ripgrep to call the rga-preproc binary for each found file.

rga-preproc

rga-preproc then takes the configured list of adapters and

Adapters

An adapter converts one input file (e.g. a zip archive) to a stream of the "files" within that file. Adapters are composable / recursive. An adapter recieves binary data and may output binary data if there's another adapter intended to process the output to finally convert it to text.

In Rust, an adapter basically consists of a function adapt(Box<dyn AsyncRead + Send>) -> Result<Stream<Item = Box<dyn AsyncRead + Send>>> plus some config functions. So each adapter takes a readable stream and returns an iterator of readable streams that represent the files within the input file.

Configurable / spawning adapters

A spawning adapter adapts a file by creating a child process, sending the input file to stdin and recieving the output file on stdout. New spawning adapters can be added via the config file. Currently spawning adapters always only have one output. Spawning adapters are similar to the basic --pre xxx argument to ripgrep except with included caching of the result and distinguishing by file types.

Caching

While reading the final output from the adapter chain, rga-preproc compresses the output with zstd and stores the result in a cache database in ~/.cache assuming the total size in bytes is less than a configured threshold.