Add option (--scan-list) to scan a list of files and/or directories

[lmzke]

Continuation of #1254.

Adds a new mode (--scan-list) to /usr/bin/yara that scans multiple files and directories (with or without recursion/-r, as usual). FILE (final argument) is not scanned itself in this mode, but instead contains the scan list, one item (file or directory) per line.

The use case is a vast collection of potential targets (in a flat directory or a dense tree) and curated set of actual scan targets (e.g., contextual tags).

Code footprint is small, dovetailing into existing scan_dir and file_queue_put.

Command-line convention is maintained:

• yara [OPTION]... [NAMESPACE:]RULES_FILE... FILE | DIR | PID.

A very rough test script (not intended to thoroughly exercise):

#!/bin/bash -x

YARA=/build/yara-e66b1385/bin/yara
#FLAGS=-r --scan-list
FLAGS=--scan-list

DIR=$(mktemp -d)
RULE_FILE=$DIR/rule.yara
SCAN_FILE=$DIR/to-scan.txt
FILES=$DIR/files

mkdir -p $FILES/manual $FILES/wanted $FILES/unwanted
echo "rule hit {condition: true}" > $RULE_FILE
for FILE in \
    $FILES/wanted/yes \
    $FILES/unwanted/no \
    $FILES/manual/yes \
    $FILES/manual/no; do
  touch $FILE
done
echo -e "$FILES/wanted\n$FILES/manual/yes" > $SCAN_FILE
$YARA $FLAGS $RULE_FILE $SCAN_FILE
echo "User needs to manually clean: $DIR"

$ ./run-test.sh
+ YARA=/build/yara-e66b1385/bin/yara
+ FLAGS=--scan-list
++ mktemp -d
+ DIR=/tmp/tmp.TInnsm9Gse
+ RULE_FILE=/tmp/tmp.TInnsm9Gse/rule.yara
+ SCAN_FILE=/tmp/tmp.TInnsm9Gse/to-scan.txt
+ FILES=/tmp/tmp.TInnsm9Gse/files
+ mkdir -p /tmp/tmp.TInnsm9Gse/files/manual /tmp/tmp.TInnsm9Gse/files/wanted /tmp/tmp.TInnsm9Gse/files/unwanted
+ echo 'rule hit {condition: true}'
+ for FILE in $FILES/wanted/yes $FILES/unwanted/no $FILES/manual/yes $FILES/manual/no
+ touch /tmp/tmp.TInnsm9Gse/files/wanted/yes
+ for FILE in $FILES/wanted/yes $FILES/unwanted/no $FILES/manual/yes $FILES/manual/no
+ touch /tmp/tmp.TInnsm9Gse/files/unwanted/no
+ for FILE in $FILES/wanted/yes $FILES/unwanted/no $FILES/manual/yes $FILES/manual/no
+ touch /tmp/tmp.TInnsm9Gse/files/manual/yes
+ for FILE in $FILES/wanted/yes $FILES/unwanted/no $FILES/manual/yes $FILES/manual/no
+ touch /tmp/tmp.TInnsm9Gse/files/manual/no
+ echo -e '/tmp/tmp.TInnsm9Gse/files/wanted\n/tmp/tmp.TInnsm9Gse/files/manual/yes'
+ /build/yara-e66b1385/bin/yara --scan-list /tmp/tmp.TInnsm9Gse/rule.yara /tmp/tmp.TInnsm9Gse/to-scan.txt
hit /tmp/tmp.TInnsm9Gse/files/wanted/yes
hit /tmp/tmp.TInnsm9Gse/files/manual/yes
+ echo 'User needs to manually clean: /tmp/tmp.TInnsm9Gse'
User needs to manually clean: /tmp/tmp.TInnsm9Gse

As discussed in #1254, this PR uses POSIX.1-2008 getline(3) (strok_s for Windows) to parse the scan list file line by line.

See also historical #550. I believe the code contribution here is value-add.

[googlebot]

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project (if not, look below for help). Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please visit https://cla.developers.google.com/ to sign.

Once you've signed (or fixed any issues), please reply here with @googlebot I signed it! and we'll verify it.

---

What to do if you already signed the CLA

Individual signers

• It's possible we don't have your GitHub username or you're using a different email address on your commit. Check your existing CLA data and verify that your email is set on your git commits.

Corporate signers

• Your company has a Point of Contact who decides which employees are authorized to participate. Ask your POC to be added to the group of authorized contributors. If you don't know who your Point of Contact is, direct the Google project maintainer to go/cla#troubleshoot (Public version).
• The email used to register you as an authorized contributor must be the email used for the Git commit. Check your existing CLA data and verify that your email is set on your git commits.
• The email used to register you as an authorized contributor must also be attached to your GitHub account.

ℹ️ Googlers: Go here for more info.

[lmzke]

@googlebot I signed it!

[googlebot]

CLAs look good, thanks!

ℹ️ Googlers: Go here for more info.

[plusvic]

This is failing to compile in Cygwin because strtok_s not being defined:
https://ci.appveyor.com/project/plusvic/yara/builds/32465401/job/q7i6uk8ug2ig8yo7

Fixed in #1264