hex colors, ANSI on Windows, coloring docs, remove unsafe

FAQ.md

@@ -6,6 +6,8 @@
 * [Does ripgrep have a man page?](#manpage)
 * [Does ripgrep have support for shell auto-completion?](#complete)
 * [How do I use lookaround and/or backreferences?](#fancy)
+* [How do I configure ripgrep's colors?](#colors)
+* [How do I enable true colors on Windows?](#truecolors-windows)
 * [How do I stop ripgrep from messing up colors when I kill it?](#stop-ripgrep)
 * [How can I get results in a consistent order?](#order)
 * [How do I search files that aren't UTF-8?](#encoding)
@@ -24,7 +26,8 @@
 Does ripgrep support configuration files?
 </h3>
 
-Yes. See the [guide's section on configuration files](#configuration-file).
+Yes. See the
+[guide's section on configuration files](GUIDE.md#configuration-file).
 
 
 <h3 name="changelog">
@@ -160,6 +163,102 @@ written in Rust, then it is possible ripgrep will provide it as an opt-in
 feature.
 
 
+<h3 name="colors">
+How do I configure ripgrep's colors?
+</h3>
+
+ripgrep has two flags related to colors:
+
+* `--color` controls *when* to use colors.
+* `--colors` controls *which* colors to use.
+
+The `--color` flag accepts one of the following possible values: `never`,
+`auto`, `always` or `ansi`. The `auto` value is the default and will cause
+ripgrep to only enable colors when it is printing to a terminal. But if you
+pipe ripgrep to a file or some other process, then it will suppress colors.
+
+The --colors` flag is a bit more complicated. The general format is:
+
+```
+--colors '{type}:{attribute}:{value}'
+```
+
+* `{type}` should be one of `path`, `line`, `column` or `match`. Each of these
+  correspond to the four different types of things that ripgrep will add color
+  to in its output. Select the type whose color you want to change.
+* `{attribute}` should be one of `fg`, `bg` or `style`, corresponding to
+  foreground color, background color, or miscellaneous styling (such as whether
+  to bold the output or not).
+* `{value}` is determined by the value of `{attribute}`. If `{attribute}` is
+  `style`, then `{value}` should be one of `nobold`, `bold`, `nointense` or
+  `intense`. If `{attribute}` is `fg` or `bg`, then `{value}` should be a
+  color.
+
+A color is specified by either one of eight of English names, a single 256-bit
+number or an RGB triple (with over 16 million possible values, or "true
+color").
+
+The color names are `red`, `blue`, `green`, `cyan`, `magenta`, `yellow`,
+`white` or `black`.
+
+A single 256-bit number is a value in the range 0-255 (inclusive). It can
+either be in decimal format (e.g., `62`) or hexadecimal format (e.g., `0x3E`).
+
+An RGB triple corresponds to three numbers (decimal or hexadecimal) separated
+by commas.
+
+As a special case, `--colors '{type}:none'` will clear all colors and styles
+associated with `{type}`, which lets you start with a clean slate (instead of
+building on top of ripgrep's default color settings).
+
+Here's an example that makes highlights the matches with a nice blue background
+with bolded white text:
+
+```
+$ rg somepattern \
+    --colors 'match:none' \
+    --colors 'match:bg:0x33,0x66,0xFF' \
+    --colors 'match:fg:white' \
+    --colors 'match:style:bold'
+```
+
+Colors are an ideal candidate to set in your
+[configuration file](GUIDE.md#configuration-file). See the
+[question on emulating The Silver Searcher's output style](#silver-searcher-output)
+for an example specific to colors.
+
+
+<h3 name="truecolors-windows">
+How do I enable true colors on Windows?
+</h3>
+
+First, see the previous question's
+[answer on configuring colors](#colors).
+
+Secondly, coloring on Windows is a bit complicated. If you're using a terminal
+like Cygwin, then it's likely true color support already works out of the box.
+However, if you are using a normal Windows console (`cmd` or `PowerShell`) and
+a version of Windows prior to 10, then there is no known way to get true
+color support. If you are on Windows 10 and using a Windows console, then
+true colors should work out of the box with one caveat: you might need to
+clear ripgrep's default color settings first. That is, instead of this:
+
+```
+$ rg somepattern --colors 'match:fg:0x33,0x66,0xFF'
+```
+
+you should do this
+
+```
+$ rg somepattern --colors 'match:none' --colors 'match:fg:0x33,0x66,0xFF'
+```
+
+This is because ripgrep might set the default style for `match` to `bold`, and
+it seems like Windows 10's VT100 support doesn't permit bold and true color
+ANSI escapes to be used simultaneously. The work-around above will clear
+ripgrep's default styling, allowing you to craft it exactly as desired.
+
+
 <h3 name="stop-ripgrep">
 How do I stop ripgrep from messing up colors when I kill it?
 </h3>

ci/script.sh

@@ -7,9 +7,6 @@ set -ex
 . "$(dirname $0)/utils.sh"
 
 main() {
-    # Travis sometimes caches the target directory, which makes testing the
-    # output of cargo a little trickier. So just wipe it.
-    cargo clean
     # Test a normal debug build.
     cargo build --target "$TARGET" --verbose --all
 

globset/src/glob.rs

@@ -1,4 +1,3 @@
-use std::ffi::{OsStr, OsString};
 use std::fmt;
 use std::hash;
 use std::iter;
@@ -28,7 +27,7 @@ pub enum MatchStrategy {
     BasenameLiteral(String),
     /// A pattern matches if and only if the file path's extension matches this
     /// literal string.
-    Extension(OsString),
+    Extension(String),
     /// A pattern matches if and only if this prefix literal is a prefix of the
     /// candidate file path.
     Prefix(String),
@@ -47,7 +46,7 @@ pub enum MatchStrategy {
     /// extension. Note that this is a necessary but NOT sufficient criterion.
     /// Namely, if the extension matches, then a full regex search is still
     /// required.
-    RequiredExtension(OsString),
+    RequiredExtension(String),
     /// A regex needs to be used for matching.
     Regex,
 }
@@ -154,7 +153,7 @@ impl GlobStrategic {
                 lit.as_bytes() == &*candidate.basename
             }
             MatchStrategy::Extension(ref ext) => {
-                candidate.ext == ext
+                ext.as_bytes() == &*candidate.ext
             }
             MatchStrategy::Prefix(ref pre) => {
                 starts_with(pre.as_bytes(), byte_path)
@@ -166,7 +165,8 @@ impl GlobStrategic {
                 ends_with(suffix.as_bytes(), byte_path)
             }
             MatchStrategy::RequiredExtension(ref ext) => {
-                candidate.ext == ext && self.re.is_match(byte_path)
+                let ext = ext.as_bytes();
+                &*candidate.ext == ext && self.re.is_match(byte_path)
             }
             MatchStrategy::Regex => self.re.is_match(byte_path),
         }
@@ -295,7 +295,7 @@ impl Glob {
     /// std::path::Path::extension returns. Namely, this extension includes
     /// the '.'. Also, paths like `.rs` are considered to have an extension
     /// of `.rs`.
-    fn ext(&self) -> Option<OsString> {
+    fn ext(&self) -> Option<String> {
         if self.opts.case_insensitive {
             return None;
         }
@@ -319,11 +319,11 @@ impl Glob {
             Some(&Token::Literal('.')) => {}
             _ => return None,
         }
-        let mut lit = OsStr::new(".").to_os_string();
+        let mut lit = ".".to_string();
         for t in self.tokens[start + 2..].iter() {
             match *t {
                 Token::Literal('.') | Token::Literal('/') => return None,
-                Token::Literal(c) => lit.push(c.to_string()),
+                Token::Literal(c) => lit.push(c),
                 _ => return None,
             }
         }
@@ -337,7 +337,7 @@ impl Glob {
     /// This is like `ext`, but returns an extension even if it isn't sufficent
     /// to imply a match. Namely, if an extension is returned, then it is
     /// necessary but not sufficient for a match.
-    fn required_ext(&self) -> Option<OsString> {
+    fn required_ext(&self) -> Option<String> {
         if self.opts.case_insensitive {
             return None;
         }
@@ -360,7 +360,7 @@ impl Glob {
             None
         } else {
             ext.reverse();
-            Some(OsString::from(ext.into_iter().collect::<String>()))
+            Some(ext.into_iter().collect())
         }
     }
 
@@ -927,8 +927,6 @@ fn ends_with(needle: &[u8], haystack: &[u8]) -> bool {
 
 #[cfg(test)]
 mod tests {
-    use std::ffi::{OsStr, OsString};
-
     use {GlobSetBuilder, ErrorKind};
     use super::{Glob, GlobBuilder, Token};
     use super::Token::*;
@@ -1021,7 +1019,6 @@ mod tests {
     }
 
     fn s(string: &str) -> String { string.to_string() }
-    fn os(string: &str) -> OsString { OsStr::new(string).to_os_string() }
 
     fn class(s: char, e: char) -> Token {
         Class { negated: false, ranges: vec![(s, e)] }
@@ -1319,19 +1316,19 @@ mod tests {
         Literal('f'), Literal('o'), ZeroOrMore, Literal('o'),
     ]), SLASHLIT);
 
-    ext!(extract_ext1, "**/*.rs", Some(os(".rs")));
+    ext!(extract_ext1, "**/*.rs", Some(s(".rs")));
     ext!(extract_ext2, "**/*.rs.bak", None);
-    ext!(extract_ext3, "*.rs", Some(os(".rs")));
+    ext!(extract_ext3, "*.rs", Some(s(".rs")));
     ext!(extract_ext4, "a*.rs", None);
     ext!(extract_ext5, "/*.c", None);
     ext!(extract_ext6, "*.c", None, SLASHLIT);
-    ext!(extract_ext7, "*.c", Some(os(".c")));
+    ext!(extract_ext7, "*.c", Some(s(".c")));
 
-    required_ext!(extract_req_ext1, "*.rs", Some(os(".rs")));
-    required_ext!(extract_req_ext2, "/foo/bar/*.rs", Some(os(".rs")));
-    required_ext!(extract_req_ext3, "/foo/bar/*.rs", Some(os(".rs")));
-    required_ext!(extract_req_ext4, "/foo/bar/.rs", Some(os(".rs")));
-    required_ext!(extract_req_ext5, ".rs", Some(os(".rs")));
+    required_ext!(extract_req_ext1, "*.rs", Some(s(".rs")));
+    required_ext!(extract_req_ext2, "/foo/bar/*.rs", Some(s(".rs")));
+    required_ext!(extract_req_ext3, "/foo/bar/*.rs", Some(s(".rs")));
+    required_ext!(extract_req_ext4, "/foo/bar/.rs", Some(s(".rs")));
+    required_ext!(extract_req_ext5, ".rs", Some(s(".rs")));
     required_ext!(extract_req_ext6, "./rs", None);
     required_ext!(extract_req_ext7, "foo", None);
     required_ext!(extract_req_ext8, ".foo/", None);

globset/src/lib.rs

@@ -108,7 +108,7 @@ extern crate regex;
 use std::borrow::Cow;
 use std::collections::{BTreeMap, HashMap};
 use std::error::Error as StdError;
-use std::ffi::{OsStr, OsString};
+use std::ffi::OsStr;
 use std::fmt;
 use std::hash;
 use std::path::Path;
@@ -458,7 +458,7 @@ impl GlobSetBuilder {
 pub struct Candidate<'a> {
     path: Cow<'a, [u8]>,
     basename: Cow<'a, [u8]>,
-    ext: &'a OsStr,
+    ext: Cow<'a, [u8]>,
 }
 
 impl<'a> Candidate<'a> {
@@ -469,7 +469,7 @@ impl<'a> Candidate<'a> {
         Candidate {
             path: normalize_path(path_bytes(path)),
             basename: os_str_bytes(basename),
-            ext: file_name_ext(basename).unwrap_or(OsStr::new("")),
+            ext: file_name_ext(basename).unwrap_or(Cow::Borrowed(b"")),
         }
     }
 
@@ -584,30 +584,30 @@ impl BasenameLiteralStrategy {
 }
 
 #[derive(Clone, Debug)]
-struct ExtensionStrategy(HashMap<OsString, Vec<usize>, Fnv>);
+struct ExtensionStrategy(HashMap<Vec<u8>, Vec<usize>, Fnv>);
 
 impl ExtensionStrategy {
     fn new() -> ExtensionStrategy {
         ExtensionStrategy(HashMap::with_hasher(Fnv::default()))
     }
 
-    fn add(&mut self, global_index: usize, ext: OsString) {
-        self.0.entry(ext).or_insert(vec![]).push(global_index);
+    fn add(&mut self, global_index: usize, ext: String) {
+        self.0.entry(ext.into_bytes()).or_insert(vec![]).push(global_index);
     }
 
     fn is_match(&self, candidate: &Candidate) -> bool {
         if candidate.ext.is_empty() {
             return false;
         }
-        self.0.contains_key(candidate.ext)
+        self.0.contains_key(&*candidate.ext)
     }
 
     #[inline(never)]
     fn matches_into(&self, candidate: &Candidate, matches: &mut Vec<usize>) {
         if candidate.ext.is_empty() {
             return;
         }
-        if let Some(hits) = self.0.get(candidate.ext) {
+        if let Some(hits) = self.0.get(&*candidate.ext) {
             matches.extend(hits);
         }
     }
@@ -670,14 +670,14 @@ impl SuffixStrategy {
 }
 
 #[derive(Clone, Debug)]
-struct RequiredExtensionStrategy(HashMap<OsString, Vec<(usize, Regex)>, Fnv>);
+struct RequiredExtensionStrategy(HashMap<Vec<u8>, Vec<(usize, Regex)>, Fnv>);
 
 impl RequiredExtensionStrategy {
     fn is_match(&self, candidate: &Candidate) -> bool {
         if candidate.ext.is_empty() {
             return false;
         }
-        match self.0.get(candidate.ext) {
+        match self.0.get(&*candidate.ext) {
             None => false,
             Some(regexes) => {
                 for &(_, ref re) in regexes {
@@ -695,7 +695,7 @@ impl RequiredExtensionStrategy {
         if candidate.ext.is_empty() {
             return;
         }
-        if let Some(regexes) = self.0.get(candidate.ext) {
+        if let Some(regexes) = self.0.get(&*candidate.ext) {
             for &(global_index, ref re) in regexes {
                 if re.is_match(&*candidate.path) {
                     matches.push(global_index);
@@ -775,16 +775,19 @@ impl MultiStrategyBuilder {
 
 #[derive(Clone, Debug)]
 struct RequiredExtensionStrategyBuilder(
-    HashMap<OsString, Vec<(usize, String)>>,
+    HashMap<Vec<u8>, Vec<(usize, String)>>,
 );
 
 impl RequiredExtensionStrategyBuilder {
     fn new() -> RequiredExtensionStrategyBuilder {
         RequiredExtensionStrategyBuilder(HashMap::new())
     }
 
-    fn add(&mut self, global_index: usize, ext: OsString, regex: String) {
-        self.0.entry(ext).or_insert(vec![]).push((global_index, regex));
+    fn add(&mut self, global_index: usize, ext: String, regex: String) {
+        self.0
+            .entry(ext.into_bytes())
+            .or_insert(vec![])
+            .push((global_index, regex));
     }
 
     fn build(self) -> Result<RequiredExtensionStrategy, Error> {