Add a "not" operator for hex bytes (#1676)

* Add not operator (~) for bytes in hex strings.

* Describe not hexstring not operator in documentation.

* Basic tests for the hextring NOT operator

* Fix not operator documentation ordering

docs/writingrules.rst

@@ -146,8 +146,8 @@ shown below.
 Hexadecimal strings
 -------------------
 
-Hexadecimal strings allow three special constructions that make them more
-flexible: wild-cards, jumps, and alternatives. Wild-cards are just placeholders
+Hexadecimal strings allow four special constructions that make them more
+flexible: wild-cards, not operators, jumps, and alternatives. Wild-cards are just placeholders
 that you can put into the string indicating that some bytes are unknown and they
 should match anything. The placeholder character is the question mark (?). Here
 you have an example of a hexadecimal string with wild-cards:
@@ -166,7 +166,25 @@ you have an example of a hexadecimal string with wild-cards:
 As shown in the example the wild-cards are nibble-wise, which means that you can
 define just one nibble of the byte and leave the other unknown.
 
-Wild-cards are useful when defining strings whose content can vary but you know
+In some cases you may wish to specify that a byte is not a specific value. For
+that you can use the not operator with a byte value:
+
+.. code-block:: yara
+
+    rule NotExample
+    {
+        strings:
+            $hex_string = { F4 23 ~00 62 B4 }
+
+        condition:
+            $hex_string
+    }
+
+In the example above we have a byte prefixed with a tilda (~), which is the not operator.
+This defines that the byte in that location can take any value except the value specified.
+In this case the string will only match if the byte is not 00.
+
+Wild-cards and not operators are useful when defining strings whose content can vary but you know
 the length of the variable chunks, however, this is not always the case. In some
 circumstances you may need to define strings with chunks of variable content and
 length. In those situations you can use jumps instead of wild-cards:

libyara/atoms.c

@@ -1142,6 +1142,7 @@ static int _yr_atoms_extract_from_re(
       case RE_NODE_ANCHOR_END:
       case RE_NODE_WORD_BOUNDARY:
       case RE_NODE_NON_WORD_BOUNDARY:
+      case RE_NODE_NOT_LITERAL:
 
         si.new_appending_node = current_appending_node;
         si.re_node = NULL;