Initial public commit

CHANGELOG.md

@@ -0,0 +1,11 @@
+# Changelog
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [Unreleased]
+
+## [0.5.0] - YYYY-MM-DD
+### Added
+- Initial public release of Dlint

CONTRIBUTING.md

@@ -0,0 +1,51 @@
+# Contributing
+
+`Dlint` welcomes contributions from anyone. If you have an idea for a linter
+but don't know how to implement one please [create a new issue](https://github.com/duo-labs/dlint/issues).
+With `dlint` we can find security bugs, encourage best practices, and eliminate
+anti-patterns across the Python ecosystem.
+
+`Dlint` is built on top of Python's [AST](https://docs.python.org/3/library/ast.html)
+module and the [`flake8` plugin system](http://flake8.pycqa.org/en/latest/user/using-plugins.html).
+It may be helpful to review those systems before beginning `dlint` development,
+but `dlint` aims to be easily extendable without requiring a lot of background
+knowledge. **Further, please check out our brief section on [developing](https://github.com/duo-labs/dlint#developing)
+`dlint` before making changes.**
+
+# New Linters
+
+When adding new linters:
+
+* New linters should be added to the `dlint/linters/` directory.
+* Add a new file and class inheriting from `base.BaseLinter` for each new linter.
+* Add a "pass-through" import of the new class to `dlint.linters.__init__.py`.
+* Add the new class to `ALL` in `dlint.linters.__init__.py`.
+* Ensure new rules are properly tested (high or complete test coverage).
+* Ensure new code adheres to the style guide/linting process.
+* Add new rule information to `CHANGELOG.md` under `Unreleased` section, `Added` sub-section.
+
+From here, please create a [pull request](https://github.com/duo-labs/dlint/pulls)
+with your changes and wait for a review.
+
+# Fixing/Reporting Bugs
+
+When fixing or reporting bugs in `dlint` please [create a new issue](https://github.com/duo-labs/dlint/issues)
+first. This issue should include a snippet of code for reproducing the bug.
+
+E.g.
+
+*I expected `dlint` to flag the following code for faulty use of the `foo` module:*
+
+```
+from bar import foo
+
+var = result + 7
+widget = foo.baz(var)
+send_result(widget)
+```
+
+*Please update `dlint` to catch this. Thanks!*
+
+After reporting the issue, if you'd like to help fix it, please create a
+[pull request](https://github.com/duo-labs/dlint/pulls) with the
+fix applied and wait for a review.

LICENSE

@@ -0,0 +1,26 @@
+Copyright 2019 Duo Security
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are met:
+
+1. Redistributions of source code must retain the above copyright notice, this
+list of conditions and the following disclaimer.
+
+2. Redistributions in binary form must reproduce the above copyright notice,
+this list of conditions and the following disclaimer in the documentation
+and/or other materials provided with the distribution.
+
+3. Neither the name of the copyright holder nor the names of its contributors
+may be used to endorse or promote products derived from this software without
+specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
+ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
+FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

README.md

@@ -0,0 +1,142 @@
+# Dlint
+
+Dlint is a tool for encouraging best coding practices and helping ensure we're
+writing secure code.
+
+> The most important thing I have done as a programmer in recent years is to
+> aggressively pursue static code analysis. Even more valuable than the
+> hundreds of serious bugs I have prevented with it is the change in mindset
+> about the way I view software reliability and code quality.
+> - [John Carmack, 2011](https://www.gamasutra.com/view/news/128836/InDepth_Static_Code_Analysis.php)
+
+> For a static analysis project to succeed, developers must feel they benefit
+> from and enjoy using it.
+> - [Lessons from Building Static Analysis Tools at Google](https://cacm.acm.org/magazines/2018/4/226371-lessons-from-building-static-analysis-tools-at-google/fulltext)
+
+# Installing
+
+**TODO: Update with PyPI instructions**
+
+And double check that it was installed correctly:
+
+```
+$ python -m flake8 -h
+Usage: flake8 [options] file file ...
+
+...
+
+Installed plugins: dlint: 0.5.0, mccabe: 0.5.3, pycodestyle: 2.2.0, pyflakes: 1.3.0
+```
+
+Note the `dlint: 0.5.0`.
+
+# Using
+
+Dlint uses `flake8` to perform its linting functionality. This allows us to
+utilize many useful `flake8` features without re-inventing the wheel.
+
+## CLI
+
+Let's run a simple check:
+
+```
+$ cat test.py
+#!/usr/bin/env python
+
+print("TEST1")
+
+exec('print("TEST2")')
+```
+
+```
+$ python -m flake8 --select=DUO test.py
+test.py:5:1: DUO105 use of "exec" not allowed
+```
+
+The `--select=DUO` flag tells `flake8` to only run Dlint lint rules.
+
+## Arc Lint
+
+Dlint is easily integrated with [Arcanist's](https://secure.phabricator.com/book/phabricator/article/arcanist/)
+linting process via the [.arclint](https://secure.phabricator.com/book/phabricator/article/arcanist_lint/)
+configuration file. Dlint rules will automatically be run via `flake8` once
+it's installed, so the standard `flake8` configuration will work:
+
+```
+{
+    "linters": {
+        "sample": {
+            "type": "flake8"
+        }
+    }
+}
+```
+
+You can also utilize more granular control over the linting process:
+
+```
+{
+    "linters": {
+        "sample": {
+            "type": "flake8"
+        },
+        "bin": ["python2.7", "python2"],
+        "flags": ["-m", "flake8", "--select", "DUO"]
+    }
+}
+```
+
+## Inline Editor
+
+Dlint results can also be included inline in your editor for fast feedback.
+This typically requires an editor plugin or extension. Here are some starting
+points for common editors:
+
+* Vim: [https://github.com/vim-syntastic/syntastic](https://github.com/vim-syntastic/syntastic)
+* Emacs: [https://github.com/flycheck/flycheck](https://github.com/flycheck/flycheck)
+* Sublime: [https://github.com/SublimeLinter/SublimeLinter-flake8](https://github.com/SublimeLinter/SublimeLinter-flake8)
+* PyCharm: [https://foxmask.net/post/2016/02/17/pycharm-running-flake8/](https://foxmask.net/post/2016/02/17/pycharm-running-flake8/)
+* Atom: [https://atom.io/packages/linter-flake8](https://atom.io/packages/linter-flake8)
+* Visual Studio Code: [https://code.visualstudio.com/docs/python/linting#_flake8](https://code.visualstudio.com/docs/python/linting#_flake8)
+
+# Custom Plugins
+
+Dlint's custom plugins are built on a [simple naming convention](https://packaging.python.org/guides/creating-and-discovering-plugins/#using-naming-convention),
+and rely on [Python modules](https://docs.python.org/3/distutils/examples.html#pure-python-distribution-by-module).
+To make a Dlint custom plugin use the following conventions:
+
+* The Python module name **must** start with `dlint_plugin_`.
+* The linter class name **must** start with `Dlint`.
+* The linter class **should** inherit from `dlint.linters.base.BaseLinter`.
+  * If for some reason you'd like to avoid this, then you **must** implement
+	the `get_results` function appropriately and inherit from `ast.NodeVisitor`.
+
+See an [example plugin](https://github.com/duo-labs/dlint-plugin-example) for further details.
+
+# Developing
+
+First, install development packages:
+
+```
+$ pip install -r requirements.txt
+$ pip install -r requirements-dev.txt
+$ pip install -e .
+```
+
+## Testing
+
+```
+$ pytest
+```
+
+## Linting
+
+```
+$ flake8
+```
+
+## Coverage
+
+```
+$ pytest --cov
+```

dlint/__init__.py

@@ -0,0 +1,14 @@
+from __future__ import (
+    absolute_import,
+    division,
+    print_function,
+    unicode_literals,
+)
+
+from . import linters  # noqa F401
+from . import test  # noqa F401
+from . import tree  # noqa F401
+from . import util  # noqa F401
+
+__name__ = 'dlint'
+__version__ = '0.5.0'

dlint/extension.py

@@ -0,0 +1,58 @@
+#!/usr/bin/env python
+
+from __future__ import (
+    absolute_import,
+    division,
+    print_function,
+    unicode_literals,
+)
+
+import importlib
+import inspect
+import pkgutil
+
+import dlint
+
+
+class Flake8Extension(object):
+    name = dlint.__name__
+    version = dlint.__version__
+
+    def __init__(self, tree, filename):
+        self.tree = tree
+        self.filename = filename
+
+    @staticmethod
+    def get_plugin_classes():
+        module_prefix = 'dlint_plugin_'
+        class_prefix = 'Dlint'
+
+        plugin_modules = [
+            importlib.import_module(name)
+            for finder, name, ispkg in pkgutil.iter_modules()
+            if name.startswith(module_prefix)
+        ]
+        plugin_classes = [
+            cls
+            for module in plugin_modules
+            for name, cls in inspect.getmembers(module, predicate=inspect.isclass)
+            if name.startswith(class_prefix)
+        ]
+
+        return plugin_classes
+
+    def run(self):
+        plugin_classes = self.get_plugin_classes()
+        linters = dlint.linters.ALL + tuple(plugin_classes)
+
+        for linter in linters:
+            linter_instance = linter()
+            linter_instance.visit(self.tree)
+
+            for result in linter_instance.get_results():
+                yield (
+                    result.lineno,
+                    result.col_offset,
+                    result.message,
+                    linter
+                )

dlint/linters/__init__.py

@@ -0,0 +1,73 @@
+from __future__ import (
+    absolute_import,
+    division,
+    print_function,
+    unicode_literals,
+)
+
+from . import base  # noqa F401
+from . import helpers  # noqa F401
+
+from .bad_commands_use import BadCommandsUseLinter
+from .bad_compile_use import BadCompileUseLinter
+from .bad_dl_use import BadDlUseLinter
+from .bad_duo_client_use import BadDuoClientUseLinter
+from .bad_gl_use import BadGlUseLinter
+from .bad_eval_use import BadEvalUseLinter
+from .bad_exec_use import BadExecUseLinter
+from .bad_input_use import BadInputUseLinter
+from .bad_marshal_use import BadMarshalUseLinter
+from .bad_onelogin_kwarg_use import BadOneLoginKwargUseLinter
+from .bad_onelogin_module_attribute_use import BadOneLoginModuleAttributeUseLinter
+from .bad_os_use import BadOSUseLinter
+from .bad_popen2_use import BadPopen2UseLinter
+from .bad_random_generator_use import BadRandomGeneratorUseLinter
+from .bad_requests_use import BadRequestsUseLinter
+from .bad_shelve_use import BadShelveUseLinter
+from .bad_subprocess_use import BadSubprocessUseLinter
+from .bad_ssl_module_attribute_use import BadSSLModuleAttributeUseLinter
+from .bad_sys_use import BadSysUseLinter
+from .bad_tarfile_use import BadTarfileUseLinter
+from .bad_tempfile_use import BadTempfileUseLinter
+from .bad_pickle_use import BadPickleUseLinter
+from .bad_xml_use import BadXMLUseLinter
+from .bad_xmlrpc_use import BadXmlrpcUseLinter
+from .bad_yaml_use import BadYAMLUseLinter
+from .bad_zipfile_use import BadZipfileUseLinter
+from .format_string import FormatStringLinter
+from .inlinecallbacks_yield_statement import InlineCallbacksYieldStatementLinter
+from .returnvalue_in_inlinecallbacks import ReturnValueInInlineCallbacksLinter
+from .yield_return_statement import YieldReturnStatementLinter
+
+ALL = (
+    BadCommandsUseLinter,
+    BadCompileUseLinter,
+    BadDlUseLinter,
+    BadDuoClientUseLinter,
+    BadGlUseLinter,
+    BadEvalUseLinter,
+    BadExecUseLinter,
+    BadInputUseLinter,
+    BadMarshalUseLinter,
+    BadOneLoginKwargUseLinter,
+    BadOneLoginModuleAttributeUseLinter,
+    BadOSUseLinter,
+    BadPopen2UseLinter,
+    BadRandomGeneratorUseLinter,
+    BadRequestsUseLinter,
+    BadShelveUseLinter,
+    BadSSLModuleAttributeUseLinter,
+    BadSysUseLinter,
+    BadSubprocessUseLinter,
+    BadTempfileUseLinter,
+    BadTarfileUseLinter,
+    BadPickleUseLinter,
+    BadXMLUseLinter,
+    BadXmlrpcUseLinter,
+    BadYAMLUseLinter,
+    BadZipfileUseLinter,
+    FormatStringLinter,
+    InlineCallbacksYieldStatementLinter,
+    ReturnValueInInlineCallbacksLinter,
+    YieldReturnStatementLinter,
+)