updated package setup (typeddjango#485)

* updated package setup

* updated to use python 3.9

* fixed test runner

* fixed typecheck tests

* fixed discrepencies

* added override to runner

* updated travis

* updated pre-commit hooks

* updated dep

.gitignore

@@ -1,15 +1,11 @@
 *.egg-info
-__pycache__/
-out/
-/test_sqlite.py
-/django
+.DS_Store
 .idea/
 .mypy_cache/
-build/
-dist/
-pip-wheel-metadata/
 .pytest_cache/
-/.envrc
-/.direnv
-django-sources/
-.venv/
+.venv/
+__pycache__/
+django-source/
+out/
+pip-wheel-metadata/
+stubgen/

.pre-commit-config.yaml

@@ -0,0 +1,39 @@
+# See https://pre-commit.com for more information
+# See https://pre-commit.com/hooks.html for more hooks
+default_language_version:
+  python: python3.9
+repos:
+  - repo: https://github.com/pre-commit/pre-commit-hooks
+    rev: v3.3.0
+    hooks:
+      - id: check-yaml
+      - id: trailing-whitespace
+      - id: check-executables-have-shebangs
+      - id: debug-statements
+      - id: check-merge-conflict
+  - repo: https://github.com/asottile/pyupgrade
+    rev: v2.7.3
+    hooks:
+      - id: pyupgrade
+        args: ["--py36-plus"]
+  - repo: https://github.com/pre-commit/mirrors-isort
+    rev: v5.6.4
+    hooks:
+      - id: isort
+  - repo: https://github.com/psf/black
+    rev: 20.8b1
+    hooks:
+      - id: black
+  - repo: https://gitlab.com/pycqa/flake8
+    rev: 3.8.4
+    hooks:
+      - id: flake8
+  - repo: local
+    hooks:
+      - id: mypy
+        name: mypy
+        entry: mypy
+        language: system
+        types: [ python ]
+        exclude: "scripts/*"
+        args: [ "--cache-dir=/dev/null", "--no-incremental" ]

.travis.yml

@@ -2,55 +2,51 @@ language: python
 cache: pip
 dist: xenial
 sudo: required
+
 jobs:
   include:
-    - name: Run plugin test suite with python 3.8
-      python: 3.8
-      script: 'pytest'
-
-    - name: Run plugin test suite with python 3.7
-      python: 3.7
-      script: 'pytest'
-
+    - name: Typecheck Django 3.0 test suite with python 3.9-dev
+      python: 3.9-dev
+      script: |
+        python ./scripts/typecheck_tests.py --django_version=3.0
     - name: Typecheck Django 3.0 test suite with python 3.8
       python: 3.8
       script: |
         python ./scripts/typecheck_tests.py --django_version=3.0
-
     - name: Typecheck Django 3.0 test suite with python 3.7
       python: 3.7
       script: |
         python ./scripts/typecheck_tests.py --django_version=3.0
-
     - name: Typecheck Django 3.0 test suite with python 3.6
       python: 3.6
       script: |
         python ./scripts/typecheck_tests.py --django_version=3.0
 
+    - name: Typecheck Django 2.2 test suite with python 3.9-dev
+      python: 3.9-dev
+      script: |
+        python ./scripts/typecheck_tests.py --django_version=2.2
+    - name: Typecheck Django 2.2 test suite with python 3.8
+      python: 3.8
+      script: |
+        python ./scripts/typecheck_tests.py --django_version=2.2
     - name: Typecheck Django 2.2 test suite with python 3.7
       python: 3.7
       script: |
         python ./scripts/typecheck_tests.py --django_version=2.2
+    - name: Typecheck Django 2.2 test suite with python 3.6
+      python: 3.6
+      script: |
+        python ./scripts/typecheck_tests.py --django_version=2.2
 
-    - name: Mypy for plugin code
-      python: 3.7
-      script: 'mypy ./mypy_django_plugin'
-
-    - name: Lint with black
-      python: 3.7
-      script: 'black --check django-stubs/ setup.py'
-
-    - name: Lint plugin code with flake8
-      python: 3.7
-      script: 'flake8'
-
-    - name: Lint stubs with flake8-pyi and check for unused imports
-      python: 3.7
-      script: 'flake8 --config flake8-pyi.ini'
+    - name: Lint using pre-commit
+      python: 3.9-dev
+      script:
+        - pre-commit install && pre-commit run --all-files
 
-    - name: Lint plugin code with isort
-      python: 3.7
-      script: 'isort --check --diff'
+    - name: Run plugin test suite with python 3.9-dev
+      python: 3.9-dev
+      script: pytest
 
 before_install: |
   sudo add-apt-repository ppa:ubuntugis/ubuntugis-unstable -y

CONTRIBUTING.md

@@ -1,111 +1,106 @@
-# How to contribute
+# Contribution Guide
+
+This project is open source and community driven. As such we encourage code contributions of all kinds. Some areas you can contribute in:
+
+1. Improve the stubs
+2. Sync stubs with the latest version of Django
+3. Improve plugin code and extend its capabilities
+4. Write tests
+5. Update dependencies
 
 ## Tutorials
 
-If you want to start working on this project,
-you will need to get familiar with these projects:
+If you want to start working on this project, you will need to get familiar with python typings.
+The Mypy documentation offers an excellent resource for this, as well as the python official documentation:
+
+- [Mypy typing documentation](https://mypy.readthedocs.io/en/stable/#overview-type-system-reference)
+- [Python official typing documentation](https://docs.python.org/3/library/typing.html)
+- [Typing in Python](https://inventwithpython.com/blog/2019/11/24/type-hints-for-busy-python-programmers/) article
+
+Additionally, the following resources might be useful:
 
-- [Django docs](https://docs.djangoproject.com/en/dev/)
-- [Typing in Python](https://inventwithpython.com/blog/2019/11/24/type-hints-for-busy-python-programmers/)
 - [How to write custom mypy plugins](https://mypy.readthedocs.io/en/stable/extending_mypy.html)
 - [Typechecking Django and DRF](https://sobolevn.me/2019/08/typechecking-django-and-drf) guide
 - [Testing mypy stubs, plugins, and types](https://sobolevn.me/2019/08/testing-mypy-types) guide
+- [Awesome Python Typing](https://github.com/typeddjango/awesome-python-typing) list
 
-It is also recommended to take a look at these resources:
-
-- [Awesome Python Typing](https://github.com/typeddjango/awesome-python-typing)
+## Dev setup
 
+### Repository Setup
 
-## Dev documentation
+As a first step you will need to fork this repository and clone your fork locally.
+In order to be able to continously sync your fork with the origin repository's master branch, you will need to set up an upstream master. To do so follow this [official github guide](https://docs.github.com/en/free-pro-team@latest/github/collaborating-with-issues-and-pull-requests/syncing-a-fork).
 
-TODO
+### Dependency Setup
 
+After your repository is setup you will then need to create and activate a git ignored virtual env, e.g.:
 
-## Dependencies
-
-We use `pip` to manage the dependencies.
+```bash
+python3 -m venv .venv
+source .venv/bin/activate
+```
 
-To install them you would need to activate your `virtualenv` and run `install` command:
+Then install the dev requirements:
 
 ```bash
 pip install -r ./dev-requirements.txt
 ```
 
+Finally, install the pre-commit hooks:
 
-## Tests and linters
-
-We use `mypy`, `pytest`, `flake8`, and `black` for quality control.
-Here's [how we run our CI](https://github.com/typeddjango/django-stubs/blob/master/.travis.yml).
+```bash
+pre-commit install
+```
 
-### Typechecking
+### Testing and Linting
 
-To run typechecking use:
+We use `mypy`, `pytest`, `flake8`, and `black` for quality control. All tools except pytest are executed using pre-commit when you make a commit.
+To ensure there are not formatting or typing issues in the entire repository you can run:
 
 ```bash
-mypy ./mypy_django_plugin
+pre-commit run --all-files
 ```
 
-### Testing
-
-There are unit tests and type-related tests.
+NOTE: This command will not only lint but also modify files - so make sure to commit whatever changes you've made before hand.
+You can also run pre-commit per file or for a specific path, simply replace "--all-files" with a target (see [this guide](https://codeburst.io/tool-your-django-project-pre-commit-hooks-e1799d84551f) for more info).
 
-To run unit tests:
+To execute the unit tests, simply run:
 
 ```bash
 pytest
 ```
 
-Type-related tests ensure that different Django versions do work correctly.
-To run type-related tests:
+We also test the stubs against the Django's own test suite. This is done in CI but you can also do this locally.
+To execute the script run:
 
 ```bash
-python ./scripts/typecheck_tests.py --django_version=2.2
-python ./scripts/typecheck_tests.py --django_version=3.0
+python ./scripts/typecheck_tests.py --django_version 3.0
 ```
 
-Currently we only support two Django versions.
 
-### Linting
+### Generating Stubs using Stubgen
 
-To run auto-formatting:
+The stubs are based on auto-generated code created by Mypy's stubgen tool (see: [the stubgen docs](https://mypy.readthedocs.io/en/stable/stubgen.html)).
+To make life easier we have a helper script that auto generates these stubs. To use it you can run:
 
 ```bash
-isort -rc .
-black django-stubs/
+python ./scripts/stubgen-django.py --django_version 3.1
 ```
 
-To run linting:
+You can also pass an optional commit hash as a second kwarg to checkout a specific commit, e.g.
 
 ```bash
-flake8
-flake8 --config flake8-pyi.ini
+python ./scripts/stubgen-django.py --django_version 3.1 --commit_sha <commit_sha>
 ```
 
+The output for this is a gitignored folder called "stubgen" in the repo's root.
 
-## Submitting your code
-
-We use [trunk based](https://trunkbaseddevelopment.com/)
-development (we also sometimes call it `wemake-git-flow`).
-
-What the point of this method?
-
-1. We use protected `master` branch,
-   so the only way to push your code is via pull request
-2. We use issue branches: to implement a new feature or to fix a bug
-   create a new branch named `issue-$TASKNUMBER`
-3. Then create a pull request to `master` branch
-4. We use `git tag`s to make releases, so we can track what has changed
-   since the latest release
-
-So, this way we achieve an easy and scalable development process
-which frees us from merging hell and long-living branches.
-
-In this method, the latest version of the app is always in the `master` branch.
-
+## Submission Guidelines
 
-## Other help
+The workflow for contributions is fairly simple:
 
-You can contribute by spreading a word about this library.
-It would also be a huge contribution to write
-a short article on how you are using this project.
-You can also share your best practices with us.
+1. fork and setup the repository as in the previous step.
+2. create a local branch.
+3. make whatever changes you want to contribute.
+4. ensure your contribution does not introduce linting issues or breaks the tests by linting and testing the code.
+5. make a pull request with an adequate description.

README.md

@@ -1,11 +1,12 @@
 <img src="http://mypy-lang.org/static/mypy_light.svg" alt="mypy logo" width="300px"/>
 
-# Typesafe Django Framework
+# pep484 stubs for Django
 
 [![Build Status](https://travis-ci.com/typeddjango/django-stubs.svg?branch=master)](https://travis-ci.com/typeddjango/django-stubs)
 [![Checked with mypy](http://www.mypy-lang.org/static/mypy_badge.svg)](http://mypy-lang.org/)
 [![Gitter](https://badges.gitter.im/mypy-django/Lobby.svg)](https://gitter.im/mypy-django/Lobby)
 
+
 This package contains [type stubs](https://www.python.org/dev/peps/pep-0561/) and a custom mypy plugin to provide more precise static types and type inference for Django framework. Django uses some Python "magic" that makes having precise types for some code patterns problematic. This is why we need this project. The final goal is to be able to get precise types for most common patterns.
 
 
@@ -15,18 +16,13 @@ This package contains [type stubs](https://www.python.org/dev/peps/pep-0561/) an
 pip install django-stubs
 ```
 
-See [Configuration](#configuration) section to get started.
-
-
-## Configuration
-
-To make `mypy` happy, you will need to add:
+To make mypy aware of the plugin, you need to add
 
 ```ini
 [mypy]
 plugins =
     mypy_django_plugin.main
-    
+
 [mypy.plugins.django-stubs]
 django_settings_module = "myproject.settings"
 ```
@@ -40,7 +36,6 @@ Two things happeining here:
 
 This fully working [typed boilerplate](https://github.com/wemake-services/wemake-django-template) can serve you as an example.
 
-
 ## Version compatibility
 
 We rely on different `django` and `mypy` versions:
@@ -73,15 +68,15 @@ But, it does not make any sense to use this project without `mypy`.
 
 ### mypy crashes when I run it with this plugin installed
 
-The current implementation uses Django runtime to extract models information, so it will crash, if your installed apps or `models.py` is not correct. For this same reason, you cannot use `reveal_type` inside global scope of any Python file that will be executed for `django.setup()`. 
+Current implementation uses Django runtime to extract models information, so it will crash, if your installed apps or `models.py` is not correct. For this same reason, you cannot use `reveal_type` inside global scope of any Python file that will be executed for `django.setup()`.
 
-In other words, if your `manage.py runserver` crashes, mypy will crash too. 
-You can also run `mypy` with the [`--tb`](https://mypy.readthedocs.io/en/stable/command_line.html#cmdoption-mypy-show-traceback)
-option to get extra information about errors.
+In other words, if your `manage.py runserver` crashes, mypy will crash too.
+You can also run `mypy` with [`--tb`](https://mypy.readthedocs.io/en/stable/command_line.html#cmdoption-mypy-show-traceback)
+option to get extra information about the error.
 
 ### I cannot use QuerySet or Manager with type annotations
 
-You can get a `TypeError: 'type' object is not subscriptable` 
+You can get a `TypeError: 'type' object is not subscriptable`
 when you will try to use `QuerySet[MyModel]` or `Manager[MyModel]`.
 
 This happens because Django classes do not support [`__class_getitem__`](https://www.python.org/dev/peps/pep-0560/#class-getitem) magic method.
@@ -122,5 +117,14 @@ And then use `AuthenticatedHttpRequest` instead of the standard `HttpRequest` fo
 ## To get help
 
 We have Gitter here: <https://gitter.im/mypy-django/Lobby>
+If you think you have more generic typing issue, please refer to <https://github.com/python/mypy> and their Gitter.
+
+## Contributing
+
+This project is open source and community driven. As such we encourage contributions big and small. You can contribute by doing any of the following:
+
+1. Contribute code (e.g. improve stubs, add plugin capabilities, write tests etc) - to do so please follow the [contribution guide](./CONTRIBUTING.md).
+2. Assist in code reviews and discussions in issues.
+3. Identify bugs and issues and report these
 
-If you think you have a more generic typing issue, please refer to <https://github.com/python/mypy> and their Gitter.
+You can always also reach out in gitter to discuss your contributions!