Skip to content

Commit 0c5935a

Browse files
feluellefeluelle
committed
Initial commit
- project initialization - set of first rules
1 parent 57c92bc commit 0c5935a

14 files changed

+3247
-0
lines changed

‎.bandit

+4
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
assert_used:
2+
skips:
3+
- 'airflint/rules/*.py'
4+
- 'tests/*.py'

‎.gitignore

+152
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,152 @@
1+
# Byte-compiled / optimized / DLL files
2+
__pycache__/
3+
*.py[cod]
4+
*$py.class
5+
6+
# C extensions
7+
*.so
8+
9+
# Distribution / packaging
10+
.Python
11+
build/
12+
develop-eggs/
13+
dist/
14+
downloads/
15+
eggs/
16+
.eggs/
17+
lib/
18+
lib64/
19+
parts/
20+
sdist/
21+
var/
22+
wheels/
23+
share/python-wheels/
24+
*.egg-info/
25+
.installed.cfg
26+
*.egg
27+
MANIFEST
28+
29+
# PyInstaller
30+
# Usually these files are written by a python script from a template
31+
# before PyInstaller builds the exe, so as to inject date/other infos into it.
32+
*.manifest
33+
*.spec
34+
35+
# Installer logs
36+
pip-log.txt
37+
pip-delete-this-directory.txt
38+
39+
# Unit test / coverage reports
40+
htmlcov/
41+
.tox/
42+
.nox/
43+
.coverage
44+
.coverage.*
45+
.cache
46+
nosetests.xml
47+
coverage.xml
48+
*.cover
49+
*.py,cover
50+
.hypothesis/
51+
.pytest_cache/
52+
cover/
53+
54+
# Translations
55+
*.mo
56+
*.pot
57+
58+
# Django stuff:
59+
*.log
60+
local_settings.py
61+
db.sqlite3
62+
db.sqlite3-journal
63+
64+
# Flask stuff:
65+
instance/
66+
.webassets-cache
67+
68+
# Scrapy stuff:
69+
.scrapy
70+
71+
# Sphinx documentation
72+
docs/_build/
73+
74+
# PyBuilder
75+
.pybuilder/
76+
target/
77+
78+
# Jupyter Notebook
79+
.ipynb_checkpoints
80+
81+
# IPython
82+
profile_default/
83+
ipython_config.py
84+
85+
# pyenv
86+
# For a library or package, you might want to ignore these files since the code is
87+
# intended to run in multiple environments; otherwise, check them in:
88+
# .python-version
89+
90+
# pipenv
91+
# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
92+
# However, in case of collaboration, if having platform-specific dependencies or dependencies
93+
# having no cross-platform support, pipenv may install dependencies that don't work, or not
94+
# install all needed dependencies.
95+
#Pipfile.lock
96+
97+
# poetry
98+
# Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.
99+
# This is especially recommended for binary packages to ensure reproducibility, and is more
100+
# commonly ignored for libraries.
101+
# https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control
102+
#poetry.lock
103+
104+
# PEP 582; used by e.g. github.com/David-OConnor/pyflow
105+
__pypackages__/
106+
107+
# Celery stuff
108+
celerybeat-schedule
109+
celerybeat.pid
110+
111+
# SageMath parsed files
112+
*.sage.py
113+
114+
# Environments
115+
.env
116+
.venv
117+
env/
118+
venv/
119+
ENV/
120+
env.bak/
121+
venv.bak/
122+
123+
# Spyder project settings
124+
.spyderproject
125+
.spyproject
126+
127+
# Rope project settings
128+
.ropeproject
129+
130+
# mkdocs documentation
131+
/site
132+
133+
# mypy
134+
.mypy_cache/
135+
.dmypy.json
136+
dmypy.json
137+
138+
# Pyre type checker
139+
.pyre/
140+
141+
# pytype static type analyzer
142+
.pytype/
143+
144+
# Cython debug symbols
145+
cython_debug/
146+
147+
# PyCharm
148+
# JetBrains specific template is maintained in a separate JetBrains.gitignore that can
149+
# be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
150+
# and can be added to the global gitignore or merged into this file. For a more nuclear
151+
# option (not recommended) you can uncomment the following to ignore the entire idea folder.
152+
#.idea/

‎.pre-commit-config.yaml

+99
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,99 @@
1+
default_language_version:
2+
python: python3.9
3+
exclude: examples/
4+
repos:
5+
- repo: meta
6+
hooks:
7+
- id: check-hooks-apply
8+
- repo: https://github.com/pre-commit/pre-commit-hooks
9+
rev: v4.1.0
10+
hooks:
11+
- id: check-ast
12+
- id: check-builtin-literals
13+
- id: check-docstring-first
14+
- id: check-toml
15+
- id: debug-statements
16+
- id: end-of-file-fixer
17+
- id: trailing-whitespace
18+
- repo: https://github.com/pre-commit/mirrors-mypy
19+
rev: v0.942
20+
hooks:
21+
- id: mypy
22+
additional_dependencies:
23+
- types-requests
24+
- types-toml
25+
- repo: https://github.com/pre-commit/pygrep-hooks
26+
rev: v1.9.0
27+
hooks:
28+
- id: python-check-blanket-noqa
29+
- id: python-check-blanket-type-ignore
30+
- id: python-check-mock-methods
31+
- id: python-no-eval
32+
- id: python-no-log-warn
33+
- id: python-use-type-annotations
34+
- repo: https://github.com/asottile/add-trailing-comma
35+
rev: v2.2.1
36+
hooks:
37+
- id: add-trailing-comma
38+
- repo: https://github.com/asottile/pyupgrade
39+
rev: v2.31.1
40+
hooks:
41+
- id: pyupgrade
42+
- repo: https://github.com/asottile/yesqa
43+
rev: v1.3.0
44+
hooks:
45+
- id: yesqa
46+
- repo: https://github.com/asottile/dead
47+
rev: v1.4.0
48+
hooks:
49+
- id: dead
50+
args: ["--exclude", "airflint/rules/|examples/|airflint/utils.py"]
51+
- repo: https://github.com/psf/black
52+
rev: 22.3.0
53+
hooks:
54+
- id: black
55+
- repo: https://github.com/PyCQA/flake8
56+
rev: 4.0.1
57+
hooks:
58+
- id: flake8
59+
args: ["--max-line-length", "88", "--extend-ignore", "E203,E501,B008"]
60+
additional_dependencies:
61+
- flake8-bugbear
62+
- flake8-builtins
63+
- repo: https://github.com/PyCQA/bandit
64+
rev: 1.7.4
65+
hooks:
66+
- id: bandit
67+
args: ["-c", ".bandit"]
68+
- repo: https://github.com/PyCQA/pydocstyle
69+
rev: 6.1.1
70+
hooks:
71+
- id: pydocstyle
72+
args: ["--convention", "pep257", "--add-ignore", "D100,D107,D102"]
73+
additional_dependencies:
74+
- toml
75+
exclude: airflint/__main__.py|tests/
76+
- repo: https://github.com/pycqa/isort
77+
rev: 5.10.1
78+
hooks:
79+
- id: isort
80+
args: ["--profile", "black"]
81+
- repo: https://github.com/crate-ci/typos
82+
rev: v1.5.0
83+
hooks:
84+
- id: typos
85+
- repo: https://github.com/pycqa/autoflake
86+
rev: v1.4
87+
hooks:
88+
- id: autoflake
89+
args: ["--remove-all-unused-imports", "--in-place"]
90+
- repo: local
91+
hooks:
92+
- id: pytest
93+
name: Run pytest
94+
entry: .venv/bin/pytest tests/
95+
language: script
96+
pass_filenames: false
97+
# alternatively you could `types: [python]` so it only runs when python files change
98+
# though tests might be invalidated if you were to say change a data file
99+
always_run: true

‎.pre-commit-hooks.yaml

+6
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
- id: airflint
2+
name: Airflint
3+
description: A linter for your Airflow DAGs to ensure Best Practices are being used.
4+
entry: airflint
5+
language: python
6+
types: [python]

‎README.md

+16
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
# airflint
2+
3+
> A linter for your Airflow DAGs to ensure Best Practices are being used.
4+
5+
## 🧑‍🏫 Rules
6+
7+
- [x] use datetime instead of days_ago in DAG start_date
8+
- [x] task decorator instead of PythonOperator and PythonVenvOperator
9+
- [x] dag decorator instead of DAG
10+
- [x] jinja string instead of Variable.get
11+
12+
> ⚠️ airflint does not remove imports. For removing unused imports please use [autoflake](https://github.com/PyCQA/autoflake) additionally.
13+
14+
## 💡 Future Ideas
15+
16+
- create refactorings for Airflow `DeprecationWarning`s

‎airflint/__init__.py

+8
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
"""Top-level package for airflint."""
2+
from importlib.metadata import version
3+
from os import getcwd
4+
from os.path import dirname, join, realpath
5+
6+
__app_name__ = "airflint"
7+
__version__ = version(__name__)
8+
__location__ = realpath(join(getcwd(), dirname(__file__)))

‎airflint/__main__.py

+16
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
from refactor import run
2+
3+
from airflint.rules import dag, task, variable
4+
5+
6+
def main():
7+
rules = []
8+
rules.extend(dag.EnforceStaticStartDate)
9+
rules.extend(dag.EnforceTaskFlowApi)
10+
rules.extend(task.EnforceTaskFlowApi)
11+
rules.append(variable.ReplaceVariableGetByJinja)
12+
run(rules=rules)
13+
14+
15+
if __name__ == "__main__":
16+
main()

0 commit comments

Comments
 (0)