chore: add ruff linter #532
Conversation
WalkthroughThe changes add the Python package Changes
Sequence Diagram(s)sequenceDiagram
actor Dev as Developer
participant CI as GitHub Actions
participant Env as Python Environment
Dev->>CI: Push code change
CI->>Env: Activate Python Environment
Env->>CI: Run "Run Chronon Python lint (flake8)"
CI->>Env: Run "Run Chronon Python lint (ruff)"
Env-->>CI: Return lint results
CI-->>Dev: Build complete
Possibly related PRs
Suggested reviewers
Poem
Warning Review ran into problems🔥 ProblemsGitHub Actions and Pipeline Checks: Resource not accessible by integration - https://docs.github.com/rest/actions/workflow-runs#list-workflow-runs-for-a-repository. Please grant the required permissions to the CodeRabbit GitHub App under the organization or repository settings. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
🔭 Outside diff range comments (1)
api/py/test/sample/staging_queries/sample_team/sample_staging_query.py (1)
38-40: 🛠️ Refactor suggestion
⚠️ Potential issueFix JSON formatting.
The JSON string intablePropertiesseems malformed. Consider changing:- "sample_config_json": """{"sample_key": "sample value}""", + "sample_config_json": """{"sample_key": "sample value"}""",
🧹 Nitpick comments (18)
api/py/test/sample/staging_queries/kaggle/outbrain.py (1)
16-16: Import order updated.
Reordered to "MetaData, StagingQuery" for consistency.api/py/test/sample/staging_queries/quickstart/checkouts_external.py (1)
15-15: Import order updated.
Ordering now matches other files.api/py/test/sample/staging_queries/sample_team/sample_staging_query.py (1)
15-15: Import order updated.
Now imports as "MetaData, StagingQuery".api/py/test/sample/group_bys/sample_team/chaining_group_by.py (1)
3-3: Avoid star import.
Consider explicit imports to mitigate potential namespace issues.🧰 Tools
🪛 Ruff (0.8.2)
3-3:
from ai.chronon.types import *used; unable to detect undefined names(F403)
api/py/ai/chronon/cli/compile/display/diff_result.py (1)
31-31: Typo detected. Consider renamingudpatedtoupdated.api/py/ai/chronon/cli/compile/compiler.py (2)
4-5: Refined Imports: Clear, explicit imports for dataclass and typing improve clarity.
11-12: Grouped Dependencies: Organizing CompileContext, ConfigInfo, and CompiledObj imports enhances consistency.api/py/ai/chronon/repo/compile.py (2)
31-32: Improved Grouping: Including theteamsimport within the grouped constants boosts readability.
37-39: Restored Import: Reintroducingget_join_output_columnsensures join processing remains complete.api/py/ai/chronon/repo/default_runner.py (1)
7-15: Constants Clarity: Listing each constant on its own line enhances import clarity.api/py/test/test_utils.py (1)
16-21: Test Import Tidy-Up: Prioritizingpytestand reorganizing module imports streamlines the test file.api/py/ai/chronon/repo/hub_uploader.py (1)
2-2: Explicit Hashlib: Clearly importinghashlibensures MD5 hashing works as intended.api/py/ai/chronon/repo/runner.py (1)
33-33: Consider keeping exception context.The
from Nonesuppresses the original exception context, making debugging harder.- ) from None + ).github/workflows/test_python.yaml (1)
55-55: Remove trailing whitespace.Line has unnecessary trailing spaces.
- +🧰 Tools
🪛 YAMLlint (1.35.1)
[error] 55-55: trailing spaces
(trailing-spaces)
api/py/pyproject.toml (1)
1-41: Good ruff configuration.The setup properly:
- Excludes appropriate directories
- Sets reasonable line length (100)
- Targets Python 3.9
- Selects essential linting rules
- Ignores sensible exceptions
- Disables specific checks for generated code
Consider adding comments explaining why certain rules are ignored.
api/py/ai/chronon/repo/aws.py (1)
59-59: Improved exception handling with chaining.Using
from epreserves the original exception traceback.api/py/ai/chronon/eval/sample_tables.py (1)
61-65: Concise Import Cleanup: Consider moving these imports to the module level to avoid repeated heavy imports.api/py/test/sample/joins/sample_team/sample_label_join_with_agg.py (1)
24-24: Reorder Import:test_sourcesshould come afterJoinimports per linter guidelines.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro (Legacy)
📒 Files selected for processing (106)
.github/image/Dockerfile(1 hunks).github/workflows/test_python.yaml(1 hunks)api/py/ai/chronon/cli/compile/compile_context.py(2 hunks)api/py/ai/chronon/cli/compile/compiler.py(1 hunks)api/py/ai/chronon/cli/compile/conf_validator.py(1 hunks)api/py/ai/chronon/cli/compile/display/class_tracker.py(2 hunks)api/py/ai/chronon/cli/compile/display/compile_status.py(1 hunks)api/py/ai/chronon/cli/compile/display/diff_result.py(1 hunks)api/py/ai/chronon/cli/compile/parse_configs.py(1 hunks)api/py/ai/chronon/cli/compile/parse_teams.py(1 hunks)api/py/ai/chronon/cli/compile/serializer.py(1 hunks)api/py/ai/chronon/cli/entrypoint.py(1 hunks)api/py/ai/chronon/cli/git_utils.py(3 hunks)api/py/ai/chronon/cli/plan/controller_iface.py(0 hunks)api/py/ai/chronon/cli/plan/physical_index.py(1 hunks)api/py/ai/chronon/cli/plan/physical_node.py(0 hunks)api/py/ai/chronon/eval/__init__.py(1 hunks)api/py/ai/chronon/eval/sample_tables.py(2 hunks)api/py/ai/chronon/eval/table_scan.py(1 hunks)api/py/ai/chronon/group_by.py(2 hunks)api/py/ai/chronon/join.py(1 hunks)api/py/ai/chronon/model.py(1 hunks)api/py/ai/chronon/query.py(2 hunks)api/py/ai/chronon/repo/__init__.py(1 hunks)api/py/ai/chronon/repo/aws.py(3 hunks)api/py/ai/chronon/repo/compile.py(1 hunks)api/py/ai/chronon/repo/compilev2.py(1 hunks)api/py/ai/chronon/repo/default_runner.py(1 hunks)api/py/ai/chronon/repo/explore.py(2 hunks)api/py/ai/chronon/repo/gcp.py(4 hunks)api/py/ai/chronon/repo/hub_uploader.py(1 hunks)api/py/ai/chronon/repo/init.py(1 hunks)api/py/ai/chronon/repo/run.py(1 hunks)api/py/ai/chronon/repo/runner.py(2 hunks)api/py/ai/chronon/repo/serializer.py(1 hunks)api/py/ai/chronon/repo/utils.py(1 hunks)api/py/ai/chronon/repo/validator.py(2 hunks)api/py/ai/chronon/repo/zipline.py(1 hunks)api/py/ai/chronon/types.py(1 hunks)api/py/ai/chronon/utils.py(1 hunks)api/py/ai/chronon/windows.py(1 hunks)api/py/pyproject.toml(1 hunks)api/py/setup.py(1 hunks)api/py/test/canary/group_bys/aws/plaid_fv.py(1 hunks)api/py/test/canary/group_bys/aws/purchases.py(1 hunks)api/py/test/canary/group_bys/gcp/purchases.py(1 hunks)api/py/test/conftest.py(1 hunks)api/py/test/sample/group_bys/kaggle/clicks.py(1 hunks)api/py/test/sample/group_bys/kaggle/outbrain.py(1 hunks)api/py/test/sample/group_bys/quickstart/purchases.py(1 hunks)api/py/test/sample/group_bys/quickstart/returns.py(1 hunks)api/py/test/sample/group_bys/quickstart/schema.py(1 hunks)api/py/test/sample/group_bys/quickstart/users.py(1 hunks)api/py/test/sample/group_bys/risk/merchant_data.py(1 hunks)api/py/test/sample/group_bys/risk/transaction_events.py(1 hunks)api/py/test/sample/group_bys/risk/user_data.py(1 hunks)api/py/test/sample/group_bys/sample_team/chaining_group_by.py(1 hunks)api/py/test/sample/group_bys/sample_team/entity_sample_group_by_from_module.py(1 hunks)api/py/test/sample/group_bys/sample_team/event_sample_group_by.py(1 hunks)api/py/test/sample/group_bys/sample_team/group_by_with_kwargs.py(1 hunks)api/py/test/sample/group_bys/sample_team/label_part_group_by.py(1 hunks)api/py/test/sample/group_bys/sample_team/mutation_sample_group_by.py(1 hunks)api/py/test/sample/group_bys/sample_team/sample_chaining_group_by.py(1 hunks)api/py/test/sample/group_bys/sample_team/sample_group_by.py(1 hunks)api/py/test/sample/group_bys/sample_team/sample_group_by_from_join_part.py(1 hunks)api/py/test/sample/group_bys/sample_team/sample_group_by_from_module.py(1 hunks)api/py/test/sample/group_bys/sample_team/sample_group_by_group_by.py(1 hunks)api/py/test/sample/group_bys/sample_team/sample_group_by_missing_input_column.py(1 hunks)api/py/test/sample/group_bys/sample_team/sample_group_by_with_derivations.py(1 hunks)api/py/test/sample/group_bys/sample_team/sample_group_by_with_incorrect_derivations.py(1 hunks)api/py/test/sample/group_bys/sample_team/sample_non_prod_group_by.py(1 hunks)api/py/test/sample/joins/kaggle/outbrain.py(1 hunks)api/py/test/sample/joins/quickstart/training_set.py(1 hunks)api/py/test/sample/joins/risk/user_transactions.py(1 hunks)api/py/test/sample/joins/sample_team/sample_backfill_mutation_join.py(1 hunks)api/py/test/sample/joins/sample_team/sample_chaining_join.py(1 hunks)api/py/test/sample/joins/sample_team/sample_chaining_join_parent.py(1 hunks)api/py/test/sample/joins/sample_team/sample_join.py(1 hunks)api/py/test/sample/joins/sample_team/sample_join_bootstrap.py(1 hunks)api/py/test/sample/joins/sample_team/sample_join_derivation.py(1 hunks)api/py/test/sample/joins/sample_team/sample_join_external_parts.py(1 hunks)api/py/test/sample/joins/sample_team/sample_join_from_group_by_from_join.py(1 hunks)api/py/test/sample/joins/sample_team/sample_join_from_module.py(1 hunks)api/py/test/sample/joins/sample_team/sample_join_from_module_skipped.py(1 hunks)api/py/test/sample/joins/sample_team/sample_join_with_derivations_on_external_parts.py(1 hunks)api/py/test/sample/joins/sample_team/sample_label_join.py(1 hunks)api/py/test/sample/joins/sample_team/sample_label_join_with_agg.py(1 hunks)api/py/test/sample/joins/sample_team/sample_online_join.py(1 hunks)api/py/test/sample/models/quickstart/test.py(1 hunks)api/py/test/sample/models/risk/transaction_model.py(1 hunks)api/py/test/sample/scripts/fetch_online_jar.py(1 hunks)api/py/test/sample/scripts/yarn_list.py(0 hunks)api/py/test/sample/sources/kaggle/outbrain.py(1 hunks)api/py/test/sample/sources/test_sources.py(1 hunks)api/py/test/sample/staging_queries/kaggle/outbrain.py(1 hunks)api/py/test/sample/staging_queries/quickstart/checkouts_external.py(1 hunks)api/py/test/sample/staging_queries/sample_team/sample_staging_query.py(1 hunks)api/py/test/sample/teams.py(0 hunks)api/py/test/test_compile.py(1 hunks)api/py/test/test_explore.py(1 hunks)api/py/test/test_git_utils.py(1 hunks)api/py/test/test_group_by.py(3 hunks)api/py/test/test_join.py(0 hunks)api/py/test/test_run.py(1 hunks)api/py/test/test_teams.py(1 hunks)api/py/test/test_utils.py(1 hunks)
💤 Files with no reviewable changes (5)
- api/py/test/sample/teams.py
- api/py/test/sample/scripts/yarn_list.py
- api/py/ai/chronon/cli/plan/controller_iface.py
- api/py/ai/chronon/cli/plan/physical_node.py
- api/py/test/test_join.py
🧰 Additional context used
🧬 Code Definitions (46)
api/py/test/sample/group_bys/sample_team/event_sample_group_by.py (2)
api/py/ai/chronon/group_by.py (3) (3)
Aggregation(193-241)GroupBy(429-673)Operation(60-146)api/src/main/scala/ai/chronon/api/Builders.scala (2) (2)
Aggregation(87-104)GroupBy(142-168)
api/py/test/sample/group_bys/risk/merchant_data.py (3)
api/py/ai/chronon/source.py (1) (1)
EntitySource(38-71)api/py/ai/chronon/group_by.py (1) (1)
GroupBy(429-673)api/py/ai/chronon/query.py (2) (2)
Query(20-84)selects(87-106)
api/py/test/sample/joins/sample_team/sample_backfill_mutation_join.py (1)
api/py/ai/chronon/join.py (2) (2)
Join(397-596)JoinPart(30-108)
api/py/ai/chronon/cli/compile/compiler.py (2)
api/py/ai/chronon/cli/compile/compile_context.py (2) (2)
CompileContext(26-167)ConfigInfo(19-22)api/py/ai/chronon/cli/compile/display/compiled_obj.py (1) (1)
CompiledObj(6-12)
api/py/test/sample/group_bys/sample_team/entity_sample_group_by_from_module.py (1)
api/py/ai/chronon/group_by.py (3) (3)
Aggregation(193-241)GroupBy(429-673)Operation(60-146)
api/py/test/sample/sources/test_sources.py (1)
api/py/ai/chronon/query.py (2) (2)
Query(20-84)selects(87-106)
api/py/ai/chronon/repo/compile.py (2)
api/py/ai/chronon/repo/serializer.py (1) (1)
thrift_simple_json_protected(127-141)api/py/ai/chronon/cli/compile/conf_validator.py (2) (2)
get_group_by_output_columns(83-91)get_join_output_columns(169-179)
api/py/test/sample/group_bys/sample_team/group_by_with_kwargs.py (2)
api/py/ai/chronon/group_by.py (2) (2)
Aggregation(193-241)GroupBy(429-673)api/src/main/scala/ai/chronon/api/Builders.scala (2) (2)
Aggregation(87-104)GroupBy(142-168)
api/py/test/sample/models/risk/transaction_model.py (2)
api/py/ai/chronon/model.py (2) (2)
Model(12-35)ModelType(6-8)api/py/ai/chronon/query.py (2) (2)
Query(20-84)selects(87-106)
api/py/test/sample/staging_queries/quickstart/checkouts_external.py (1)
spark/src/main/scala/ai/chronon/spark/StagingQuery.scala (2) (2)
StagingQuery(29-106)StagingQuery(108-145)
api/py/test/sample/joins/kaggle/outbrain.py (2)
api/py/test/sample/sources/kaggle/outbrain.py (1) (1)
outbrain_left_events(28-40)api/py/ai/chronon/join.py (2) (2)
Join(397-596)JoinPart(30-108)
api/py/test/test_group_by.py (3)
api/py/ai/chronon/group_by.py (1) (1)
GroupBy(429-673)api/py/test/test_utils.py (1) (1)
event_source(36-41)api/py/test/test_join.py (1) (1)
event_source(18-34)
api/py/test/sample/group_bys/sample_team/mutation_sample_group_by.py (1)
api/py/ai/chronon/group_by.py (3) (3)
Accuracy(56-57)Aggregation(193-241)GroupBy(429-673)
api/py/test/sample/joins/quickstart/training_set.py (3)
api/py/ai/chronon/source.py (1) (1)
EventSource(8-35)api/py/ai/chronon/join.py (2) (2)
Join(397-596)JoinPart(30-108)api/py/ai/chronon/query.py (2) (2)
Query(20-84)selects(87-106)
api/py/test/sample/group_bys/sample_team/sample_group_by_from_module.py (1)
api/py/ai/chronon/group_by.py (2) (2)
Aggregation(193-241)GroupBy(429-673)
api/py/test/sample/joins/sample_team/sample_label_join_with_agg.py (2)
api/py/ai/chronon/join.py (3) (3)
Join(397-596)JoinPart(30-108)LabelParts(252-329)api/src/main/scala/ai/chronon/api/Builders.scala (2) (2)
Join(170-199)JoinPart(246-259)
api/py/ai/chronon/repo/__init__.py (3)
api/py/ai/chronon/group_by.py (1) (1)
GroupBy(429-673)api/py/ai/chronon/join.py (1) (1)
Join(397-596)api/py/ai/chronon/model.py (1) (1)
Model(12-35)
api/py/ai/chronon/repo/compilev2.py (2)
api/py/ai/chronon/repo/serializer.py (1) (1)
thrift_simple_json_protected(127-141)api/py/ai/chronon/cli/compile/conf_validator.py (2) (2)
get_group_by_output_columns(83-91)get_join_output_columns(169-179)
api/py/test/sample/group_bys/sample_team/sample_group_by_with_incorrect_derivations.py (1)
api/py/ai/chronon/join.py (1) (1)
Derivation(332-356)
api/py/test/test_git_utils.py (1)
api/py/ai/chronon/cli/git_utils.py (3) (3)
get_changes_since_commit(77-127)get_changes_since_fork(130-156)get_current_branch(11-41)
api/py/test/test_utils.py (2)
api/py/ai/chronon/cli/compile/serializer.py (2) (2)
file2thrift(99-107)json2thrift(86-87)api/py/ai/chronon/repo/serializer.py (2) (2)
file2thrift(104-112)json2thrift(91-92)
api/py/test/test_run.py (3)
api/py/ai/chronon/repo/default_runner.py (1) (1)
run(148-223)api/py/ai/chronon/repo/gcp.py (1) (1)
run(282-410)api/py/ai/chronon/repo/aws.py (1) (1)
run(202-295)
api/py/test/sample/group_bys/quickstart/users.py (2)
api/py/ai/chronon/source.py (1) (1)
EntitySource(38-71)api/py/ai/chronon/query.py (2) (2)
Query(20-84)selects(87-106)
api/py/test/sample/group_bys/risk/transaction_events.py (2)
api/py/ai/chronon/source.py (1) (1)
EventSource(8-35)api/py/ai/chronon/group_by.py (3) (3)
Aggregation(193-241)GroupBy(429-673)Operation(60-146)
api/py/test/sample/group_bys/quickstart/returns.py (3)
api/py/ai/chronon/source.py (1) (1)
EventSource(8-35)api/py/ai/chronon/group_by.py (3) (3)
Aggregation(193-241)GroupBy(429-673)Operation(60-146)api/py/ai/chronon/query.py (2) (2)
Query(20-84)selects(87-106)
api/py/test/sample/group_bys/sample_team/sample_group_by_from_join_part.py (1)
api/py/ai/chronon/group_by.py (3) (3)
Accuracy(56-57)Aggregation(193-241)GroupBy(429-673)
api/py/test/sample/joins/sample_team/sample_join_derivation.py (1)
api/py/ai/chronon/join.py (3) (3)
Derivation(332-356)Join(397-596)JoinPart(30-108)
api/py/test/sample/group_bys/sample_team/sample_group_by_with_derivations.py (1)
api/py/ai/chronon/join.py (1) (1)
Derivation(332-356)
api/py/test/sample/joins/sample_team/sample_join_from_module.py (2)
api/py/ai/chronon/join.py (2) (2)
Join(397-596)JoinPart(30-108)api/src/main/scala/ai/chronon/api/Builders.scala (2) (2)
Join(170-199)JoinPart(246-259)
api/py/test/sample/group_bys/sample_team/sample_non_prod_group_by.py (1)
api/py/ai/chronon/group_by.py (2) (2)
Aggregation(193-241)GroupBy(429-673)
api/py/test/sample/group_bys/quickstart/schema.py (2)
api/py/ai/chronon/source.py (1) (1)
EventSource(8-35)api/py/ai/chronon/group_by.py (3) (3)
Aggregation(193-241)GroupBy(429-673)Operation(60-146)
api/py/test/sample/models/quickstart/test.py (1)
api/py/ai/chronon/model.py (2) (2)
Model(12-35)ModelType(6-8)
api/py/test/sample/group_bys/quickstart/purchases.py (2)
api/py/ai/chronon/source.py (1) (1)
EventSource(8-35)api/py/ai/chronon/group_by.py (3) (3)
Aggregation(193-241)GroupBy(429-673)Operation(60-146)
api/py/test/sample/joins/sample_team/sample_join_external_parts.py (1)
api/py/ai/chronon/join.py (6) (6)
ContextualSource(209-218)DataType(114-155)ExternalPart(221-249)ExternalSource(158-206)Join(397-596)JoinPart(30-108)
api/py/ai/chronon/repo/validator.py (1)
api/py/ai/chronon/group_by.py (3) (3)
Derivation(248-259)GroupBy(429-673)get_output_col_names(408-426)
api/py/ai/chronon/utils.py (1)
orchestration/src/main/scala/ai/chronon/orchestration/RepoIndex.scala (1) (1)
diff(173-187)
api/py/ai/chronon/repo/serializer.py (1)
api/py/ai/chronon/utils.py (1) (1)
JsonDiffer(60-87)
api/py/ai/chronon/cli/compile/conf_validator.py (1)
api/py/ai/chronon/group_by.py (2) (2)
Aggregation(193-241)Derivation(248-259)
api/py/test/sample/group_bys/kaggle/outbrain.py (2)
api/py/test/sample/sources/kaggle/outbrain.py (1) (1)
outbrain_left_events(28-40)api/py/ai/chronon/group_by.py (3) (3)
Accuracy(56-57)Aggregation(193-241)GroupBy(429-673)
api/py/ai/chronon/cli/compile/compile_context.py (6)
api/py/ai/chronon/cli/compile/compiler.py (1) (1)
compile(29-57)api/py/ai/chronon/cli/compile/conf_validator.py (1) (1)
ConfValidator(214-514)api/py/ai/chronon/cli/compile/display/compile_status.py (1) (1)
CompileStatus(11-77)api/py/ai/chronon/cli/compile/display/compiled_obj.py (1) (1)
CompiledObj(6-12)api/py/ai/chronon/cli/compile/serializer.py (1) (1)
file2thrift(99-107)api/py/ai/chronon/cli/logger.py (1) (1)
get_logger(32-43)
api/py/test/sample/staging_queries/sample_team/sample_staging_query.py (2)
api/src/main/scala/ai/chronon/api/Builders.scala (2) (2)
MetaData(261-315)StagingQuery(317-335)spark/src/main/scala/ai/chronon/spark/StagingQuery.scala (2) (2)
StagingQuery(29-106)StagingQuery(108-145)
api/py/test/sample/joins/sample_team/sample_join_with_derivations_on_external_parts.py (1)
api/py/ai/chronon/join.py (7) (7)
ContextualSource(209-218)DataType(114-155)Derivation(332-356)ExternalPart(221-249)ExternalSource(158-206)Join(397-596)JoinPart(30-108)
api/py/test/test_explore.py (1)
api/py/test/conftest.py (1) (1)
repo(32-33)
api/py/test/sample/group_bys/sample_team/sample_chaining_group_by.py (3)
api/py/ai/chronon/group_by.py (4) (4)
Accuracy(56-57)Aggregation(193-241)GroupBy(429-673)Operation(60-146)api/py/ai/chronon/join.py (2) (2)
Join(397-596)JoinPart(30-108)api/py/ai/chronon/source.py (1) (1)
JoinSource(74-88)
api/py/test/sample/group_bys/risk/user_data.py (3)
api/py/ai/chronon/source.py (1) (1)
EntitySource(38-71)api/py/ai/chronon/group_by.py (1) (1)
GroupBy(429-673)api/py/ai/chronon/query.py (2) (2)
Query(20-84)selects(87-106)
api/py/ai/chronon/repo/run.py (4)
api/py/ai/chronon/repo/aws.py (1) (1)
AwsRunner(31-295)api/py/ai/chronon/repo/default_runner.py (1) (1)
Runner(18-242)api/py/ai/chronon/repo/gcp.py (1) (1)
GcpRunner(31-410)api/py/ai/chronon/repo/utils.py (1) (1)
get_environ_arg(47-51)
🪛 Ruff (0.8.2)
api/py/test/sample/sources/test_sources.py
21-21: from ai.chronon.types import * used; unable to detect undefined names
(F403)
api/py/test/sample/group_bys/sample_team/chaining_group_by.py
3-3: from ai.chronon.types import * used; unable to detect undefined names
(F403)
🪛 YAMLlint (1.35.1)
.github/workflows/test_python.yaml
[error] 55-55: trailing spaces
(trailing-spaces)
⏰ Context from checks skipped due to timeout of 90000ms (2)
- GitHub Check: non_spark_tests
- GitHub Check: non_spark_tests
🔇 Additional comments (122)
api/py/ai/chronon/query.py (2)
15-16: Import reorganization looks good.Properly organized imports improve readability.
26-26: Good change to default parameter.Using
Noneinstead of[]avoids the mutable default argument issue - a Python best practice.api/py/test/sample/group_bys/risk/user_data.py (1)
1-3: Imports reordered.api/py/test/sample/group_bys/risk/merchant_data.py (1)
1-3: Imports reordered.api/py/test/sample/group_bys/quickstart/schema.py (1)
1-2: Order improved.api/py/test/sample/joins/quickstart/training_set.py (1)
19-21: Import order fixed.api/py/test/sample/group_bys/quickstart/users.py (1)
15-19: Imports reorganized.api/py/test/sample/joins/kaggle/outbrain.py (1)
16-20: Clean import reorder.
Reordered and added imports for clarity.api/py/test/canary/group_bys/aws/purchases.py (1)
1-2: Standardized imports.
Imports now follow a consistent order.api/py/test/sample/joins/risk/user_transactions.py (1)
1-5: Consistent import order.
Reordered risk module imports for uniformity.api/py/test/sample/group_bys/kaggle/clicks.py (1)
15-24: Refined import order.
Adjusted order and added staging query import; looks good.api/py/test/canary/group_bys/aws/plaid_fv.py (1)
1-2: Clear reordering.
Updated imports align with style guidelines.api/py/test/sample/group_bys/risk/transaction_events.py (1)
1-2: Imports reordered. No functionality change.api/py/test/canary/group_bys/gcp/purchases.py (1)
1-2: Reorganized imports. Looks good.api/py/test/sample/group_bys/quickstart/returns.py (1)
15-20: Updated imports. Active GroupBy import now in use.api/py/test/sample/sources/kaggle/outbrain.py (1)
15-16: Added staging query import and reordered. Clear and concise.api/py/test/sample/group_bys/quickstart/purchases.py (1)
15-16: Import order fixed. No logic changes.api/py/test/sample/group_bys/kaggle/outbrain.py (2)
15-16: New import added.
Addsoutbrain_left_eventsat the top for clarity.
17-23: Reordered group_by imports.
The updated order (Accuracy, Aggregation, GroupBy, Operation) improves readability.api/py/test/sample/joins/sample_team/sample_join_external_parts.py (1)
22-22: Consolidated imports look good.api/py/test/sample/joins/sample_team/sample_join_with_derivations_on_external_parts.py (2)
19-23: Group_bys Import Reorder:
Reordered and re-addedevent_sample_group_by.
25-33: Join Module Imports Reorder:
Adjusted order ofContextualSource,DataType,Derivation,Join, andJoinPartfor clarity.api/py/ai/chronon/eval/__init__.py (1)
1-5: Concise Import Order Improvement.
Reordered external imports to appear at the top. This enhances clarity without changing functionality.api/py/ai/chronon/cli/plan/physical_index.py (1)
9-9: Reintroduced lineage imports.
Restores Column and ColumnLineage for proper column_lineage mapping.api/py/test/sample/group_bys/sample_team/label_part_group_by.py (1)
2-2: Blank line added for clarity.
Improves readability by separating imports from code.api/py/test/conftest.py (1)
18-19: Pytest import reordered.
Positioningimport pytestafterosimproves import organization.api/py/ai/chronon/model.py (1)
3-4: Restored ttypes import.
Uncommentingai.chronon.api.ttypes as ttypesensures proper type definitions.api/py/ai/chronon/cli/compile/parse_configs.py (1)
7-8: Improved import order.
ReorderingCompiledObjimport afterCompileContextenhances clarity.api/py/ai/chronon/repo/zipline.py (1)
5-5: Import reorder accepted.api/py/test/sample/scripts/fetch_online_jar.py (1)
24-25: Import order reorg OK.api/py/test/sample/models/risk/transaction_model.py (1)
3-5: Import order tidy.api/py/ai/chronon/repo/init.py (1)
4-7: Import order refactored.api/py/test/sample/group_bys/sample_team/event_sample_group_by.py (1)
17-17: Import order updated.api/py/ai/chronon/repo/__init__.py (1)
15-15: Import order updated. Non‐functional change.api/py/test/sample/group_bys/sample_team/sample_group_by_missing_input_column.py (1)
17-20: Reordered imports. No functional impact.api/py/setup.py (1)
15-17: Addedglobimport. Import order is improved.api/py/ai/chronon/cli/compile/display/diff_result.py (1)
1-2: Clean imports. Unused ones removed.api/py/ai/chronon/join.py (1)
19-26: Reordered imports. Clean and consistent.api/py/ai/chronon/repo/utils.py (2)
8-8: Import organization improved.Consolidated datetime imports into a single line for cleaner code.
11-15: Import order standardized.Constants from
ai.chronon.repo.constantsreordered for consistency.api/py/test/sample/sources/test_sources.py (2)
15-16: Import reorganization looks good.Moved the staging query import to the top for better organization.
22-22: Utility import properly positioned.Moved
get_staging_query_output_table_nameimport further down in the import block.api/py/test/sample/models/quickstart/test.py (1)
2-2: Successfully activated required imports.Uncommented import of
ModelandModelTypewhich are needed for line 19-21.api/py/ai/chronon/cli/compile/parse_teams.py (2)
3-5: Import organization improved.Moved
deepcopyimport and added proper typing imports to enhance code clarity.
6-10: Added missing import.Activated
ExecutionInfoimport which is used on line 74.api/py/ai/chronon/eval/table_scan.py (1)
4-6: Imports reordered.api/py/test/sample/group_bys/sample_team/mutation_sample_group_by.py (1)
15-21: Import order adjusted.api/py/test/sample/joins/sample_team/sample_label_join.py (1)
19-25: Imports reorganized; redundant import removed.api/py/ai/chronon/cli/compile/conf_validator.py (1)
21-29: Clear import reordering.api/py/ai/chronon/repo/serializer.py (1)
16-27: Reordered imports. Clean and clear structure.api/py/ai/chronon/cli/compile/serializer.py (1)
18-21: Import order optimized. No functional impact.api/py/ai/chronon/cli/compile/display/compile_status.py (1)
1-8: Cleaned imports. OrderedDict added and unused types removed.api/py/test/sample/group_bys/sample_team/sample_group_by_from_join_part.py (1)
15-23: Reorganized imports. Order is now consistent.api/py/test/sample/group_bys/sample_team/sample_group_by_with_incorrect_derivations.py (1)
18-18: Import formatting updated. Single-line order set.api/py/test/sample/group_bys/sample_team/sample_group_by_group_by.py (1)
17-20: Import statements reorganized.Import statements are now better organized with standard imports before package-specific imports. Imports from
ai.chronon.group_byare alphabetically sorted.api/py/ai/chronon/types.py (1)
5-9: Import statements refactored.Package imports are now consistently organized with clear namespace aliasing.
api/py/ai/chronon/repo/validator.py (2)
22-29: Import statements reorganized.Imports are now organized in a more maintainable pattern with a dedicated typing imports section and alphabetized class names.
54-54: Unused variable now properly prefixed.Renamed unused variable
sub_dirsto_sub_dirsto indicate it's intentionally unused.api/py/ai/chronon/group_by.py (2)
17-24: Import statements organized for better readability.Standard library imports are now properly separated from third-party imports and organized consistently.
363-363: Improved error handling with exception chaining.Added exception chaining with
from eto preserve the original exception context, following Python best practices.api/py/ai/chronon/utils.py (1)
66-68: Fix mutable default argument issue.Replacing empty list default with None and initializing in the method body addresses the mutable default argument problem.
api/py/ai/chronon/cli/compile/display/class_tracker.py (2)
2-2: Import ordering adjusted.Alphabetical ordering of type imports.
83-83: Removed f-string prefix.Removed unnecessary f-string prefix from string without formatting.
api/py/test/sample/group_bys/sample_team/sample_chaining_group_by.py (2)
19-22: Added event_sample_group_by import.Uncommented/added import for required module.
25-33: Consolidated type imports.Grouped related type imports from ai.chronon.types into a single import block.
api/py/test/sample/joins/sample_team/sample_chaining_join.py (1)
20-20: Reorganized import.Cleaned up import statement formatting.
api/py/test/sample/group_bys/sample_team/entity_sample_group_by_from_module.py (1)
21-22: Import order is correct.api/py/test/sample/group_bys/sample_team/sample_group_by.py (1)
17-18: Import order updated, looks good.api/py/test/sample/joins/sample_team/sample_join_from_module_skipped.py (1)
20-21: Import adjustment is fine.api/py/test/test_teams.py (1)
18-19: Import order update is safe.api/py/test/sample/joins/sample_team/sample_backfill_mutation_join.py (1)
20-23: Consistent import reordering, approved.api/py/ai/chronon/windows.py (1)
49-49: Improved error handling with exception chainingGood update to maintain original exception context.
api/py/test/sample/joins/sample_team/sample_join_from_group_by_from_join.py (1)
16-16: Import reorderingRuff-standard import arrangement.
api/py/test/sample/joins/sample_team/sample_join.py (1)
16-22: Import reorderingImports correctly reordered per ruff standard.
api/py/test/test_git_utils.py (1)
4-10: Fixed importsImport order improved and added missing
get_current_branchimport that's used on lines 38 and 50..github/image/Dockerfile (1)
59-59: LGTM: Added ruff linter packageAdding ruff alongside existing testing/linting tools matches PR objective.
api/py/test/sample/group_bys/sample_team/group_by_with_kwargs.py (1)
21-25: LGTM: Properly added necessary importGroupBy import added correctly as it's used at line 27.
api/py/ai/chronon/cli/entrypoint.py (2)
1-1: LGTM: Explicit datetime importsProperly moved datetime imports to top level.
7-9: LGTM: Reorganized importsProperly organized imports for better readability.
api/py/ai/chronon/repo/compilev2.py (2)
26-30: LGTM: Added necessary importAdded teams to import group where it belongs.
32-36: LGTM: Restored necessary importRestored get_join_output_columns import which is used in show_lineage_and_schema function.
api/py/test/test_compile.py (1)
21-21: Import restored correctly.This import is necessary for the tests to execute properly.
api/py/test/sample/group_bys/sample_team/sample_group_by_from_module.py (1)
23-23: Import enabled correctly.GroupBy import is necessary for line 27 instantiation.
api/py/ai/chronon/repo/runner.py (1)
3-3: Required import added.The os module is used throughout the file for path operations.
api/py/test/sample/joins/sample_team/sample_join_from_module.py (2)
21-21: Import uncommented correctly.This import is required for line 31 where sample_group_by_from_module.v1 is used.
23-25: Import organization improved.Import reorganization maintains proper functionality while improving readability.
api/py/test/sample/group_bys/sample_team/sample_non_prod_group_by.py (1)
16-21: Uncommented GroupBy import - good catch.Activating the previously commented GroupBy import aligns with its usage on line 23.
.github/workflows/test_python.yaml (2)
47-47: Renamed step for clarity.Good clarification that this step uses flake8 specifically.
56-62: Added ruff linter step.Adds new linting capability alongside existing flake8 checks.
api/py/test/test_group_by.py (5)
16-19: Clean import organization.Improves readability with better import structure.
147-156: Simplified test code.Directly calling GroupBy instead of using an intermediate variable is cleaner.
158-162: Consistent approach to testing exceptions.Maintains consistency with direct function calls in pytest.raises contexts.
164-168: Simplified test pattern continued.Consistent with the direct function call pattern throughout tests.
169-173: Final case adjustment.Follows same pattern of direct function calls for consistency.
api/py/ai/chronon/cli/git_utils.py (3)
4-4: Improved import specificity.Removed unused
Setfrom typing import.
33-33: Better exception handling.Changed bare
excepttoexcept Exceptionto avoid catching system-exiting exceptions.
73-73: Better exception handling.Changed bare
excepttoexcept Exceptionto avoid catching system-exiting exceptions.api/py/test/sample/joins/sample_team/sample_join_derivation.py (2)
19-23: Improved import organization.Reordered imports for better readability.
25-25: Alphabetized imports.Reordered imports alphabetically (Derivation, Join, JoinPart).
api/py/test/sample/joins/sample_team/sample_chaining_join_parent.py (1)
1-5: Improved import organization.Added
event_sample_group_byand reordered imports.api/py/test/test_run.py (1)
23-26: Improved import organization.Moved
clickimport and consolidated repo imports on a single line.api/py/ai/chronon/cli/compile/compile_context.py (2)
2-12: Import reorganization looks good.Imports are now better organized for clarity.
131-131: Good use of underscore prefix for unused variable.Properly indicating that
_sub_dirsis an unused variable fromos.walk().api/py/ai/chronon/repo/explore.py (1)
22-23: Import organization improved.Standard library imports now properly positioned at the top.
api/py/ai/chronon/repo/aws.py (1)
13-15: Imports reorganized.Added missing
check_callimport and reordered imports.api/py/test/test_explore.py (1)
19-31: Import organization improved.Standard library imports moved to the top and import order standardized.
api/py/test/sample/joins/sample_team/sample_join_bootstrap.py (3)
19-23: Import reorganization looks good.The imported modules are now properly organized with
event_sample_group_byincluded, which is used in the code.
24-24: Import moved to its own line.Clean separation of import sources.
26-26: Consolidated type imports.All required types are now imported in a single alphabetized statement.
api/py/test/sample/joins/sample_team/sample_online_join.py (2)
19-23: Import reorganization looks good.The
event_sample_group_byimport is now properly included, which is used in the code.
24-24: Import moved to its own line.Clean separation of import sources.
api/py/ai/chronon/repo/run.py (5)
23-24: Moved import to separate section.Good separation between standard library and third-party imports.
25-29: Uncommented AwsRunner import.AwsRunner is used in the code, so it should be properly imported.
30-40: Reorganized constants imports.Cleaner organization of imports from constants module.
41-46: Uncommented GcpRunner import.GcpRunner is used in the code, so it should be properly imported.
47-47: Reorganized utility imports.Cleaner organization of utility function imports.
api/py/ai/chronon/repo/gcp.py (4)
8-8: Moved import to separate line.Improved readability.
12-21: Reorganized utility imports.Better organization of imported utility functions.
71-71: Added exception chaining.Good practice to preserve original traceback with
from e.
89-89: Added exception chaining.Good practice to preserve original traceback with
from e.api/py/test/sample/group_bys/sample_team/sample_group_by_with_derivations.py (1)
16-17: Reordered Imports:
The new single-line import order is concise and consistent.api/py/test/sample/joins/sample_team/sample_label_join_with_agg.py (1)
26-26: Import Grouping:Join, JoinPart, LabelPartsimport is properly grouped.
| for filepath, auth_str in git_info([file], exclude).items(): | ||
| file_to_author[filepath] = auth_str.split("/")[-2:] |
There was a problem hiding this comment.
Fixed variable shadowing issue.
Renamed loop variable from file to filepath to avoid shadowing the parameter name.
| user_args: str, | ||
| job_type: JobType = JobType.SPARK, | ||
| local_files_to_upload: List[str] = [], | ||
| local_files_to_upload: List[str] = None, |
There was a problem hiding this comment.
Fixed mutable default argument.
Changed from [] to None to avoid the mutable default argument pitfall.
- local_files_to_upload: List[str] = None,
+ local_files_to_upload: List[str] = None,
+ if local_files_to_upload is None:
+ local_files_to_upload = []Consider initializing the list inside the function body for completeness.
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| local_files_to_upload: List[str] = None, | |
| def some_function( | |
| other_param: int, | |
| local_files_to_upload: List[str] = None, | |
| another_param: str = "" | |
| ): | |
| if local_files_to_upload is None: | |
| local_files_to_upload = [] | |
| # ... rest of the function body ... |
| job_type: JobType = JobType.SPARK, | ||
| version: str = "latest", | ||
| local_files_to_upload: List[str] = [], | ||
| local_files_to_upload: List[str] = None, |
There was a problem hiding this comment.
Potential TypeError with None default.
Changed default from empty list to None, but the function doesn't check if local_files_to_upload is None before iterating over it at line 195.
def generate_dataproc_submitter_args(
self,
user_args: str,
job_type: JobType = JobType.SPARK,
version: str = "latest",
- local_files_to_upload: List[str] = None,
+ local_files_to_upload: List[str] = None,
):
customer_warehouse_bucket_name = f"zipline-warehouse-{get_customer_id()}"
gcs_files = []
+ local_files_to_upload = local_files_to_upload or []
for source_file in local_files_to_upload:
# upload to `metadata` folder
destination_file_path = (
f"metadata/{extract_filename_from_path(source_file)}"
)📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| local_files_to_upload: List[str] = None, | |
| def generate_dataproc_submitter_args( | |
| self, | |
| user_args: str, | |
| job_type: JobType = JobType.SPARK, | |
| version: str = "latest", | |
| local_files_to_upload: List[str] = None, | |
| ): | |
| customer_warehouse_bucket_name = f"zipline-warehouse-{get_customer_id()}" | |
| gcs_files = [] | |
| local_files_to_upload = local_files_to_upload or [] | |
| for source_file in local_files_to_upload: | |
| # upload to `metadata` folder | |
| destination_file_path = ( | |
| f"metadata/{extract_filename_from_path(source_file)}" | |
| ) |
## Summary ## Checklist - [ ] Added Unit Tests - [x] Covered by existing CI - [ ] Integration tested - [ ] Documentation update <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **New Features** - Integrated a new linting tool into the development workflow for improved code quality. - **Refactor/Style** - Streamlined and reorganized code structure by reordering import statements and adjusting default parameters. - Enhanced error handling to provide clearer diagnostics without impacting functionality. - **Chores** - Updated configuration and build settings to standardize the codebase and CI processes. - Added a new configuration file for the Ruff linter, specifying linting settings and rules. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
## Summary ## Checklist - [ ] Added Unit Tests - [x] Covered by existing CI - [ ] Integration tested - [ ] Documentation update <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **New Features** - Integrated a new linting tool into the development workflow for improved code quality. - **Refactor/Style** - Streamlined and reorganized code structure by reordering import statements and adjusting default parameters. - Enhanced error handling to provide clearer diagnostics without impacting functionality. - **Chores** - Updated configuration and build settings to standardize the codebase and CI processes. - Added a new configuration file for the Ruff linter, specifying linting settings and rules. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
## Summary ## Checklist - [ ] Added Unit Tests - [x] Covered by existing CI - [ ] Integration tested - [ ] Documentation update <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **New Features** - Integrated a new linting tool into the development workflow for improved code quality. - **Refactor/Style** - Streamlined and reorganized code structure by reordering import statements and adjusting default parameters. - Enhanced error handling to provide clearer diagnostics without impacting functionality. - **Chores** - Updated configuration and build settings to standardize the codebase and CI processes. - Added a new configuration file for the Ruff linter, specifying linting settings and rules. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
## Summary ## Checklist - [ ] Added Unit Tests - [x] Covered by existing CI - [ ] Integration tested - [ ] Documentation update <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **New Features** - Integrated a new linting tool into the development workflow for improved code quality. - **Refactor/Style** - Streamlined and reorganized code structure by reordering import statements and adjusting default parameters. - Enhanced error handling to provide clearer diagnostics without impacting functionality. - **Chores** - Updated configuration and build settings to standardize the codebase and CI processes. - Added a new configuration file for the Ruff linter, specifying linting settings and rules. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
## Summary ## Cheour clientslist - [ ] Added Unit Tests - [x] Covered by existing CI - [ ] Integration tested - [ ] Documentation update <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **New Features** - Integrated a new linting tool into the development workflow for improved code quality. - **Refactor/Style** - Streamlined and reorganized code structure by reordering import statements and adjusting default parameters. - Enhanced error handling to provide clearer diagnostics without impacting functionality. - **Chores** - Updated configuration and build settings to standardize the codebase and CI processes. - Added a new configuration file for the Ruff linter, specifying linting settings and rules. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
Summary
Checklist
Summary by CodeRabbit
New Features
Refactor/Style
Chores