Skip to content

Conversation

@nikhil-zlai
Copy link
Contributor

@nikhil-zlai nikhil-zlai commented Apr 10, 2025

Summary

  • remove non-determinism in join output column name listing
  • use console.print instead of logger for consistency
  • use relative path for source file names.
  • remove dead code from when planner was going to be in python
  • remove offset setting api from LabelParts method / fixes the errors from the compiler.
  • lint fixes
  • automatically add cwd to the python path if not already present
Screenshot 2025-04-09 at 9 20 00 PM

Checklist

  • Added Unit Tests
  • Covered by existing CI
  • Integration tested
  • Documentation update

Summary by CodeRabbit

Summary by CodeRabbit

  • New Features

    • Enhanced console output with modern formatting for improved feedback.
    • Simplified configuration for label joining, offering a more intuitive experience.
    • Introduced a Spark session initialization for data generation.
    • Added a parameter to control console output in the team loading function.
  • Refactor

    • Optimized messaging and error display for clearer output.
    • Streamlined method interfaces and file path handling for consistency.
    • Improved code readability through formatting adjustments across various scripts.
  • Chores

    • Removed legacy command-line operations for data backfill and deployment.
    • Cleaned up obsolete components and redundant code.

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Apr 10, 2025

Walkthrough

This pull request enhances code readability and output formatting across multiple modules. It introduces a Rich-based console for improved output, replaces direct print calls with console.print, and simplifies method signatures. Additionally, it reformats error handling and object instantiation, adjusts file path handling via os.path.relpath, and removes outdated CLI commands along with legacy physical indexing files. The underlying functionality remains intact while the overall code structure is streamlined.

Changes

File(s) Change Summary
api/.../cli/compile/compiler.py, api/.../cli/compile/display/compile_status.py, api/.../cli/compile/display/console.py, api/.../cli/compile/display/class_tracker.py Replaced print with console.print, added a new Rich console file, reformatted error messages, and renamed a display method for clarity.
api/.../cli/compile/parse_configs.py, api/.../cli/compile/parse_teams.py, api/.../cli/compile/utils.py Adjusted file path assignment using os.path.relpath, introduced a new print parameter for conditional console output, and reformatted object instantiation and method chaining.
api/.../cli/entrypoint.py, api/.../cli/plan/physical_index.py, api/.../cli/plan/physical_node.py, api/.../cli/plan/controller_iface.py Removed obsolete CLI commands, deleted physical index/node files, and eliminated an unused import.
api/.../chronon/join.py, api/.../repo/compilev3.py, api/.../repo/validator.py Updated LabelParts by removing offset parameters, enhanced compile_v3 with a default root and sys.path check, and ensured sorted join output columns.

Sequence Diagram(s)

sequenceDiagram
    participant U as User
    participant C as compile_v3
    participant Sys as sys.path
    participant Con as console

    U->>C: call compile_v3(chronon_root)
    C->>Con: print blank line
    C->>Sys: check if chronon_root in sys.path
    alt Not in path
        C->>Con: print "adding chronon_root to sys.path"
        C->>Sys: append chronon_root
    else Already in path
        C->>Con: print "chronon_root found in sys.path"
    end
    C->>U: return compilation result
Loading
sequenceDiagram
    participant Caller as load_teams Caller
    participant LT as load_teams
    participant Con as console

    Caller->>LT: call load_teams(conf_root, print)
    alt print is True
        LT->>Con: console.print "pulling team configuration"
    end
    LT->>Caller: return team dict
Loading

Possibly related PRs

Suggested reviewers

  • tchow-zlai

Poem

In code we trust, in text we gleam,
With console flair and output beam,
Old commands fade as new lights shine,
Paths and prints now realign.
A dance of logic, crisp and clear—
A code celebration, loud and dear!
🎉🚀

Warning

Review ran into problems

🔥 Problems

GitHub Actions and Pipeline Checks: Resource not accessible by integration - https://docs.github.com/rest/actions/workflow-runs#list-workflow-runs-for-a-repository.

Please grant the required permissions to the CodeRabbit GitHub App under the organization or repository settings.


📜 Recent review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro (Legacy)

📥 Commits

Reviewing files that changed from the base of the PR and between 7f1675d and fe8f429.

📒 Files selected for processing (1)
  • api/python/ai/chronon/cli/compile/parse_teams.py (3 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • api/python/ai/chronon/cli/compile/parse_teams.py
⏰ Context from checks skipped due to timeout of 90000ms (4)
  • GitHub Check: non_spark_tests
  • GitHub Check: non_spark_tests
  • GitHub Check: python_tests
  • GitHub Check: enforce_triggered_workflows

🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai plan to trigger planning for file edits and PR creation.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

)
def compile_v3(chronon_root):

print()
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this a stray?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wanted a newline actually

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro (Legacy)

📥 Commits

Reviewing files that changed from the base of the PR and between d1524ca and 6b0a16c.

📒 Files selected for processing (14)
  • api/python/ai/chronon/cli/compile/compiler.py (4 hunks)
  • api/python/ai/chronon/cli/compile/display/class_tracker.py (0 hunks)
  • api/python/ai/chronon/cli/compile/display/compile_status.py (4 hunks)
  • api/python/ai/chronon/cli/compile/display/console.py (1 hunks)
  • api/python/ai/chronon/cli/compile/parse_configs.py (3 hunks)
  • api/python/ai/chronon/cli/compile/parse_teams.py (3 hunks)
  • api/python/ai/chronon/cli/entrypoint.py (0 hunks)
  • api/python/ai/chronon/cli/plan/controller_iface.py (0 hunks)
  • api/python/ai/chronon/cli/plan/physical_index.py (0 hunks)
  • api/python/ai/chronon/cli/plan/physical_node.py (0 hunks)
  • api/python/ai/chronon/join.py (1 hunks)
  • api/python/ai/chronon/repo/compilev3.py (1 hunks)
  • api/python/ai/chronon/repo/validator.py (1 hunks)
  • api/python/ai/chronon/utils.py (1 hunks)
💤 Files with no reviewable changes (5)
  • api/python/ai/chronon/cli/compile/display/class_tracker.py
  • api/python/ai/chronon/cli/plan/controller_iface.py
  • api/python/ai/chronon/cli/plan/physical_index.py
  • api/python/ai/chronon/cli/plan/physical_node.py
  • api/python/ai/chronon/cli/entrypoint.py
🧰 Additional context used
🧬 Code Graph Analysis (5)
api/python/ai/chronon/cli/compile/parse_teams.py (1)
api/python/ai/chronon/cli/compile/compiler.py (1)
  • compile (30-59)
api/python/ai/chronon/cli/compile/display/compile_status.py (2)
api/python/ai/chronon/cli/compile/compiler.py (1)
  • compile (30-59)
api/python/ai/chronon/cli/compile/display/diff_result.py (1)
  • render (12-46)
api/python/ai/chronon/utils.py (1)
api/python/ai/chronon/cli/compile/parse_teams.py (1)
  • load_teams (42-69)
api/python/ai/chronon/repo/compilev3.py (2)
api/python/ai/chronon/cli/compile/compiler.py (1)
  • compile (30-59)
api/python/ai/chronon/cli/compile/compile_context.py (1)
  • CompileContext (25-166)
api/python/ai/chronon/cli/compile/compiler.py (2)
api/python/ai/chronon/cli/compile/display/compile_status.py (2)
  • render (66-85)
  • print_live_console (25-27)
api/python/ai/chronon/cli/compile/display/diff_result.py (1)
  • render (12-46)
🪛 Ruff (0.8.2)
api/python/ai/chronon/cli/compile/display/compile_status.py

7-7: ai.chronon.cli.compile.display.console imported but unused

Remove unused import: ai.chronon.cli.compile.display.console

(F401)

🔇 Additional comments (24)
api/python/ai/chronon/cli/compile/display/console.py (1)

1-3: Good addition of Rich's Console for better formatting

Creates a centralized console instance for formatted output, replacing logger.

api/python/ai/chronon/repo/validator.py (1)

186-188: Nice fix for deterministic column ordering

Sorting the output columns addresses non-determinism in join outputs.

api/python/ai/chronon/utils.py (1)

327-327: Good update to suppress console output

Passes print=False to avoid unnecessary output when loading teams from utils.

api/python/ai/chronon/cli/compile/parse_teams.py (3)

15-15: Appropriate import for console

Imports the centralized console object for formatted output.


42-42: Good API enhancement

Adding print parameter with default value maintains backward compatibility.


59-63: Clean implementation of conditional output

Uses Rich formatting for better visibility when output is desired.

api/python/ai/chronon/cli/compile/parse_configs.py (3)

34-36: Good use of relative paths.

Using relative paths to the chronon root makes the source file references more portable.


59-65: Better formatting.

Improved readability by breaking the CompiledObj instantiation across multiple lines.


69-71: Consistent formatting.

The multi-line format matches the style used for CompiledObj above.

api/python/ai/chronon/repo/compilev3.py (3)

4-4: Required import added.

The sys module is needed for path manipulation.


16-16: Sensible default added.

Using current working directory as default for chronon_root improves usability.


20-30: Path management improved.

Added Python path handling helps ensure imports work correctly during compilation. Good user feedback included.

api/python/ai/chronon/join.py (1)

252-305:

✅ Verification successful

API simplification: removed offset parameters.

Removed left_start_offset and left_end_offset parameters from LabelParts function, relying instead on training windows specified in label GroupBy-s.


🏁 Script executed:

#!/bin/bash
# Check if there are any existing calls to LabelParts with the removed parameters
rg "LabelParts\s*\(" --type py | grep -v "join\.py" | xargs grep -l "left_start_offset" || echo "No usage of left_start_offset found"
rg "LabelParts\s*\(" --type py | grep -v "join\.py" | xargs grep -l "left_end_offset" || echo "No usage of left_end_offset found"

Length of output: 653


Action: API simplification verified – offset parameters removed.

  • In api/python/ai/chronon/join.py, the left_start_offset and left_end_offset parameters have been removed from LabelParts, relying on the training windows specified in label GroupBy-s.
  • Shell script verification confirms there are no existing calls referencing these parameters.
api/python/ai/chronon/cli/compile/display/compile_status.py (3)

22-22: Quote style change.

Maintaining consistent quote style.


66-66:

✅ Verification successful

Method renamed to "render".

Renamed from generate_update_display_text to the simpler render for consistency with other UI components.


🏁 Script executed:

#!/bin/bash
# Check if any code still calls the old method name
rg "generate_update_display_text" --type py || echo "No references to old method name found"

Length of output: 130


Rename Verified: Method now consistently uses "render".

  • File: api/python/ai/chronon/cli/compile/display/compile_status.py, Line 66
  • The shell script confirmed no remaining references to generate_update_display_text.

90-95:

✅ Verification successful

Temporary code removal.

Commented out display update code with a TODO for implementing live_crop. Ensure this doesn't break functionality.


🏁 Script executed:

#!/bin/bash
# Look for code that might depend on _update_display functionality
rg "\._update_display" --type py | grep -v "compile_status\.py" || echo "No external calls to _update_display found"

Length of output: 160


Verified: Temporary Code Removal Appears Safe

The temporary removal of the live update is fine. The grep search found no external calls to _update_display, so there’s no immediate risk. Please ensure regression tests cover this behavior until live_crop is implemented.

api/python/ai/chronon/cli/compile/compiler.py (8)

7-7: Added Rich console import

New import adds the Rich-based console for formatted output, replacing logger in some places.


30-30: Simplified method signature formatting

Method signature formatting change improves readability.


37-37: Method call simplified

Method call now on single line. Cleaner code.


57-57: Replaced print with console.print

Changed from direct printing to Rich console rendering, aligning with PR objective to replace logger with console.print.


101-103: Improved error message formatting

Error message formatting now more readable with consistent style.


105-106: Simplified traceback printing

Traceback printing formatted consistently with error messages.


114-116: Consistent error formatting

File error message formatting matches the style used elsewhere.


118-118: Simplified traceback printing

Traceback formatting consistent with other error handling.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🔭 Outside diff range comments (2)
api/python/ai/chronon/join.py (2)

252-257: ⚠️ Potential issue

Parameter inconsistency in LabelParts function.

The function signature still includes left_start_offset and left_end_offset parameters, but they aren't included in the return statement. This mismatch could cause compiler errors.

-def LabelParts(
-    labels: List[api.JoinPart],
-    left_start_offset: int,
-    left_end_offset: int,
-    label_offline_schedule: str = "@daily",
-) -> api.LabelParts:
+def LabelParts(
+    labels: List[api.JoinPart],
+    label_offline_schedule: str = "@daily",
+) -> api.LabelParts:

Also applies to: 302-305


500-509: ⚠️ Potential issue

Remove leftStartOffset and leftEndOffset from LabelParts initialization.

These parameters are still being used in the creation of the LabelParts object but aren't included in the function's return value.

-    label_part = api.LabelParts(
-        labels=label_part.labels,
-        leftStartOffset=label_part.leftStartOffset,
-        leftEndOffset=label_part.leftEndOffset,
-        metaData=label_metadata,
-    )
+    label_part = api.LabelParts(
+        labels=label_part.labels,
+        metaData=label_metadata,
+    )
♻️ Duplicate comments (1)
api/python/ai/chronon/repo/compilev3.py (1)

20-20: Remove empty print statement

Empty print statement serves no purpose.

-    print()
🧹 Nitpick comments (1)
api/python/ai/chronon/cli/compile/display/compile_status.py (1)

7-7: Remove unused import

Static analysis shows this import isn't used in the file.

-from ai.chronon.cli.compile.display import console
🧰 Tools
🪛 Ruff (0.8.2)

7-7: ai.chronon.cli.compile.display.console imported but unused

Remove unused import: ai.chronon.cli.compile.display.console

(F401)

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro (Legacy)

📥 Commits

Reviewing files that changed from the base of the PR and between d1524ca and f587df3.

📒 Files selected for processing (14)
  • api/python/ai/chronon/cli/compile/compiler.py (4 hunks)
  • api/python/ai/chronon/cli/compile/display/class_tracker.py (0 hunks)
  • api/python/ai/chronon/cli/compile/display/compile_status.py (4 hunks)
  • api/python/ai/chronon/cli/compile/display/console.py (1 hunks)
  • api/python/ai/chronon/cli/compile/parse_configs.py (3 hunks)
  • api/python/ai/chronon/cli/compile/parse_teams.py (3 hunks)
  • api/python/ai/chronon/cli/entrypoint.py (0 hunks)
  • api/python/ai/chronon/cli/plan/controller_iface.py (0 hunks)
  • api/python/ai/chronon/cli/plan/physical_index.py (0 hunks)
  • api/python/ai/chronon/cli/plan/physical_node.py (0 hunks)
  • api/python/ai/chronon/join.py (1 hunks)
  • api/python/ai/chronon/repo/compilev3.py (1 hunks)
  • api/python/ai/chronon/repo/validator.py (1 hunks)
  • api/python/ai/chronon/utils.py (1 hunks)
💤 Files with no reviewable changes (5)
  • api/python/ai/chronon/cli/plan/controller_iface.py
  • api/python/ai/chronon/cli/plan/physical_node.py
  • api/python/ai/chronon/cli/plan/physical_index.py
  • api/python/ai/chronon/cli/compile/display/class_tracker.py
  • api/python/ai/chronon/cli/entrypoint.py
🧰 Additional context used
🧬 Code Graph Analysis (3)
api/python/ai/chronon/utils.py (1)
api/python/ai/chronon/cli/compile/parse_teams.py (1)
  • load_teams (43-70)
api/python/ai/chronon/repo/compilev3.py (2)
api/python/ai/chronon/cli/compile/compiler.py (2)
  • compile (30-59)
  • Compiler (25-134)
api/python/ai/chronon/cli/compile/compile_context.py (1)
  • CompileContext (25-166)
api/python/ai/chronon/cli/compile/display/compile_status.py (2)
api/python/ai/chronon/cli/compile/compiler.py (1)
  • compile (30-59)
api/python/ai/chronon/cli/compile/display/diff_result.py (1)
  • render (12-46)
🪛 Ruff (0.8.2)
api/python/ai/chronon/cli/compile/display/compile_status.py

7-7: ai.chronon.cli.compile.display.console imported but unused

Remove unused import: ai.chronon.cli.compile.display.console

(F401)

⏰ Context from checks skipped due to timeout of 90000ms (2)
  • GitHub Check: non_spark_tests
  • GitHub Check: non_spark_tests
🔇 Additional comments (14)
api/python/ai/chronon/cli/compile/display/console.py (1)

1-3: Rich console implementation.

Creates a reusable console object using Rich library for improved output formatting.

api/python/ai/chronon/repo/validator.py (1)

186-188: Column sorting adds determinism.

Sorting output columns ensures consistent order in join outputs, eliminating non-deterministic behavior.

api/python/ai/chronon/utils.py (1)

327-330: Silent team loading and improved readability.

Added print=False parameter to avoid unnecessary console output and improved code readability with line breaks in method chaining.

api/python/ai/chronon/cli/compile/parse_teams.py (3)

15-15: Console import added.

Imports the rich console object for enhanced output formatting.


43-43: Added print parameter.

Function signature updated with optional boolean parameter for controlling output.


59-64: Logger replaced with rich console.

Conditional printing with rich console formatting instead of logging, improving output consistency.

api/python/ai/chronon/cli/compile/display/compile_status.py (1)

66-66:

✅ Verification successful

Method renamed from generate_update_display_text to render

API change that aligns with the render method in other modules.


🏁 Script executed:

#!/bin/bash
# Find any remaining calls to the old method name
rg "generate_update_display_text" -t py

Length of output: 39


Verified Method Rename:
The method was successfully renamed to render with no remaining calls to generate_update_display_text in the codebase.

api/python/ai/chronon/repo/compilev3.py (2)

16-16: Good default parameter addition

Setting default chronon_root to current directory improves usability.


22-30: Good addition of python path management

Ensures chronon_root is in sys.path for proper module imports.

api/python/ai/chronon/cli/compile/parse_configs.py (2)

33-35: Improved path handling with relative paths

Using relative paths instead of absolute makes file references more portable.


58-64: Better code formatting

Multi-line parameter formatting improves readability.

Also applies to: 68-70

api/python/ai/chronon/cli/compile/compiler.py (2)

7-7: Replaced print with console.print

Switched to Rich console for enhanced output formatting.

Also applies to: 57-57


101-103: Improved error message formatting

Better multi-line formatting for error messages and traceback printing.

Also applies to: 105-106, 114-116, 118-118

api/python/ai/chronon/join.py (1)

261-261: Documentation update matches implementation change.

The documentation now correctly reflects that labels are joined based on user-specified settings within label GroupBy-s rather than using explicit offsets.

Comment on lines +90 to +95
# TODO: add this after live_crop is implemented
# text = self.display_text()
# if self.use_live:
# self.live.update(text, refresh=True)
# return text
pass
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Verification agent

🧩 Analysis chain

Commented-out implementation replaced with pass

The _update_display method now does nothing but is still called in multiple places.


🏁 Script executed:

#!/bin/bash
# Find all calls to _update_display to assess impact
rg -A 3 "_update_display" api/python/ai/chronon/cli/compile/display/compile_status.py

Length of output: 758


Issue: _update_display is a no-op and is called in multiple methods

  • Called in: add_object_update_display, add_existing_object_update_display, close_cls, and close.
  • Currently, it only executes a "pass" with a TODO note.
  • Please add a minimal stub (e.g., logging) or remove redundant calls until live_crop is implemented.

@nikhil-zlai nikhil-zlai merged commit e058461 into main Apr 10, 2025
7 checks passed
@nikhil-zlai nikhil-zlai deleted the nikhil/compiler_fixes branch April 10, 2025 19:42
kumar-zlai pushed a commit that referenced this pull request Apr 25, 2025
kumar-zlai pushed a commit that referenced this pull request Apr 29, 2025
chewy-zlai pushed a commit that referenced this pull request May 15, 2025
chewy-zlai pushed a commit that referenced this pull request May 15, 2025
chewy-zlai pushed a commit that referenced this pull request May 16, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants