Use teams.py instead of teams.json for compile #541
Conversation
WalkthroughThis pull request introduces extensive modifications across the codebase. Major updates include refactoring error handling to use lists of exceptions instead of strings, updating import sources, consolidating team configuration parameters, and introducing a new CLI compile command ( Changes
Sequence Diagram(s)sequenceDiagram
participant User
participant CLI
participant Compiler
participant Context
participant Display
User->>CLI: Execute "compile_v3" with --chronon_root
CLI->>CLI: Expand chronon_root & check for teams file
CLI->>Context: Instantiate CompileContext (use_live flag)
Context->>Compiler: Pass context for compilation
Compiler->>Compiler: Process configurations and write objects
alt Errors found
Compiler->>Display: Print errors via live console
else No errors
Compiler->>Display: Update display with compilation result
end
Display-->>User: Show results
Possibly related PRs
Suggested reviewers
Poem
Warning Review ran into problems🔥 ProblemsGitHub Actions and Pipeline Checks: Resource not accessible by integration - https://docs.github.com/rest/actions/workflow-runs#list-workflow-runs-for-a-repository. Please grant the required permissions to the CodeRabbit GitHub App under the organization or repository settings. 📜 Recent review detailsConfiguration used: CodeRabbit UI 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
⏰ Context from checks skipped due to timeout of 90000ms (16)
🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (2)
api/python/test/canary/teams.py (2)
7-7: Update placeholder customer emailReplace
<customer>with actual email domain.
100-108: Consider expanding AWS configurationMinimal AWS setup lacks the detailed configurations seen in GCP team. Consider adding AWS-specific settings if needed.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro (Legacy)
📒 Files selected for processing (5)
api/python/ai/chronon/cli/compile/compiler.py(5 hunks)api/python/ai/chronon/cli/compile/parse_teams.py(5 hunks)api/python/test/canary/teams.py(1 hunks)api/src/main/scala/ai/chronon/api/Extensions.scala(1 hunks)api/thrift/common.thrift(2 hunks)
🚧 Files skipped from review as they are similar to previous changes (2)
- api/src/main/scala/ai/chronon/api/Extensions.scala
- api/python/ai/chronon/cli/compile/compiler.py
🧰 Additional context used
🧬 Code Definitions (1)
api/python/ai/chronon/cli/compile/parse_teams.py (1)
api/python/ai/chronon/cli/logger.py (1)
get_logger(32-43)
⏰ Context from checks skipped due to timeout of 90000ms (4)
- GitHub Check: non_spark_tests
- GitHub Check: non_spark_tests
- GitHub Check: scala_compile_fmt_fix
- GitHub Check: enforce_triggered_workflows
🔇 Additional comments (11)
api/python/ai/chronon/cli/compile/parse_teams.py (6)
2-4: Import updates and constant renaming look good.The new imports support dynamic module loading, and renaming _DEFAULT_CONF_TEAM from "common" to "default" aligns with teams.py approach.
Also applies to: 6-6, 14-14, 18-18
21-37: New module import function is well implemented.Clean implementation of dynamic Python module loading that follows standard importlib patterns.
44-56: Assert statements and logging improve error handling.Using assertions instead of require() prevents sys.exit() calls, making errors catchable. Added logging helps with debugging.
67-86: Assert statements in update_metadata improve error clarity.Error messages are now more descriptive when handling missing or invalid team configs.
106-106: Type signature update is more accurate.Changed from List[Dict[str, str]] to Optional[Dict[str, str]] to better match actual usage.
124-163: Mode map merging logic is more robust.Added null checks for modesMap prevents errors, and common map handling is improved.
api/python/test/canary/teams.py (3)
13-16: Remove placeholder TODOsAddress all placeholder TODOs before merging.
31-48: Good test configuration setupClear separation of common settings and mode-specific configurations using the new modesMap structure.
51-98: Well-structured GCP configurationComprehensive setup with appropriate Spark settings for GCP environment.
api/thrift/common.thrift (2)
40-41: Improved environment variables structureSimplifying from multiple optional maps to a required common map and nested modesMap improves organization.
56-57: Consistent configuration properties structureMatches the EnvironmentVariables pattern for better consistency across the codebase.
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (1)
api/python/test/canary/teams.py (1)
64-64: Clarify "dummy test config" commentsRemove or replace these comments if this configuration is intended for actual use.
Also applies to: 93-93
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro (Legacy)
📒 Files selected for processing (2)
api/python/ai/chronon/cli/compile/parse_teams.py(5 hunks)api/python/test/canary/teams.py(1 hunks)
⏰ Context from checks skipped due to timeout of 90000ms (4)
- GitHub Check: non_spark_tests
- GitHub Check: non_spark_tests
- GitHub Check: scala_compile_fmt_fix
- GitHub Check: enforce_triggered_workflows
🔇 Additional comments (14)
api/python/ai/chronon/cli/compile/parse_teams.py (10)
2-6: Import modifications for dynamic loading.Added necessary imports for dynamic module loading functionality.
18-18: Team name convention update.Changed default configuration team name from "common" to "default".
21-38: New dynamic module loading function.Clean implementation for importing Python modules from file paths.
44-46: Improved error handling with assertions.Replaced
requirewithassertfor better exception handling.
48-53: Updated team module loading approach.Now using dynamic module loading to import teams configuration.
56-56: Added logging for team processing.Provides clarity on which teams file is being processed.
67-87: Consistent error handling for metadata validation.Replaced all
requirecalls withassertfor consistent error handling.
106-106: Improved type annotation.Changed parameter type from
List[Dict[str, str]]to more accurateOptional[Dict[str, str]].
124-124: Refined type annotation.More precise Union type for mode maps parameters.
131-163: Enhanced configuration merging logic.Improved handling of common configurations and mode-specific maps.
api/python/test/canary/teams.py (4)
7-7: Remove placeholder TODOsReplace these placeholders with actual values before merging.
Also applies to: 13-15
1-4: LGTM!Imports are well-structured.
28-48: Good test team configurationAppropriately uses modesMap for different run modes.
99-107: Verify AWS configuration completenessAWS configuration is minimal compared to GCP. Confirm it's sufficient.
api/thrift/common.thrift
Outdated
| 4: optional map<string, string> streaming | ||
| 5: optional map<string, string> serving | ||
| 1: map<string, string> common | ||
| 2: map<string, map<string, string>> modesMap |
There was a problem hiding this comment.
lets call it modeEnvironments
There was a problem hiding this comment.
we were getting lucky that Environment and Config both had the same thrift schema here haha:
renaming them to be modeEnvironments and modeConfigs breaks that. lemme refactor
api/thrift/common.thrift
Outdated
| 4: optional map<string, string> streaming | ||
| 5: optional map<string, string> serving | ||
| 1: map<string, string> common | ||
| 2: map<string, map<string, string>> modesMap |
There was a problem hiding this comment.
lets call it modeConfigs
| "source-job": "source-job", | ||
| "join-part-job": "join-part-job", | ||
| "merge-job": "merge-job", | ||
| RunMode.BACKFILL: "join", |
nikhil-zlai
left a comment
nikhil-zlai
left a comment
There was a problem hiding this comment.
Solid PR David!
One comment, but looks great
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (1)
api/python/test/canary/teams.py (1)
64-64: Replace dummy test configsUpdate "dummy test config" comments with actual configuration descriptions or remove them.
Also applies to: 93-93
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro (Legacy)
📒 Files selected for processing (5)
api/python/ai/chronon/cli/compile/parse_teams.py(5 hunks)api/python/ai/chronon/repo/utils.py(4 hunks)api/python/test/canary/teams.py(1 hunks)api/python/test/sample/teams.py(2 hunks)api/thrift/common.thrift(2 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
- api/python/test/sample/teams.py
⏰ Context from checks skipped due to timeout of 90000ms (3)
- GitHub Check: non_spark_tests
- GitHub Check: non_spark_tests
- GitHub Check: scala_compile_fmt_fix
🔇 Additional comments (23)
api/thrift/common.thrift (2)
40-41: Good change - more flexible architecture.Replacing separate mode-specific fields with nested map structure provides greater extensibility.
56-57: Good schema alignment.Schema now matches between environment variables and config properties structures.
api/python/test/canary/teams.py (3)
7-7: Update team email addressReplace placeholder with actual infra team email.
13-15: Remove placeholder TODOsReplace TODOs with actual values for HADOOP_DIR, CHRONON_ONLINE_CLASS, and CHRONON_ONLINE_ARGS.
1-107: Well-structured configuration approachGood job organizing team configurations in a Python file. This approach provides better type safety and easier maintenance than JSON.
api/python/ai/chronon/repo/utils.py (6)
167-168: Function looks good!Simple utility function to get the teams.py file path.
170-227: New runtime environment setup function handles configuration elegantly.The new function properly processes configuration files and sets environment variables. It's well-structured with good error handling.
228-228: Remember to remove this TODO comment.The TODO indicates this code will be removed after complete migration to teams.py.
Is there a timeline for this cleanup?
262-269: Good validation to prevent ambiguity.Checking for both files and raising a clear error message helps ensure clean migration.
271-274: Clean refactoring.Extraction of teams.json handling logic into separate function improves maintainability.
320-320: Plan for removing legacy code.As commented on line 228, this function is also marked for deletion after migration.
api/python/ai/chronon/cli/compile/parse_teams.py (12)
19-19: Good renaming of default team constant.Consistent with PR objectives to use teams.py instead of teams.json.
22-39: Well-implemented module importing functionality.This function correctly handles dynamic module loading from file paths using importlib.
45-54: Assert statements improve error handling.Changed from require() to assert with clear error messages.
57-57: Helpful logging.Adds transparency about which teams file is being processed.
77-87: Improved validation with clear error messages.Assertions ensure proper team configuration with helpful error guidance.
98-99: Good use of enum for attribute specification.Makes the code more type-safe and self-documenting.
105-106: Consistent parameter usage between env and config.Same pattern as line 98-99.
109-109: Improved type hints for merged maps.Optional typing makes function more flexible and safer.
127-130: Well-defined enum for attribute types.Adds clarity to the API and prevents string typos.
133-134: Updated function signature with enum parameter.Adds type safety while maintaining compatibility with existing code.
152-159: Flexible attribute access using getattr.Allows code reuse between environment and config processing.
167-175: Consistent attribute handling.Using getattr with the enum parameter maintains consistent code pattern.
…chronon into davidhan/compile_switch_fork
## Summary ^^^ Ran both AWS and GCP integration tests. Following this PR, i'll work on rebasing #549 which applies the confs to the submitter interface. Previously, to compile it would be: ``` zipline compile --conf=group_bys/aws/purchases.py ``` but now `--conf` or `--input-path` is not required, just: ``` zipline compile ``` and this compiles the entire repo. In addition, the compiled files gets persisted in the `compiled` folder. No longer are compiled files persisted by environment like `production/...` ## Checklist - [ ] Added Unit Tests - [x] Covered by existing CI - [x] Integration tested - [ ] Documentation update GCP and AWS integration tests pass ``` <<<<<.....................................COMPILE.....................................>>>>> + zipline compile --chronon_root=/Users/davidhan/zipline/chronon/api/python/test/canary 17:49:15 INFO parse_teams.py:56 - Processing teams from path /Users/davidhan/zipline/chronon/api/python/test/canary/teams.py INFO:ai.chronon.cli.logger:Processing teams from path /Users/davidhan/zipline/chronon/api/python/test/canary/teams.py GroupBy-s: Compiled 5 objects from 3 files. Added aws.plaid_fv.v1 Added aws.purchases.v1_dev Added aws.purchases.v1_test Added gcp.purchases.v1_dev Added gcp.purchases.v1_test .... <<<<<<<<<<<<<<<<-----------------JOB STATUS----------------->>>>>>>>>>>>>>>>> + aws emr wait step-complete --cluster-id j-13BASWFP15TLR --step-id s-02878283D08B3510ZGCZ ++ aws emr describe-step --cluster-id j-13BASWFP15TLR --step-id s-02878283D08B3510ZGCZ --query Step.Status.State ++ tr -d '"' + STEP_STATE=COMPLETED + '[' COMPLETED '!=' COMPLETED ']' + echo succeeded succeeded + echo -e '\033[0;32m<<<<<.....................................SUCCEEDED!!!.....................................>>>>>\033[0m' <<<<<.....................................SUCCEEDED!!!.....................................>>>>> ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit ## Summary by CodeRabbit - **New Features** - Introduced an enhanced CLI compile command with an option to specify a custom configuration root. - Enabled dynamic loading of team configurations from both JSON and Python formats. - Added support for multiple error reporting in various components. - **Improvements** - Streamlined error and status messages for clearer, simplified feedback. - Enhanced error handling mechanisms across the application. - Updated quickstart scripts for AWS and GCP to utilize a unified, precompiled configuration set. - Modified the way environment variables are structured for improved clarity and management. - **Dependency Updates** - Added new dependencies to improve functionality. - Upgraded several existing dependencies to boost performance and enhance output formatting. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Nikhil Simha <[email protected]>
## Summary ^^^ Ran both AWS and GCP integration tests. Following this PR, i'll work on rebasing #549 which applies the confs to the submitter interface. Previously, to compile it would be: ``` zipline compile --conf=group_bys/aws/purchases.py ``` but now `--conf` or `--input-path` is not required, just: ``` zipline compile ``` and this compiles the entire repo. In addition, the compiled files gets persisted in the `compiled` folder. No longer are compiled files persisted by environment like `production/...` ## Checklist - [ ] Added Unit Tests - [x] Covered by existing CI - [x] Integration tested - [ ] Documentation update GCP and AWS integration tests pass ``` <<<<<.....................................COMPILE.....................................>>>>> + zipline compile --chronon_root=/Users/davidhan/zipline/chronon/api/python/test/canary 17:49:15 INFO parse_teams.py:56 - Processing teams from path /Users/davidhan/zipline/chronon/api/python/test/canary/teams.py INFO:ai.chronon.cli.logger:Processing teams from path /Users/davidhan/zipline/chronon/api/python/test/canary/teams.py GroupBy-s: Compiled 5 objects from 3 files. Added aws.plaid_fv.v1 Added aws.purchases.v1_dev Added aws.purchases.v1_test Added gcp.purchases.v1_dev Added gcp.purchases.v1_test .... <<<<<<<<<<<<<<<<-----------------JOB STATUS----------------->>>>>>>>>>>>>>>>> + aws emr wait step-complete --cluster-id j-13BASWFP15TLR --step-id s-02878283D08B3510ZGCZ ++ aws emr describe-step --cluster-id j-13BASWFP15TLR --step-id s-02878283D08B3510ZGCZ --query Step.Status.State ++ tr -d '"' + STEP_STATE=COMPLETED + '[' COMPLETED '!=' COMPLETED ']' + echo succeeded succeeded + echo -e '\033[0;32m<<<<<.....................................SUCCEEDED!!!.....................................>>>>>\033[0m' <<<<<.....................................SUCCEEDED!!!.....................................>>>>> ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit ## Summary by CodeRabbit - **New Features** - Introduced an enhanced CLI compile command with an option to specify a custom configuration root. - Enabled dynamic loading of team configurations from both JSON and Python formats. - Added support for multiple error reporting in various components. - **Improvements** - Streamlined error and status messages for clearer, simplified feedback. - Enhanced error handling mechanisms across the application. - Updated quickstart scripts for AWS and GCP to utilize a unified, precompiled configuration set. - Modified the way environment variables are structured for improved clarity and management. - **Dependency Updates** - Added new dependencies to improve functionality. - Upgraded several existing dependencies to boost performance and enhance output formatting. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Nikhil Simha <[email protected]>
## Summary ^^^ Ran both AWS and GCP integration tests. Following this PR, i'll work on rebasing #549 which applies the confs to the submitter interface. Previously, to compile it would be: ``` zipline compile --conf=group_bys/aws/purchases.py ``` but now `--conf` or `--input-path` is not required, just: ``` zipline compile ``` and this compiles the entire repo. In addition, the compiled files gets persisted in the `compiled` folder. No longer are compiled files persisted by environment like `production/...` ## Checklist - [ ] Added Unit Tests - [x] Covered by existing CI - [x] Integration tested - [ ] Documentation update GCP and AWS integration tests pass ``` <<<<<.....................................COMPILE.....................................>>>>> + zipline compile --chronon_root=/Users/davidhan/zipline/chronon/api/python/test/canary 17:49:15 INFO parse_teams.py:56 - Processing teams from path /Users/davidhan/zipline/chronon/api/python/test/canary/teams.py INFO:ai.chronon.cli.logger:Processing teams from path /Users/davidhan/zipline/chronon/api/python/test/canary/teams.py GroupBy-s: Compiled 5 objects from 3 files. Added aws.our clients_fv.v1 Added aws.purchases.v1_dev Added aws.purchases.v1_test Added gcp.purchases.v1_dev Added gcp.purchases.v1_test .... <<<<<<<<<<<<<<<<-----------------JOB STATUS----------------->>>>>>>>>>>>>>>>> + aws emr wait step-complete --cluster-id j-13BASWFP15TLR --step-id s-02878283D08B3510ZGCZ ++ aws emr describe-step --cluster-id j-13BASWFP15TLR --step-id s-02878283D08B3510ZGCZ --query Step.Status.State ++ tr -d '"' + STEP_STATE=COMPLETED + '[' COMPLETED '!=' COMPLETED ']' + echo succeeded succeeded + echo -e '\033[0;32m<<<<<.....................................SUCCEEDED!!!.....................................>>>>>\033[0m' <<<<<.....................................SUCCEEDED!!!.....................................>>>>> ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit ## Summary by CodeRabbit - **New Features** - Introduced an enhanced CLI compile command with an option to specify a custom configuration root. - Enabled dynamic loading of team configurations from both JSON and Python formats. - Added support for multiple error reporting in various components. - **Improvements** - Streamlined error and status messages for clearer, simplified feedback. - Enhanced error handling mechanisms across the application. - Updated quickstart scripts for AWS and GCP to utilize a unified, precompiled configuration set. - Modified the way environment variables are structured for improved clarity and management. - **Dependency Updates** - Added new dependencies to improve functionality. - Upgraded several existing dependencies to boost performance and enhance output formatting. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Nikhil Simha <[email protected]>
## Summary ^^^ Ran both AWS and GCP integration tests. Following this PR, i'll work on rebasing #549 which applies the confs to the submitter interface. Previously, to compile it would be: ``` zipline compile --conf=group_bys/aws/purchases.py ``` but now `--conf` or `--input-path` is not required, just: ``` zipline compile ``` and this compiles the entire repo. In addition, the compiled files gets persisted in the `compiled` folder. No longer are compiled files persisted by environment like `production/...` ## Checklist - [ ] Added Unit Tests - [x] Covered by existing CI - [x] Integration tested - [ ] Documentation update GCP and AWS integration tests pass ``` <<<<<.....................................COMPILE.....................................>>>>> + zipline compile --chronon_root=/Users/davidhan/zipline/chronon/api/python/test/canary 17:49:15 INFO parse_teams.py:56 - Processing teams from path /Users/davidhan/zipline/chronon/api/python/test/canary/teams.py INFO:ai.chronon.cli.logger:Processing teams from path /Users/davidhan/zipline/chronon/api/python/test/canary/teams.py GroupBy-s: Compiled 5 objects from 3 files. Added aws.our clients_fv.v1 Added aws.purchases.v1_dev Added aws.purchases.v1_test Added gcp.purchases.v1_dev Added gcp.purchases.v1_test .... <<<<<<<<<<<<<<<<-----------------JOB STATUS----------------->>>>>>>>>>>>>>>>> + aws emr wait step-complete --cluster-id j-13BASWFP15TLR --step-id s-02878283D08B3510ZGCZ ++ aws emr describe-step --cluster-id j-13BASWFP15TLR --step-id s-02878283D08B3510ZGCZ --query Step.Status.State ++ tr -d '"' + STEP_STATE=COMPLETED + '[' COMPLETED '!=' COMPLETED ']' + echo succeeded succeeded + echo -e '\033[0;32m<<<<<.....................................SUCCEEDED!!!.....................................>>>>>\033[0m' <<<<<.....................................SUCCEEDED!!!.....................................>>>>> ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit ## Summary by CodeRabbit - **New Features** - Introduced an enhanced CLI compile command with an option to specify a custom configuration root. - Enabled dynamic loading of team configurations from both JSON and Python formats. - Added support for multiple error reporting in various components. - **Improvements** - Streamlined error and status messages for clearer, simplified feedback. - Enhanced error handling mechanisms across the application. - Updated quickstart scripts for AWS and GCP to utilize a unified, precompiled configuration set. - Modified the way environment variables are structured for improved clarity and management. - **Dependency Updates** - Added new dependencies to improve functionality. - Upgraded several existing dependencies to boost performance and enhance output formatting. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Nikhil Simha <[email protected]>
## Summary ^^^ Ran both AWS and GCP integration tests. Following this PR, i'll work on rebasing #549 which applies the confs to the submitter interface. Previously, to compile it would be: ``` zipline compile --conf=group_bys/aws/purchases.py ``` but now `--conf` or `--input-path` is not required, just: ``` zipline compile ``` and this compiles the entire repo. In addition, the compiled files gets persisted in the `compiled` folder. No longer are compiled files persisted by environment like `production/...` ## Cheour clientslist - [ ] Added Unit Tests - [x] Covered by existing CI - [x] Integration tested - [ ] Documentation update GCP and AWS integration tests pass ``` <<<<<.....................................COMPILE.....................................>>>>> + zipline compile --chronon_root=/Users/davidhan/zipline/chronon/api/python/test/canary 17:49:15 INFO parse_teams.py:56 - Processing teams from path /Users/davidhan/zipline/chronon/api/python/test/canary/teams.py INFO:ai.chronon.cli.logger:Processing teams from path /Users/davidhan/zipline/chronon/api/python/test/canary/teams.py GroupBy-s: Compiled 5 objects from 3 files. Added aws.our clients_fv.v1 Added aws.purchases.v1_dev Added aws.purchases.v1_test Added gcp.purchases.v1_dev Added gcp.purchases.v1_test .... <<<<<<<<<<<<<<<<-----------------JOB STATUS----------------->>>>>>>>>>>>>>>>> + aws emr wait step-complete --cluster-id j-13BASWFP15TLR --step-id s-02878283D08B3510ZGCZ ++ aws emr describe-step --cluster-id j-13BASWFP15TLR --step-id s-02878283D08B3510ZGCZ --query Step.Status.State ++ tr -d '"' + STEP_STATE=COMPLETED + '[' COMPLETED '!=' COMPLETED ']' + echo succeeded succeeded + echo -e '\033[0;32m<<<<<.....................................SUCCEEDED!!!.....................................>>>>>\033[0m' <<<<<.....................................SUCCEEDED!!!.....................................>>>>> ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit ## Summary by CodeRabbit - **New Features** - Introduced an enhanced CLI compile command with an option to specify a custom configuration root. - Enabled dynamic loading of team configurations from both JSON and Python formats. - Added support for multiple error reporting in various components. - **Improvements** - Streamlined error and status messages for clearer, simplified feedbaour clients. - Enhanced error handling mechanisms across the application. - Updated quiour clientsstart scripts for AWS and GCP to utilize a unified, precompiled configuration set. - Modified the way environment variables are structured for improved clarity and management. - **Dependency Updates** - Added new dependencies to improve functionality. - Upgraded several existing dependencies to boost performance and enhance output formatting. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Nikhil Simha <[email protected]>
Summary
^^^
Ran both AWS and GCP integration tests.
Following this PR, i'll work on rebasing #549 which applies the confs to the submitter interface.
Previously, to compile it would be:
but now
--confor--input-pathis not required, just:and this compiles the entire repo.
In addition, the compiled files gets persisted in the
compiledfolder. No longer are compiled files persisted by environment likeproduction/...Checklist
GCP and AWS integration tests pass
Summary by CodeRabbit
Summary by CodeRabbit
New Features
Improvements
Dependency Updates