Skip to content

feat(tdl): Add Spider TDL component; Add boilerplate for TDL parser's AST node. - #175

Merged
LinZhihao-723 merged 4 commits into
y-scope:mainfrom
LinZhihao-723:tdl-ast
Aug 5, 2025
Merged

feat(tdl): Add Spider TDL component; Add boilerplate for TDL parser's AST node.#175
LinZhihao-723 merged 4 commits into
y-scope:mainfrom
LinZhihao-723:tdl-ast

Conversation

@LinZhihao-723

@LinZhihao-723 LinZhihao-723 commented Aug 5, 2025

Copy link
Copy Markdown
Member

Description

This PR serves as a boilerplate for TDL (task definition language) as a new component for Spider, with a simple declaration for its parser AST node.

This component is presented as a new target in the cmake, spider_tdl. We introduce a simple unit test case to ensure this component can be successfully linked by the unit test executable.

Checklist

  • The PR satisfies the contribution guidelines.
  • This is a breaking change and that has been indicated in the PR title, OR this isn't a
    breaking change.
  • Necessary docs have been updated, OR no docs need to be updated.

Validation performed

  • Ensure all workflows passed.
  • Ensure the new cmake target can be successfully linked and used in the unit tests.

Summary by CodeRabbit

  • New Features
    • Introduced a new library component for the TDL parser, including an abstract base class for AST nodes and integrated error handling.
  • Tests
    • Added new unit tests to verify error handling for AST node classes.
  • Chores
    • Updated build configuration to include the new library and corresponding tests.

@LinZhihao-723
LinZhihao-723 requested review from a team and sitaowang1998 as code owners August 5, 2025 02:45
@coderabbitai

coderabbitai Bot commented Aug 5, 2025

Copy link
Copy Markdown
Contributor

Walkthrough

A new library target, spider_tdl, has been introduced for the "spider task definition language" component. This includes new source and header files for AST node error handling, integration with an external error-handling library, and corresponding unit tests. Build scripts for both the main project and tests were updated to include these additions.

Changes

Cohort / File(s) Change Summary
Build System Updates
src/spider/CMakeLists.txt, tests/CMakeLists.txt
Added new spider_tdl library target, its sources, headers, dependencies, and an alias; updated test build to include the new library and test file.
AST Node Error Handling Implementation
src/spider/tdl/parser/ast/Node.cpp, src/spider/tdl/parser/ast/Node.hpp
Introduced an abstract Node class with error code enum and error handling integration; implemented error category specialization for AST node errors.
Unit Test for AST Node
tests/tdl/test-parser-ast.cpp
Added a test verifying error handling behaviour of the new AST Node class using the Catch2 framework.

Sequence Diagram(s)

sequenceDiagram
    participant TestCase as test-parser-ast.cpp
    participant Node as spider::tdl::parser::ast::Node
    participant ErrorHandling as ystdlib::error_handling
    TestCase->>Node: Create ErrorCode with PlaceholderError
    TestCase->>ErrorHandling: Wrap ErrorCode in Result<void>
    TestCase->>ErrorHandling: Query Result<void> for error state
    ErrorHandling-->>TestCase: Returns has_error() == true
Loading

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~8 minutes

Suggested reviewers

  • davidlion

Note

⚡️ Unit Test Generation is now available in beta!

Learn more here, or try it out under "Finishing Touches" below.


📜 Recent review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 16f1dc0 and 9669e2b.

📒 Files selected for processing (1)
  • src/spider/tdl/parser/ast/Node.cpp (1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • src/spider/tdl/parser/ast/Node.cpp
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
  • GitHub Check: lint
  • GitHub Check: non-storage-unit-tests (ubuntu-24.04)
  • GitHub Check: non-storage-unit-tests (ubuntu-22.04)
✨ Finishing Touches
  • 📝 Generate Docstrings
🧪 Generate unit tests
  • Create PR with unit tests
  • Post copyable unit tests in a comment

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Explain this complex logic.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai explain this code block.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and explain its main purpose.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR.
  • @coderabbitai generate sequence diagram to generate a sequence diagram of the changes in this PR.
  • @coderabbitai generate unit tests to generate unit tests for this PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between aa0100b and 16f1dc0.

📒 Files selected for processing (5)
  • src/spider/CMakeLists.txt (1 hunks)
  • src/spider/tdl/parser/ast/Node.cpp (1 hunks)
  • src/spider/tdl/parser/ast/Node.hpp (1 hunks)
  • tests/CMakeLists.txt (2 hunks)
  • tests/tdl/test-parser-ast.cpp (1 hunks)
🧰 Additional context used
🧠 Learnings (4)
📓 Common learnings
Learnt from: davidlion
PR: y-scope/spider#100
File: src/spider/worker/worker.cpp:205-230
Timestamp: 2025-04-09T17:15:24.552Z
Learning: Documentation should be added to new functions in the spider codebase, as already discussed with the user.
Learnt from: sitaowang1998
PR: y-scope/spider#172
File: build-tasks.yaml:21-45
Timestamp: 2025-08-03T01:49:27.764Z
Learning: In the spider project, external tools like `uv`, Python, and Task are documented as requirements in README.md and are expected to be pre-installed by users rather than bootstrapped by the build system.
Learnt from: sitaowang1998
PR: y-scope/spider#168
File: lint-tasks.yaml:62-65
Timestamp: 2025-07-17T19:44:06.132Z
Learning: ANTLR-generated C++ code in the G_SRC_DSL_DIR (src/stdl) should not be included in linting tasks because it's auto-generated code that doesn't follow manual coding standards.
📚 Learning: antlr-generated c++ code in the g_src_dsl_dir (src/stdl) should not be included in linting tasks bec...
Learnt from: sitaowang1998
PR: y-scope/spider#168
File: lint-tasks.yaml:62-65
Timestamp: 2025-07-17T19:44:06.132Z
Learning: ANTLR-generated C++ code in the G_SRC_DSL_DIR (src/stdl) should not be included in linting tasks because it's auto-generated code that doesn't follow manual coding standards.

Applied to files:

  • tests/CMakeLists.txt
  • src/spider/CMakeLists.txt
📚 Learning: in the spider project, internal dependencies between packages (like spider-client depending on spide...
Learnt from: sitaowang1998
PR: y-scope/spider#172
File: python/spider/client/pyproject.toml:6-7
Timestamp: 2025-08-03T01:52:26.847Z
Learning: In the spider project, internal dependencies between packages (like spider-client depending on spider-core) are managed without version pinning since they are developed together in the same repository and versions are kept synchronized.

Applied to files:

  • tests/CMakeLists.txt
  • src/spider/CMakeLists.txt
📚 Learning: documentation should be added to new functions in the spider codebase, as already discussed with the...
Learnt from: davidlion
PR: y-scope/spider#100
File: src/spider/worker/worker.cpp:205-230
Timestamp: 2025-04-09T17:15:24.552Z
Learning: Documentation should be added to new functions in the spider codebase, as already discussed with the user.

Applied to files:

  • src/spider/CMakeLists.txt
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
  • GitHub Check: non-storage-unit-tests (ubuntu-22.04)
  • GitHub Check: non-storage-unit-tests (ubuntu-24.04)
  • GitHub Check: lint
🔇 Additional comments (4)
tests/CMakeLists.txt (1)

5-5: LGTM! Clean integration of TDL test component.

The addition of the new test source file and library dependency follows the established patterns in the CMake configuration and properly integrates the TDL component into the test build system.

Also applies to: 43-43

tests/tdl/test-parser-ast.cpp (1)

1-19: LGTM! Well-structured unit test for AST node error handling.

The test properly validates the integration between the Node class and the error handling system. The comprehensive NOLINT directives and clean test structure follow established patterns in the codebase.

src/spider/tdl/parser/ast/Node.hpp (1)

9-22: LGTM! Appropriate foundational AST node structure.

The Node class provides a solid foundation for the TDL parser AST with proper error handling integration. The current minimal implementation is appropriate for boilerplate/foundational code as mentioned in the PR objectives.

src/spider/CMakeLists.txt (1)

198-217: LGTM! Well-structured CMake configuration for the new TDL library.

The spider_tdl library definition follows established patterns in the project, with proper separation of sources and headers, appropriate PUBLIC linkage for ystdlib::error_handling, and consistent alias target creation.

Comment thread src/spider/tdl/parser/ast/Node.cpp
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants