Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Executions queries #988

Merged
merged 13 commits into from
Dec 25, 2024
Merged

Executions queries #988

merged 13 commits into from
Dec 25, 2024

Conversation

whiterabbit1983
Copy link
Contributor

@whiterabbit1983 whiterabbit1983 commented Dec 24, 2024

PR Type

Enhancement, Bug fix


Description

Major refactoring of execution queries to migrate from Cozo to PostgreSQL:

  • Migrated all execution-related queries to use PostgreSQL instead of Cozo
  • Implemented proper error handling with specific HTTP exceptions
  • Added new table 'latest_executions' for optimized queries
  • Simplified execution transitions by removing async versions
  • Combined multiple queries into single efficient PostgreSQL queries
  • Updated test suite to work with PostgreSQL and async/await
  • Improved code organization and reduced complexity
  • Fixed various bugs in execution handling and transitions
  • Added proper connection pooling support

Changes walkthrough 📝

Relevant files
Enhancement
13 files
__init__.py
Simplify execution transitions module imports                       

agents-api/agents_api/queries/executions/init.py

  • Removed async version of create_execution_transition
  • Removed update_execution import
  • +1/-5     
    count_executions.py
    Refactor count executions query for PostgreSQL                     

    agents-api/agents_api/queries/executions/count_executions.py

  • Changed SQL query to use latest_executions table
  • Updated return type signature to include fetch mode
  • +5/-9     
    create_execution.py
    Migrate execution creation to PostgreSQL                                 

    agents-api/agents_api/queries/executions/create_execution.py

  • Converted query to use PostgreSQL instead of Cozo
  • Added proper error handling and status updates
  • Implemented execution creation with proper metadata
  • +68/-54 
    create_execution_transition.py
    Convert execution transition creation to PostgreSQL           

    agents-api/agents_api/queries/executions/create_execution_transition.py

  • Migrated from Cozo to PostgreSQL
  • Simplified transition creation logic
  • Removed execution status update functionality
  • +104/-206
    create_temporal_lookup.py
    Migrate temporal lookup creation to PostgreSQL                     

    agents-api/agents_api/queries/executions/create_temporal_lookup.py

  • Converted query to PostgreSQL format
  • Simplified error handling
  • Removed unnecessary developer verification
  • +41/-49 
    get_execution.py
    Enhance execution retrieval with PostgreSQL                           

    agents-api/agents_api/queries/executions/get_execution.py

  • Updated query to use latest_executions table
  • Added proper error handling for no data found
  • Fixed execution ID mapping in transform function
  • +17/-19 
    get_execution_transition.py
    Convert execution transition retrieval to PostgreSQL         

    agents-api/agents_api/queries/executions/get_execution_transition.py

  • Migrated query to PostgreSQL
  • Added proper error handling
  • Simplified transition data transformation
  • +40/-50 
    get_paused_execution_token.py
    Migrate paused execution token retrieval to PostgreSQL     

    agents-api/agents_api/queries/executions/get_paused_execution_token.py

  • Converted to PostgreSQL query
  • Added proper error handling
  • Removed unnecessary status validation
  • +34/-50 
    list_executions.py
    Add PostgreSQL-based execution listing functionality         

    agents-api/agents_api/queries/executions/list_executions.py

  • Added new file for listing executions
  • Implemented PostgreSQL query with sorting and pagination
  • Added proper error handling
  • +76/-0   
    lookup_temporal_data.py
    Add PostgreSQL-based temporal data lookup                               

    agents-api/agents_api/queries/executions/lookup_temporal_data.py

  • Added new file for temporal data lookup
  • Implemented PostgreSQL query
  • Added proper error handling
  • +37/-0   
    prepare_execution_input.py
    Optimize execution input preparation with PostgreSQL         

    agents-api/agents_api/queries/executions/prepare_execution_input.py

  • Converted to PostgreSQL query
  • Simplified data preparation logic
  • Combined multiple queries into single query
  • +64/-188
    update_execution.py
    Remove separate execution update functionality                     

    agents-api/agents_api/queries/executions/update_execution.py

    • Removed file as functionality merged into transitions
    +0/-130 
    create_task_execution.py
    Update task execution creation with new database approach

    agents-api/agents_api/routers/tasks/create_task_execution.py

  • Updated to use connection pool instead of client
  • Changed error handling to use transitions
  • Removed update execution dependency
  • +12/-10 
    Tests
    2 files
    fixtures.py
    Update test fixtures for PostgreSQL migration                       

    agents-api/tests/fixtures.py

  • Updated test fixtures for PostgreSQL
  • Added execution and transition test fixtures
  • Fixed imports and dependencies
  • +70/-66 
    test_execution_queries.py
    Convert execution query tests to PostgreSQL                           

    agents-api/tests/test_execution_queries.py

  • Updated tests for PostgreSQL queries
  • Added async/await syntax
  • Fixed test assertions and error handling
  • +159/-152

    💡 PR-Agent usage: Comment /help "your question" on any pull request to receive relevant information

    @whiterabbit1983 whiterabbit1983 changed the base branch from dev to f/switch-to-pg December 24, 2024 12:11
    Copy link

    gitguardian bot commented Dec 24, 2024

    ️✅ There are no secrets present in this pull request anymore.

    If these secrets were true positive and are still valid, we highly recommend you to revoke them.
    While these secrets were previously flagged, we no longer have a reference to the
    specific commits where they were detected. Once a secret has been leaked into a git
    repository, you should consider it compromised, even if it was deleted immediately.
    Find here more information about risks.


    🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.

    Copy link
    Contributor

    qodo-merge-pro-for-open-source bot commented Dec 24, 2024

    CI Failure Feedback 🧐

    (Checks updated until commit ce47442)

    Action: Typecheck

    Failed stage: Typecheck [❌]

    Failure summary:

    The pytype type checking action failed due to multiple import and type annotation errors:

  • Import error in agents_api.common.utils.cozo: Can't find module pycozo
  • Type annotation mismatch in agents_api.queries.docs.search_docs_hybrid
  • Import error in agents_api.activities.execute_integration: Can't find module models.tools
  • Import error in agents_api.routers.healthz.check_health: Can't find module router
  • Import error in agents_api.routers.tasks.patch_execution: Can't find module
    queries.executions.update_execution

    The root cause appears to be missing dependencies and incorrect module imports in the codebase.

  • Relevant error logs:
    1:  ##[group]Operating System
    2:  Ubuntu
    ...
    
    1209:  [15/373] check agents_api.autogen.Tasks
    1210:  [16/373] check agents_api.autogen.Agents
    1211:  [17/373] check agents_api.common.exceptions.__init__
    1212:  [18/373] check agents_api.clients.__init__
    1213:  [19/373] check agents_api.exceptions
    1214:  [20/373] check agents_api.app
    1215:  [21/373] check agents_api.autogen.openapi_model
    1216:  [22/373] check agents_api.common.utils.cozo
    1217:  FAILED: /home/runner/work/julep/julep/agents-api/.pytype/pyi/agents_api/common/utils/cozo.pyi 
    1218:  /home/runner/work/julep/julep/agents-api/.venv/bin/python -m pytype.main --disable pyi-error --imports_info /home/runner/work/julep/julep/agents-api/.pytype/imports/agents_api.common.utils.cozo.imports --module-name agents_api.common.utils.cozo --platform linux -V 3.12 -o /home/runner/work/julep/julep/agents-api/.pytype/pyi/agents_api/common/utils/cozo.pyi --analyze-annotated --nofail --none-is-not-bool --quick --strict-none-binding /home/runner/work/julep/julep/agents-api/agents_api/common/utils/cozo.py
    1219:  /home/runner/work/julep/julep/agents-api/agents_api/common/utils/cozo.py:9:1: error: in <module>: Can't find module 'pycozo'. [import-error]
    1220:  from pycozo import Client
    1221:  ~~~~~~~~~~~~~~~~~~~~~~~~~
    1222:  For more details, see https://google.github.io/pytype/errors.html#import-error
    1223:  [23/373] check agents_api.worker.codec
    1224:  [24/373] check agents_api.clients.async_s3
    1225:  [25/373] check agents_api.common.exceptions.tools
    1226:  [26/373] check agents_api.common.protocol.remote
    1227:  [27/373] check agents_api.common.protocol.developers
    1228:  [28/373] check agents_api.queries.utils
    1229:  ERROR:pytype.matcher Invalid type: <class 'pytype.abstract.function.ParamSpecMatch'>
    ...
    
    1239:  [38/373] check agents_api.common.utils.messages
    1240:  [39/373] check agents_api.common.protocol.sessions
    1241:  [40/373] check agents_api.common.interceptors
    1242:  [41/373] check agents_api.queries.sessions.get_session
    1243:  [42/373] check agents_api.queries.entries.get_history
    1244:  [43/373] check agents_api.common.retry_policies
    1245:  [44/373] check agents_api.routers.docs.router
    1246:  [45/373] check agents_api.queries.docs.search_docs_hybrid
    1247:  FAILED: /home/runner/work/julep/julep/agents-api/.pytype/pyi/agents_api/queries/docs/search_docs_hybrid.pyi 
    1248:  /home/runner/work/julep/julep/agents-api/.venv/bin/python -m pytype.main --disable pyi-error --imports_info /home/runner/work/julep/julep/agents-api/.pytype/imports/agents_api.queries.docs.search_docs_hybrid.imports --module-name agents_api.queries.docs.search_docs_hybrid --platform linux -V 3.12 -o /home/runner/work/julep/julep/agents-api/.pytype/pyi/agents_api/queries/docs/search_docs_hybrid.pyi --analyze-annotated --nofail --none-is-not-bool --quick --strict-none-binding /home/runner/work/julep/julep/agents-api/agents_api/queries/docs/search_docs_hybrid.py
    1249:  /home/runner/work/julep/julep/agents-api/agents_api/queries/docs/search_docs_hybrid.py:50:1: error: in <module>: Type annotation for embedding does not match type of assignment [annotation-type-mismatch]
    ...
    
    1346:  metadata_filter,
    1347:  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    1348:  search_language,
    1349:  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    1350:  ],
    1351:  ~~~~~~~~~~
    1352:  )
    1353:  ~~~~~
    1354:  For more details, see https://google.github.io/pytype/errors.html#annotation-type-mismatch
    ...
    
    1356:  [47/373] check agents_api.queries.docs.search_docs_by_embedding
    1357:  [48/373] check agents_api.common.utils.json
    1358:  [49/373] check agents_api.dependencies.developer_id
    1359:  [50/373] check agents_api.queries.docs.mmr
    1360:  [51/373] check agents_api.common.utils.yaml
    1361:  [52/373] check agents_api.activities.task_steps.get_value_step
    1362:  [53/373] check agents_api.queries.executions.create_execution_transition
    1363:  [54/373] check agents_api.activities.execute_integration
    1364:  FAILED: /home/runner/work/julep/julep/agents-api/.pytype/pyi/agents_api/activities/execute_integration.pyi 
    1365:  /home/runner/work/julep/julep/agents-api/.venv/bin/python -m pytype.main --disable pyi-error --imports_info /home/runner/work/julep/julep/agents-api/.pytype/imports/agents_api.activities.execute_integration.imports --module-name agents_api.activities.execute_integration --platform linux -V 3.12 -o /home/runner/work/julep/julep/agents-api/.pytype/pyi/agents_api/activities/execute_integration.pyi --analyze-annotated --nofail --none-is-not-bool --quick --strict-none-binding /home/runner/work/julep/julep/agents-api/agents_api/activities/execute_integration.py
    1366:  /home/runner/work/julep/julep/agents-api/agents_api/activities/execute_integration.py:11:1: error: in <module>: Can't find module 'models.tools'. [import-error]
    1367:  from ..models.tools import get_tool_args_from_metadata
    1368:  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    1369:  For more details, see https://google.github.io/pytype/errors.html#import-error
    ...
    
    1533:  [218/373] check agents_api.common.__init__
    1534:  [219/373] check agents_api.dependencies.__init__
    1535:  [220/373] check agents_api.common.utils.__init__
    1536:  [221/373] check tests.test_execution_workflow
    1537:  [222/373] check agents_api.routers.agents.list_agent_tools
    1538:  [223/373] check agents_api.rec_sum.entities
    1539:  [224/373] check tests.sample_tasks.test_find_selector
    1540:  [225/373] check tests.test_workflow_routes
    1541:  /home/runner/work/julep/julep/agents-api/tests/test_workflow_routes.py:65:1: error: : Stray type comment: object [ignored-type-comment]
    1542:  #   type: object~~~~~~~~~~~~~~~
    1543:  #   type: object
    1544:  /home/runner/work/julep/julep/agents-api/tests/test_workflow_routes.py:114:1: error: : Stray type comment: object [ignored-type-comment]
    1545:  #   type: object~~~~~~~~~~~~~~~
    1546:  #   type: object
    1547:  For more details, see https://google.github.io/pytype/errors.html#ignored-type-comment
    1548:  [226/373] check agents_api.rec_sum.summarize
    1549:  [227/373] check agents_api.routers.healthz.check_health
    1550:  FAILED: /home/runner/work/julep/julep/agents-api/.pytype/pyi/agents_api/routers/healthz/check_health.pyi 
    1551:  /home/runner/work/julep/julep/agents-api/.venv/bin/python -m pytype.main --disable pyi-error --imports_info /home/runner/work/julep/julep/agents-api/.pytype/imports/agents_api.routers.healthz.check_health.imports --module-name agents_api.routers.healthz.check_health --platform linux -V 3.12 -o /home/runner/work/julep/julep/agents-api/.pytype/pyi/agents_api/routers/healthz/check_health.pyi --analyze-annotated --nofail --none-is-not-bool --quick --strict-none-binding /home/runner/work/julep/julep/agents-api/agents_api/routers/healthz/check_health.py
    1552:  /home/runner/work/julep/julep/agents-api/agents_api/routers/healthz/check_health.py:5:1: error: in <module>: Can't find module 'router'. [import-error]
    1553:  from .router import router
    1554:  ~~~~~~~~~~~~~~~~~~~~~~~~~~
    1555:  For more details, see https://google.github.io/pytype/errors.html#import-error
    1556:  [228/373] check agents_api.queries.sessions.__init__
    1557:  [229/373] check agents_api.routers.tasks.get_execution_details
    1558:  [230/373] check agents_api.routers.agents.patch_agent_tool
    1559:  [231/373] check agents_api.activities.__init__
    1560:  [232/373] check tests.__init__
    1561:  [233/373] check agents_api.common.exceptions.agents
    1562:  [234/373] check agents_api.routers.tasks.patch_execution
    1563:  FAILED: /home/runner/work/julep/julep/agents-api/.pytype/pyi/agents_api/routers/tasks/patch_execution.pyi 
    1564:  /home/runner/work/julep/julep/agents-api/.venv/bin/python -m pytype.main --disable pyi-error --imports_info /home/runner/work/julep/julep/agents-api/.pytype/imports/agents_api.routers.tasks.patch_execution.imports --module-name agents_api.routers.tasks.patch_execution --platform linux -V 3.12 -o /home/runner/work/julep/julep/agents-api/.pytype/pyi/agents_api/routers/tasks/patch_execution.pyi --analyze-annotated --nofail --none-is-not-bool --quick --strict-none-binding /home/runner/work/julep/julep/agents-api/agents_api/routers/tasks/patch_execution.py
    1565:  /home/runner/work/julep/julep/agents-api/agents_api/routers/tasks/patch_execution.py:11:1: error: in <module>: Can't find module 'queries.executions.update_execution'. [import-error]
    1566:  from ...queries.executions.update_execution import (
    1567:  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    1568:  update_execution as update_execution_query,
    1569:  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    1570:  )
    1571:  ~
    1572:  For more details, see https://google.github.io/pytype/errors.html#import-error
    1573:  ninja: build stopped: cannot make progress due to previous errors.
    1574:  Computing dependencies
    1575:  Generated API key since not set in the environment: 89879384487152837339775707013338
    1576:  Sentry DSN not found. Sentry will not be enabled.
    1577:  Analyzing 348 sources with 0 local dependencies
    1578:  Leaving directory '.pytype'
    1579:  ##[error]Process completed with exit code 1.
    

    ✨ CI feedback usage guide:

    The CI feedback tool (/checks) automatically triggers when a PR has a failed check.
    The tool analyzes the failed checks and provides several feedbacks:

    • Failed stage
    • Failed test name
    • Failure summary
    • Relevant error logs

    In addition to being automatically triggered, the tool can also be invoked manually by commenting on a PR:

    /checks "https://github.com/{repo_name}/actions/runs/{run_number}/job/{job_number}"
    

    where {repo_name} is the name of the repository, {run_number} is the run number of the failed check, and {job_number} is the job number of the failed check.

    Configuration options

    • enable_auto_checks_feedback - if set to true, the tool will automatically provide feedback when a check is failed. Default is true.
    • excluded_checks_list - a list of checks to exclude from the feedback, for example: ["check1", "check2"]. Default is an empty list.
    • enable_help_text - if set to true, the tool will provide a help message with the feedback. Default is true.
    • persistent_comment - if set to true, the tool will overwrite a previous checks comment with the new feedback. Default is true.
    • final_update_message - if persistent_comment is true and updating a previous checks message, the tool will also create a new message: "Persistent checks updated to latest commit". Default is true.

    See more information about the checks tool in the docs.

    @@ -45,8 +45,8 @@
    async def get_execution(
    *,
    execution_id: UUID,
    ) -> tuple[str, dict]:
    ) -> tuple[str, list]:
    return (
    Copy link
    Contributor

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    you can add a third return value "fetchrow" and add rewrap_exception with asyncpg.exceptions.NoDataFoundError, this will return 404 in case the execution doesn't exist.

    Make sure to refactor the return type of the function, and you can look at get_agent to see an example.

    ]

    return (queries, {"task_token": task_token, "transition_id": transition_id})
    return (
    Copy link
    Contributor

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    same comment, you can add "fetchrow"

    # :assert some
    # """

    return (
    Copy link
    Contributor

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    same comment, you can add "fetchrow"


    :limit 1
    """
    execution_id = str(execution_id)

    return (
    Copy link
    Contributor

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    same comment, you can add "fetchrow"

    Comment on lines +57 to +66
    return (
    sql_query,
    Copy link
    Contributor

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    before returning you could catch 2 exceptions where limit and offset values are out of bounds (negative number for limit for example)

    @Ahmad-mtos Ahmad-mtos marked this pull request as ready for review December 25, 2024 14:07
    Copy link
    Contributor

    PR Reviewer Guide 🔍

    Here are some key observations to aid the review process:

    ⏱️ Estimated effort to review: 4 🔵🔵🔵🔵⚪
    🧪 PR contains tests
    🔒 Security concerns

    SQL injection:
    The SQL queries use parameterized queries with $1, $2 etc. placeholders which is good practice, but some queries like in prepare_execution_input.py build complex dynamic queries that should be carefully reviewed to ensure no SQL injection vulnerabilities are introduced through string concatenation.

    ⚡ Recommended focus areas for review

    Data Validation

    The execution creation logic should validate the input data before inserting into database. Missing validation could lead to data integrity issues.

        data.metadata = data.metadata or {}
        execution_data = data.model_dump()
    else:
        data["metadata"] = data.get("metadata", {})
        execution_data = data
    Error Handling

    The transition creation does not properly validate the transition state changes. Invalid transitions could corrupt execution flow.

    transition_data = data.model_dump(exclude_unset=True, exclude={"id"})
    
    # Parse the current and next targets
    validate_transition_targets(data)
    current_target = transition_data.pop("current")
    next_target = transition_data.pop("next")
    
    transition_data["current"] = (current_target["workflow"], current_target["step"])
    transition_data["next"] = next_target and (
        next_target["workflow"],
        next_target["step"],
    )
    Connection Management

    The connection pool is passed around but error handling and cleanup of connections is not clearly implemented. Could lead to connection leaks.

    async def start_execution(
        *,
        developer_id: UUID,
        task_id: UUID,
        data: CreateExecutionRequest,
        connection_pool=None,
    ) -> tuple[Execution, WorkflowHandle]:
        execution_id = uuid7()
    
        execution = await create_execution_query(
            developer_id=developer_id,
            task_id=task_id,
            execution_id=execution_id,
            data=data,
            connection_pool=connection_pool,
        )

    Copy link
    Contributor

    PR Code Suggestions ✨

    Explore these optional code suggestions:

    CategorySuggestion                                                                                                                                    Score
    Possible issue
    Properly handle error transitions by including required workflow state information

    The error transition creation is missing required current/next step information
    which will cause validation errors. Add proper workflow step information for error
    transitions.

    agents-api/agents_api/routers/tasks/create_task_execution.py [79-86]

     await create_execution_transition(
         developer_id=developer_id,
         execution_id=execution_id,
         data=CreateTransitionRequest(
             type="error",
    +        current={"workflow": "main", "step": 0},
    +        next=None,
    +        output=str(e)
         ),
         connection_pool=connection_pool,
     )
    • Apply this suggestion
    Suggestion importance[1-10]: 9

    Why: The suggestion fixes a critical issue where error transitions would fail validation due to missing required workflow state information, potentially causing the error handling to fail silently.

    9
    Implement execution status validation to prevent unauthorized token access

    The commented out status check query is critical for validating execution state
    before retrieving the token. Implement proper status validation to prevent
    retrieving tokens for non-paused executions.

    agents-api/agents_api/queries/executions/get_paused_execution_token.py [43-55]

    -# TODO: what to do with this query?
    -# check_status_query = """
    -# ?[execution_id, status] :=
    -#     *executions:execution_id_status_idx {
    -#         execution_id,
    -#         status,
    -#     },
    -#     execution_id = to_uuid($execution_id),
    -#     status = "awaiting_input"
    +status_query = """
    +SELECT status FROM latest_executions 
    +WHERE execution_id = $1 AND status = 'awaiting_input'
    +LIMIT 1;
    +"""
    +await verify_execution_status(status_query, execution_id)
    • Apply this suggestion
    Suggestion importance[1-10]: 8

    Why: The suggestion addresses an important security and data integrity issue by ensuring tokens are only retrieved for executions in the correct paused state, preventing potential misuse.

    8
    Ensure proper initialization of workflow step tracking fields in database transitions

    Add validation to ensure step_definition and step_label are properly set in the SQL
    query parameters. Currently they are hardcoded as empty dict and None which could
    cause issues with workflow execution tracking.

    agents-api/agents_api/queries/executions/create_execution_transition.py [142-157]

     return (
         sql_query,
         [
             execution_id,
             transition_id,
             data.type,
    -        {},
    -        None,
    +        transition_data.get("step_definition", {}),
    +        transition_data.get("step_label"),
             transition_data["current"],
    -        transition_data["next"],
    +        transition_data["next"], 
             data.output,
             task_token,
             data.metadata,
         ],
         "fetchrow",
     )
    • Apply this suggestion
    Suggestion importance[1-10]: 7

    Why: The suggestion addresses a potential data integrity issue by properly initializing step tracking fields from transition data instead of using hardcoded empty values, which could affect workflow execution tracking.

    7
    Add proper error handling for database operations and data validation

    The error handling decorators are commented out, leaving the function without proper
    exception handling for database errors and validation failures.

    agents-api/agents_api/queries/executions/create_temporal_lookup.py [35-42]

    -# @rewrap_exceptions(
    -#     {
    -#         AssertionError: partialclass(HTTPException, status_code=404),
    -#         QueryException: partialclass(HTTPException, status_code=400),
    -#         ValidationError: partialclass(HTTPException, status_code=400),
    -#         TypeError: partialclass(HTTPException, status_code=400),
    -#     }
    -# )
    +@rewrap_exceptions(
    +    {
    +        asyncpg.exceptions.UniqueViolationError: partialclass(HTTPException, status_code=409),
    +        asyncpg.exceptions.ForeignKeyViolationError: partialclass(HTTPException, status_code=404),
    +        ValidationError: partialclass(HTTPException, status_code=400),
    +        TypeError: partialclass(HTTPException, status_code=400),
    +    }
    +)
    • Apply this suggestion
    Suggestion importance[1-10]: 7

    Why: The suggestion improves code robustness by adding specific error handling for database operations, which is important for proper error reporting and system stability.

    7
    Security
    Add proper developer ID validation to ensure secure data access

    The developer_id parameter is marked with a TODO comment but not used in the query.
    Either remove the unused parameter or implement proper developer ID validation to
    ensure data access security.

    agents-api/agents_api/queries/executions/get_execution_transition.py [52-57]

     async def get_execution_transition(
         *,
    -    developer_id: UUID,  # TODO: what to do with this parameter?
    +    developer_id: UUID,
         transition_id: UUID | None = None,
         task_token: str | None = None,
     ) -> tuple[str, list, Literal["fetch", "fetchmany", "fetchrow"]]:
    +    # Verify developer has access to this transition
    +    await verify_developer_access(developer_id, transition_id)
    • Apply this suggestion
    Suggestion importance[1-10]: 9

    Why: The suggestion addresses a critical security concern by proposing proper validation of developer_id before allowing access to transition data, which is essential for preventing unauthorized data access.

    9

    Copy link
    Contributor

    @ellipsis-dev ellipsis-dev bot left a comment

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    👍 Looks good to me! Reviewed everything up to ce47442 in 1 minute and 47 seconds

    More details
    • Looked at 2318 lines of code in 17 files
    • Skipped 0 files when reviewing.
    • Skipped posting 5 drafted comments based on config settings.
    1. agents-api/agents_api/queries/executions/create_execution.py:94
    • Draft comment:
      Ensure execution_data contains the input key to avoid potential KeyError.
                execution_data.get("input", {})
    
    • Reason this comment was not posted:
      Decided after close inspection that this draft comment was likely wrong and/or not actionable:
      The key insight is that data is annotated with dict_like(CreateExecutionRequest), which means it must match the CreateExecutionRequest schema. If 'input' was optional, it would be marked as such in the schema. Since it's being used directly without any optional handling, it's likely a required field in CreateExecutionRequest. The type system would catch missing required fields.
      I don't have direct access to the CreateExecutionRequest schema definition, so I'm making assumptions about the required fields. The type checking could be disabled at runtime.
      The @beartype decorator enforces runtime type checking, and dict_like validation would fail if required fields were missing. The code is already properly validated.
      The comment should be deleted because 'input' is a required field that's validated by both static typing and runtime checks. Using .get() would hide real errors.
    2. agents-api/agents_api/queries/executions/create_execution_transition.py:148
    • Draft comment:
      Ensure step_definition and step_label are correctly set if needed.
    transition_data.get("step_definition", {}),
    transition_data.get("step_label", None),
    
    • Reason this comment was not posted:
      Comment did not seem useful.
    3. agents-api/agents_api/queries/executions/get_execution_transition.py:54
    • Draft comment:
      Remove unused developer_id parameter if not needed.
    • Reason this comment was not posted:
      Confidence changes required: 50%
      In get_execution_transition.py, the developer_id parameter is not used. This could be a potential oversight or a placeholder for future use. It should be removed if not needed.
    4. agents-api/agents_api/queries/executions/get_paused_execution_token.py:38
    • Draft comment:
      Remove unused developer_id parameter if not needed.
    • Reason this comment was not posted:
      Confidence changes required: 50%
      In get_paused_execution_token.py, the developer_id parameter is not used. This could be a potential oversight or a placeholder for future use. It should be removed if not needed.
    5. agents-api/agents_api/queries/executions/lookup_temporal_data.py:31
    • Draft comment:
      Remove unused developer_id parameter if not needed.
    • Reason this comment was not posted:
      Confidence changes required: 50%
      In lookup_temporal_data.py, the developer_id parameter is not used. This could be a potential oversight or a placeholder for future use. It should be removed if not needed.

    Workflow ID: wflow_YaA2hQk6Dvyv1eqf


    You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

    @Ahmad-mtos Ahmad-mtos merged commit 857bb96 into f/switch-to-pg Dec 25, 2024
    7 of 11 checks passed
    @Ahmad-mtos Ahmad-mtos deleted the f/executions-queries branch December 25, 2024 14:10
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Projects
    None yet
    Development

    Successfully merging this pull request may close these issues.

    2 participants