F/file routes: Fix failing file routes

[Vedantsahai18]

PR Type

Enhancement

---

Description

• Implemented centralized S3 client management using singleton pattern in app state
• Refactored file operations to use shared S3 client instance for better resource management
• Added proper cleanup of S3 resources in application lifecycle
• Replaced mock S3 client with localstack container for more realistic testing
• Updated dependencies to include localstack support

---

Changes walkthrough 📝

	Relevant files
Enhancement	app.py Add S3 client initialization and cleanup in app lifecycle agents-api/agents_api/app.py Added S3 client initialization in app lifespan Implemented proper cleanup of S3 client in finally block Added environment variables for S3 configuration +29/-5    async_s3.py Refactor S3 client to use singleton pattern                            agents-api/agents_api/clients/async_s3.py Refactored S3 client to use singleton instance from app state Removed redundant client creation in each function Simplified S3 operations by reusing the client +46/-75  *.py Update file routes to use shared S3 client                              agents-api/agents_api/routers/files/*.py Updated file operations to use shared S3 client Simplified file content handling for create/get/delete operations Added direct S3 bucket operations
Tests	fixtures.py Replace mock S3 client with localstack in tests                    agents-api/tests/fixtures.py Replaced mock S3 client with localstack container Added proper S3 client setup/teardown for tests +21/-4

---

💡 PR-Agent usage: Comment /help "your question" on any pull request to receive relevant information

---

Important

Refactor S3 client management using a singleton pattern and enhance testing with localstack.

• S3 Client Management:
  • Implemented singleton pattern for S3 client in app.py and async_s3.py.
  • Added S3 client initialization and cleanup in app.py lifecycle.
  • Refactored async_s3.py to use shared S3 client, removing redundant client creation.
• File Operations:
  • Updated file routes in routers/files/*.py to use shared S3 client.
  • Simplified file content handling for create/get/delete operations.
• Testing:
  • Replaced mock S3 client with localstack in fixtures.py for realistic testing.
  • Updated dependencies in pyproject.toml to include localstack support.

^{This description was created by for 32dfd5f. It will automatically update as commits are pushed.}

[qodo-merge-pro-for-open-source]

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪

🧪 PR contains tests

🔒 No security concerns identified

⚡ Recommended focus areas for review Resource Cleanup The S3 client cleanup in lifespan's finally block might not properly handle the aexit context. Consider using try/finally around the aenter call to ensure proper cleanup. app.state.s3_client = await session.create_client( "s3", aws_access_key_id=s3_access_key, aws_secret_access_key=s3_secret_key, endpoint_url=s3_endpoint, ).__aenter__() Debug Code Debug print statements were left in the code which should be removed before merging to production. print("-" * 100) print("CONTENT") print(content) print("-" * 100) Error Handling The setup() function's error handling for bucket creation could be improved to handle other potential S3 errors besides 404. try: await client.head_bucket(Bucket=blob_store_bucket) except botocore.exceptions.ClientError as e: if e.response["Error"]["Code"] == "404": await client.create_bucket(Bucket=blob_store_bucket) else: raise e

[ellipsis-dev]

❌ Changes requested. Reviewed everything up to 7913187 in 39 seconds

More details

• Looked at 630 lines of code in 13 files
• Skipped 0 files when reviewing.
• Skipped posting 1 drafted comments based on config settings.

1. agents-api/tests/test_files_routes.py:54

• Draft comment:
  Uncomment the test for the GET file route to ensure it's executed.

    response = make_request(
        method="GET",
        url=f"/files/{file_id}",
    )

    assert response.status_code == 404

• Reason this comment was not posted:
  Comment did not seem useful.

Workflow ID: wflow_HiNhTyxl2uNo5Ypt

---

Want Ellipsis to fix these issues? Tag @ellipsis-dev in a comment. You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

[ellipsis-dev]

Remove print statements used for debugging.

[qodo-merge-pro-for-open-source]

PR Code Suggestions ✨

Explore these optional code suggestions:

Category	Suggestion	Score
Security	✅ Remove debug logging of sensitive data Suggestion Impact: The commit removed the debug print statements that were exposing file content, exactly as suggested code diff: - print("-" * 100) - print("CONTENT") - print(content) - print("-" * 100) Remove debug print statements that expose potentially sensitive file content to logs. agents-api/agents_api/routers/files/get_file.py [23-26] -print("-" * 100) -print("CONTENT") -print(content) -print("-" * 100) +# Remove debug prints entirely Apply this suggestion Suggestion importance[1-10]: 9 Why: Removing debug prints that expose file content is critical for security as it prevents potential leakage of sensitive information in logs.	9
Possible issue	Add validation for required environment variables before initializing critical services Add error handling for missing S3 environment variables to prevent runtime errors. Check if required S3 credentials are present before initializing the client. agents-api/agents_api/app.py [26-32] s3_access_key = os.environ.get("S3_ACCESS_KEY") s3_secret_key = os.environ.get("S3_SECRET_KEY") s3_endpoint = os.environ.get("S3_ENDPOINT") + +if not all([s3_access_key, s3_secret_key, s3_endpoint]): + raise ValueError("Missing required S3 environment variables") if not getattr(app.state, "s3_client", None): session = get_session() app.state.s3_client = await session.create_client( Apply this suggestion Suggestion importance[1-10]: 8 Why: The suggestion adds crucial validation for required S3 credentials, preventing potential runtime errors and making the application fail fast if configuration is incomplete.	8
	Ensure proper cleanup of async context managers Properly handle the S3 client context manager exit by using aexit instead of just close() to ensure proper cleanup of resources. agents-api/agents_api/app.py [48-50] if getattr(app.state, "s3_client", None): - await app.state.s3_client.close() + await app.state.s3_client.__aexit__(None, None, None) app.state.s3_client = None Apply this suggestion Suggestion importance[1-10]: 7 Why: Using aexit instead of close() ensures proper cleanup of async resources and follows context manager protocol, preventing potential resource leaks.	7

[qodo-merge-pro-for-open-source]

CI Failure Feedback 🧐

Action: Test

Failed stage: Run tests [❌]

Failed test name: test_agent_queries

Failure summary: The tests failed because required database tables were not found: The agents table was missing, causing failures in multiple agent-related queries (create, update, create-or-update) The developers table was missing, causing a failure in the developer test The error "migrate: not found" suggests that database migrations were not run before the tests, which would have created these required tables

Relevant error logs: 1: ##[group]Operating System 2: Ubuntu ... 1286: Container started: 97aee7e7e863 1287: Waiting for container <Container: 97aee7e7e863> with image timescale/timescaledb-ha:pg17 to be ready ... 1288: Waiting for container <Container: 97aee7e7e863> with image timescale/timescaledb-ha:pg17 to be ready ... 1289: /bin/sh: 1: migrate: not found 1290: FAIL test_agent_queries:28 query: create agent sql 1% 1291: FAIL test_agent_queries:44 query: create or update agent sql 2% 1292: FAIL test_agent_queries:63 query: update agent sql 3% 1293: ─────────────────────────── query: create agent sql ──────────────────────────── 1294: Failed at tests/test_agent_queries.py ... 1313: │ │ 1314: │ /home/runner/.local/share/uv/python/cpython-3.12.8-linux-x86_64-gnu/lib/ │ 1315: │ python3.12/asyncio/runners.py:118 in run │ 1316: │ │ 1317: │ 115 │ │ │ 1318: │ 116 │ │ self._interrupt_count = 0 │ 1319: │ 117 │ │ try: │ 1320: │ ❱ 118 │ │ │ return self._loop.run_until_complete(task) │ 1321: │ 119 │ │ except exceptions.CancelledError: │ ... 1326: │ │ context = <_contextvars.Context object at 0x7fb495961240> │ │ 1327: │ │ coro = <coroutine object _ at 0x7fb49460a140> │ │ 1328: │ │ self = <asyncio.runners.Runner object at 0x7fb4947aac00> │ │ 1329: │ │ sigint_handler = functools.partial(<bound method Runner._on_sigint │ │ 1330: │ │ of <asyncio.runners.Runner object at │ │ 1331: │ │ 0x7fb4947aac00>>, main_task=<Task finished │ │ 1332: │ │ name='Task-1' coro=<_() done, defined at │ │ 1333: │ │ /home/runner/work/julep/julep/agents-api/tests/tes… │ │ 1334: │ │ exception=UndefinedTableError('relation "agents" │ │ 1335: │ │ does not exist')>) │ │ 1336: │ │ task = <Task finished name='Task-1' coro=<_() done, │ │ 1337: │ │ defined at │ │ 1338: │ │ /home/runner/work/julep/julep/agents-api/tests/tes… │ │ 1339: │ │ exception=UndefinedTableError('relation "agents" │ │ ... 1358: │ │ dsn = '***localhost:32769/test?sslmode=d… │ │ 1359: │ │ pool = <asyncpg.pool.Pool object at 0x7fb494662740> │ │ 1360: │ ╰──────────────────────────────────────────────────────────────────────╯ │ 1361: │ │ 1362: │ /home/runner/work/julep/julep/agents-api/agents_api/queries/utils.py:296 │ 1363: │ in async_wrapper │ 1364: │ │ 1365: │ 293 │ │ │ │ result: T = await func(*args, **kwargs) │ 1366: │ 294 │ │ │ except BaseException as error: │ 1367: │ 295 │ │ │ │ _check_error(error) │ 1368: │ ❱ 296 │ │ │ │ raise error │ ... 1391: │ │ 1392: │ /home/runner/work/julep/julep/agents-api/agents_api/queries/utils.py:293 │ 1393: │ in async_wrapper │ 1394: │ │ 1395: │ 290 │ │ @wraps(func) │ 1396: │ 291 │ │ async def async_wrapper(*args: P.args, **kwargs: P.kwargs) │ 1397: │ 292 │ │ │ try: │ 1398: │ ❱ 293 │ │ │ │ result: T = await func(*args, **kwargs) │ 1399: │ 294 │ │ │ except BaseException as error: │ 1400: │ 295 │ │ │ │ _check_error(error) │ 1401: │ 296 │ │ │ │ raise error │ ... 1525: │ │ │ │ │ │ {}, │ │ 1526: │ │ │ │ │ │ {} │ │ 1527: │ │ │ │ │ ], │ │ 1528: │ │ │ │ │ 'timeout': 90.0 │ │ 1529: │ │ │ │ } │ │ 1530: │ │ │ ) │ │ 1531: │ │ ] │ │ 1532: │ │ conn = <PoolConnectionProxy [released] 0x7fb4946d6bf0> │ │ 1533: │ │ connection_error = False │ │ ... 1544: │ │ │ │ model='gpt-4o-mini', │ │ 1545: │ │ │ │ instructions=[], │ │ 1546: │ │ │ │ default_settings=None │ │ 1547: │ │ │ ) │ │ 1548: │ │ } │ │ 1549: │ │ method = <bound method Connection.fetch of │ │ 1550: │ │ <PoolConnectionProxy [released] 0x7fb4946d6bf0>> │ │ 1551: │ │ method_name = 'fetch' │ │ 1552: │ │ only_on_error = False │ │ ... 1747: │ │ statement = None │ │ 1748: │ │ stmt_name = '__asyncpg_stmt_b__' │ │ 1749: │ │ timeout = 90.0 │ │ 1750: │ │ use_cache = True │ │ 1751: │ ╰──────────────────────────────────────────────────────────────────────╯ │ 1752: │ │ 1753: │ in prepare:165 │ 1754: ╰──────────────────────────────────────────────────────────────────────────╯ 1755: UndefinedTableError: relation "agents" does not exist 1756: ────────────────────── query: create or update agent sql ─────────────────────── 1757: Failed at tests/test_agent_queries.py ... 1776: │ │ 1777: │ /home/runner/.local/share/uv/python/cpython-3.12.8-linux-x86_64-gnu/lib/ │ 1778: │ python3.12/asyncio/runners.py:118 in run │ 1779: │ │ 1780: │ 115 │ │ │ 1781: │ 116 │ │ self._interrupt_count = 0 │ 1782: │ 117 │ │ try: │ 1783: │ ❱ 118 │ │ │ return self._loop.run_until_complete(task) │ 1784: │ 119 │ │ except exceptions.CancelledError: │ ... 1789: │ │ context = <_contextvars.Context object at 0x7fb495a820c0> │ │ 1790: │ │ coro = <coroutine object _ at 0x7fb494623780> │ │ 1791: │ │ self = <asyncio.runners.Runner object at 0x7fb4946d78f0> │ │ 1792: │ │ sigint_handler = functools.partial(<bound method Runner._on_sigint │ │ 1793: │ │ of <asyncio.runners.Runner object at │ │ 1794: │ │ 0x7fb4946d78f0>>, main_task=<Task finished │ │ 1795: │ │ name='Task-14' coro=<_() done, defined at │ │ 1796: │ │ /home/runner/work/julep/julep/agents-api/tests/tes… │ │ 1797: │ │ exception=UndefinedTableError('relation "agents" │ │ 1798: │ │ does not exist')>) │ │ 1799: │ │ task = <Task finished name='Task-14' coro=<_() done, │ │ 1800: │ │ defined at │ │ 1801: │ │ /home/runner/work/julep/julep/agents-api/tests/tes… │ │ 1802: │ │ exception=UndefinedTableError('relation "agents" │ │ ... 1821: │ │ dsn = '***localhost:32769/test?sslmode=d… │ │ 1822: │ │ pool = <asyncpg.pool.Pool object at 0x7fb494662f80> │ │ 1823: │ ╰──────────────────────────────────────────────────────────────────────╯ │ 1824: │ │ 1825: │ /home/runner/work/julep/julep/agents-api/agents_api/queries/utils.py:296 │ 1826: │ in async_wrapper │ 1827: │ │ 1828: │ 293 │ │ │ │ result: T = await func(*args, **kwargs) │ 1829: │ 294 │ │ │ except BaseException as error: │ 1830: │ 295 │ │ │ │ _check_error(error) │ 1831: │ ❱ 296 │ │ │ │ raise error │ ... 1856: │ │ 1857: │ /home/runner/work/julep/julep/agents-api/agents_api/queries/utils.py:293 │ 1858: │ in async_wrapper │ 1859: │ │ 1860: │ 290 │ │ @wraps(func) │ 1861: │ 291 │ │ async def async_wrapper(*args: P.args, **kwargs: P.kwargs) │ 1862: │ 292 │ │ │ try: │ 1863: │ ❱ 293 │ │ │ │ result: T = await func(*args, **kwargs) │ 1864: │ 294 │ │ │ except BaseException as error: │ 1865: │ 295 │ │ │ │ _check_error(error) │ 1866: │ 296 │ │ │ │ raise error │ ... 1996: │ │ │ │ │ │ {}, │ │ 1997: │ │ │ │ │ │ {} │ │ 1998: │ │ │ │ │ ], │ │ 1999: │ │ │ │ │ 'timeout': 90.0 │ │ 2000: │ │ │ │ } │ │ 2001: │ │ │ ) │ │ 2002: │ │ ] │ │ 2003: │ │ conn = <PoolConnectionProxy [released] 0x7fb4944f9660> │ │ 2004: │ │ connection_error = False │ │ ... 2017: │ │ │ │ model='gpt-4o-mini', │ │ 2018: │ │ │ │ instructions=['test instruction'], │ │ 2019: │ │ │ │ default_settings=None │ │ 2020: │ │ │ ) │ │ 2021: │ │ } │ │ 2022: │ │ method = <bound method Connection.fetch of │ │ 2023: │ │ <PoolConnectionProxy [released] 0x7fb4944f9660>> │ │ 2024: │ │ method_name = 'fetch' │ │ 2025: │ │ only_on_error = False │ │ ... 2221: │ │ statement = None │ │ 2222: │ │ stmt_name = '__asyncpg_stmt_16__' │ │ 2223: │ │ timeout = 90.0 │ │ 2224: │ │ use_cache = True │ │ 2225: │ ╰──────────────────────────────────────────────────────────────────────╯ │ 2226: │ │ 2227: │ in prepare:165 │ 2228: ╰──────────────────────────────────────────────────────────────────────────╯ 2229: UndefinedTableError: relation "agents" does not exist 2230: ─────────────────────────── query: update agent sql ──────────────────────────── 2231: Failed at tests/test_agent_queries.py ... 2372: │ │ 2373: │ /home/runner/.local/share/uv/python/cpython-3.12.8-linux-x86_64-gnu/lib/ │ 2374: │ python3.12/asyncio/runners.py:118 in run │ 2375: │ │ 2376: │ 115 │ │ │ 2377: │ 116 │ │ self._interrupt_count = 0 │ 2378: │ 117 │ │ try: │ 2379: │ ❱ 118 │ │ │ return self._loop.run_until_complete(task) │ 2380: │ 119 │ │ except exceptions.CancelledError: │ ... 2386: │ │ coro = <coroutine object test_developer at 0x7fb4944f26c0> │ │ 2387: │ │ self = <asyncio.runners.Runner object at 0x7fb4944fa990> │ │ 2388: │ │ sigint_handler = functools.partial(<bound method Runner._on_sigint │ │ 2389: │ │ of <asyncio.runners.Runner object at │ │ 2390: │ │ 0x7fb4944fa990>>, main_task=<Task finished │ │ 2391: │ │ name='Task-27' coro=<test_developer() done, defined │ │ 2392: │ │ at │ │ 2393: │ │ /home/runner/work/julep/julep/agents-api/tests/fix… │ │ 2394: │ │ exception=UndefinedTableError('relation │ │ 2395: │ │ "developers" does not exist')>) │ │ 2396: │ │ task = <Task finished name='Task-27' │ │ 2397: │ │ coro=<test_developer() done, defined at │ │ 2398: │ │ /home/runner/work/julep/julep/agents-api/tests/fix… │ │ 2399: │ │ exception=UndefinedTableError('relation │ │ ... 2418: │ │ dsn = '***localhost:32769/test?sslmode=d… │ │ 2419: │ │ pool = <asyncpg.pool.Pool object at 0x7fb494663a00> │ │ 2420: │ ╰──────────────────────────────────────────────────────────────────────╯ │ 2421: │ │ 2422: │ /home/runner/work/julep/julep/agents-api/agents_api/queries/utils.py:296 │ 2423: │ in async_wrapper │ 2424: │ │ 2425: │ 293 │ │ │ │ result: T = await func(*args, **kwargs) │ 2426: │ 294 │ │ │ except BaseException as error: │ 2427: │ 295 │ │ │ │ _check_error(error) │ 2428: │ ❱ 296 │ │ │ │ raise error │ ... 2442: │ │ 2443: │ /home/runner/work/julep/julep/agents-api/agents_api/queries/utils.py:293 │ 2444: │ in async_wrapper │ 2445: │ │ 2446: │ 290 │ │ @wraps(func) │ 2447: │ 291 │ │ async def async_wrapper(*args: P.args, **kwargs: P.kwargs) │ 2448: │ 292 │ │ │ try: │ 2449: │ ❱ 293 │ │ │ │ result: T = await func(*args, **kwargs) │ 2450: │ 294 │ │ │ except BaseException as error: │ 2451: │ 295 │ │ │ │ _check_error(error) │ 2452: │ 296 │ │ │ │ raise error │ ... 2507: │ │ │ │ │ │ │ │ 2508: │ │ '00000000-0000-0000-0000-000000000000' │ │ 2509: │ │ │ │ │ ], │ │ 2510: │ │ │ │ │ 'timeout': 90.0 │ │ 2511: │ │ │ │ } │ │ 2512: │ │ │ ) │ │ 2513: │ │ ] │ │ 2514: │ │ conn = <PoolConnectionProxy [released] 0x7fb4944f9e10> │ │ 2515: │ │ connection_error = False │ │ ... 2517: │ │ debug = None │ │ 2518: │ │ kwargs = { │ │ 2519: │ │ │ 'developer_id': │ │ 2520: │ │ UUID('00000000-0000-0000-0000-000000000000') │ │ 2521: │ │ } │ │ 2522: │ │ method = <bound method Connection.fetch of │ │ 2523: │ │ <PoolConnectionProxy [released] 0x7fb4944f9e10>> │ │ 2524: │ │ method_name = 'fetch' │ │ 2525: │ │ only_on_error = False │ │ ... 2659: │ │ statement = None │ │ 2660: │ │ stmt_name = '__asyncpg_stmt_21__' │ │ 2661: │ │ timeout = 90.0 │ │ 2662: │ │ use_cache = True │ │ 2663: │ ╰──────────────────────────────────────────────────────────────────────╯ │ 2664: │ │ 2665: │ in prepare:165 │ 2666: ╰──────────────────────────────────────────────────────────────────────────╯ 2667: UndefinedTableError: relation "developers" does not exist ... 2963: │ ╰──────────────────────────────────────────────────────────────────────╯ │ 2964: │ │ 2965: │ /home/runner/work/julep/julep/agents-api/.venv/lib/python3.12/site-packa │ 2966: │ ges/ward/testing.py:637 in _resolve_single_arg │ 2967: │ │ 2968: │ 634 │ │ │ else: │ 2969: │ 635 │ │ │ │ fixture.resolved_val = arg(**args_to_inject) │ 2970: │ 636 │ │ except (Exception, SystemExit) as e: │ 2971: │ ❱ 637 │ │ │ raise FixtureError(f"Unable to resolve fixture '{fixtu │ ... 3079: │ │ │ │ timer=<ward._testing._Timer object at │ │ 3080: │ │ 0x7fb4948a78f0>, │ │ 3081: │ │ │ │ tags=[] │ │ 3082: │ │ │ ), │ │ 3083: │ │ │ iteration=0 │ │ 3084: │ │ ) │ │ 3085: │ ╰──────────────────────────────────────────────────────────────────────╯ │ 3086: ╰──────────────────────────────────────────────────────────────────────────╯ 3087: FixtureError: Unable to resolve fixture 'test_developer' 3088: ──────────────────────────────────────────────────────────────────────────────── 3089: ╭──────────── Results ─────────────╮ 3090: │ 3 Tests Encountered │ 3091: │ 3 Failures (100.0%) │ 3092: ╰──────────────────────────────────╯ 3093: ─────────────────────────── FAILED in 42.23 seconds ──────────────────────────── 3094: ##[error]Process completed with exit code 1.

---

✨ CI feedback usage guide:

---

The CI feedback tool (/checks) automatically triggers when a PR has a failed check.
The tool analyzes the failed checks and provides several feedbacks:

• Failed stage
• Failed test name
• Failure summary
• Relevant error logs

In addition to being automatically triggered, the tool can also be invoked manually by commenting on a PR:

/checks "https://github.com/{repo_name}/actions/runs/{run_number}/job/{job_number}"

where {repo_name} is the name of the repository, {run_number} is the run number of the failed check, and {job_number} is the job number of the failed check.

Configuration options

• enable_auto_checks_feedback - if set to true, the tool will automatically provide feedback when a check is failed. Default is true.
• excluded_checks_list - a list of checks to exclude from the feedback, for example: ["check1", "check2"]. Default is an empty list.
• enable_help_text - if set to true, the tool will provide a help message with the feedback. Default is true.
• persistent_comment - if set to true, the tool will overwrite a previous checks comment with the new feedback. Default is true.
• final_update_message - if persistent_comment is true and updating a previous checks message, the tool will also create a new message: "Persistent checks updated to latest commit". Default is true.

See more information about the checks tool in the docs.

[ellipsis-dev]

👍 Looks good to me! Incremental review on 32dfd5f in 20 seconds

More details

• Looked at 16 lines of code in 1 files
• Skipped 0 files when reviewing.
• Skipped posting 1 drafted comments based on config settings.

1. agents-api/agents_api/routers/files/get_file.py:20

• Draft comment:
  Consider implementing streaming for large payloads to avoid loading entire files into memory, which can lead to performance issues.
• Reason this comment was not posted:
  Comment was on unchanged code.

Workflow ID: wflow_Q32KbcXVIfplh21h

---

You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.