Skip to content

fix: use-server-side-cursor is not working #746

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

fix: use-server-side-cursor is not working #746

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions application_sdk/inputs/sql_query.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
import concurrent
from typing import TYPE_CHECKING, AsyncIterator, Iterator, Optional, Union

from application_sdk.constants import USE_SERVER_SIDE_CURSOR
from application_sdk.inputs import Input
from application_sdk.observability.logger_adaptor import get_logger

Expand Down Expand Up @@ -120,6 +121,8 @@ def _execute_query(self) -> Union["pd.DataFrame", Iterator["pd.DataFrame"]]:
or iterator of DataFrames if chunked.
"""
with self.engine.connect() as conn:
if USE_SERVER_SIDE_CURSOR:
conn = conn.execution_options(yield_per=100000)
Comment on lines +124 to +125
Copy link

Copilot AI Sep 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The magic number 100000 for yield_per should be extracted to a named constant or made configurable. This hardcoded value makes it difficult to tune performance for different use cases.

Copilot uses AI. Check for mistakes.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: Server-Side Cursor Inconsistency and Yield Per Issue

Server-side cursor configuration is only applied in _execute_query, not _read_sql_query. This prevents async database connections from using server-side cursors, creating inconsistent behavior when USE_SERVER_SIDE_CURSOR is enabled. Additionally, the hardcoded yield_per=100000 value doesn't align with self.chunk_size, which could impact performance or memory.

Fix in Cursor Fix in Web

return self._execute_pandas_query(conn)

async def get_batched_dataframe(
Expand Down