LogfireQueryClient returns 500 results even when the limit is increased

[saeedesmaili]

Description

The LogfireQueryClient documentation indicates that the maximum allowed value for limit is 10,000, but it returns 500 even when limit is set for a value greater than 500:

from io import StringIO
from logfire.experimental.query_client import LogfireQueryClient
import pandas as pd

query = """
SELECT start_timestamp
FROM records
LIMIT 1000
"""

with LogfireQueryClient(read_token=LOGFIRE_READ_API_KEY, timeout=120) as client:
    df = pd.read_csv(StringIO(client.query_csv(sql=query)))

df.shape[0]
## 500

Python, Logfire & OS Versions, related packages (not required)

logfire="3.5.3"
platform="macOS-15.1.1-arm64-arm-64bit"
python="3.12.5 (main, Aug  6 2024, 19:08:49) [Clang 15.0.0 (clang-1500.3.9.4)]"
[related_packages]
requests="2.32.3"
protobuf="5.29.3"
rich="13.9.4"
executing="2.2.0"
opentelemetry-api="1.30.0"
opentelemetry-exporter-otlp-proto-common="1.30.0"
opentelemetry-exporter-otlp-proto-http="1.30.0"
opentelemetry-instrumentation="0.51b0"
opentelemetry-proto="1.30.0"
opentelemetry-sdk="1.30.0"
opentelemetry-semantic-conventions="0.51b0"

[Kludex]

On the query_csv method, there's a parameter limit. You need to set that one.

@dmontagu Would it be possible to remove the parameter and use only the SQL query?

[dmontagu]

I think we shouldn't remove the query param for legacy-ish reasons since it is already part of the API and allows you to change the value without needing to edit SQL, but we could change the default value to the max allowed (10_000).