You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
When fetching column types from the cursor.description based on the result of executing a BQ query, ARRAY types seem to be misrepresented. For example:
conn.execute("CREATE TABLE my_project.my_dataset.new_table (numbers ARRAY<INT64>)")
conn.execute("INSERT INTO my_project.my_dataset.new_table (numbers) VALUES ([1, 2, 3])")
result=conn.execute("SELECT * FROM my my_project.my_dataset.new_table")
print(result.cursor.description)
# Column(name='numbers', type_code='INTEGER', display_size=None, internal_size=None, precision=None, scale=None, null_ok=False)
This causes my consuming code to incorrectly interpret the type of that column, which leads to failures down the line.
Describe the solution you'd like
Would it be possible to have the cursor description in this case include the ARRAY type instead? I see that the values fetchall() returns from this result always include brackets [] which seems parse-able to me. Also, ARRAY already exists in the type map, so I assume there is some level of support already.
Describe alternatives you've considered
As a workaround, it is possible to get the proper ARRAY<INT64> type by instead querying INFORMATION_SCHEMA. However, it would be more convenient and performant in many cases to use the cursor description of a SELECT * ... LIMIT 0 query. More generally, I expect those types to be accurate.
Additional context
This problem remains if the LIMIT 0 is removed, by the way, as in my example at the top of the post.
The text was updated successfully, but these errors were encountered:
Perhaps the underlying difficulty here is that the structure of the cursor description itself doesn't allow any clear way to communicate that this column type is ARRAY and simultaneously that the values inside the ARRAY are of type INT64? In this case, at least returning type ARRAY would give consuming code a clear "heads up" that some special handling needs to happen for this column, rather than perpetuating the false assumption that the column is the same as any other INT64 column.
I recognize that would be a breaking change, but to me at least it seems like a clearer way of communicating the situtation based on current limitations.
Edit: even more helpful would be something like ARRAY[INT64].
Is your feature request related to a problem? Please describe.
When fetching column types from the
cursor.description
based on the result of executing a BQ query,ARRAY
types seem to be misrepresented. For example:This causes my consuming code to incorrectly interpret the type of that column, which leads to failures down the line.
Describe the solution you'd like
Would it be possible to have the cursor description in this case include the
ARRAY
type instead? I see that the valuesfetchall()
returns from this result always include brackets[]
which seems parse-able to me. Also,ARRAY
already exists in the type map, so I assume there is some level of support already.Describe alternatives you've considered
As a workaround, it is possible to get the proper
ARRAY<INT64>
type by instead queryingINFORMATION_SCHEMA
. However, it would be more convenient and performant in many cases to use the cursor description of aSELECT * ... LIMIT 0
query. More generally, I expect those types to be accurate.Additional context
This problem remains if the
LIMIT 0
is removed, by the way, as in my example at the top of the post.The text was updated successfully, but these errors were encountered: