Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement nested identifier access ( "Nested identifiers not yet supported" ) #11445

Open
alamb opened this issue Jul 12, 2024 · 4 comments
Open
Assignees
Labels
enhancement New feature or request

Comments

@alamb
Copy link
Contributor

alamb commented Jul 12, 2024

Is your feature request related to a problem or challenge?

This came up on discord: https://discord.com/channels/885562378132000778/885562378132000781/1261359404197089443

> create or replace view foo as select  {'field1': { 'subfield1': 'foo', subfield2: 'bar', subfield3: [10,20,30,40]}} as "the_column";
0 row(s) fetched.
Elapsed 0.001 seconds.

> select * from foo;
+-------------------------------------------------------------------------+
| the_column                                                              |
+-------------------------------------------------------------------------+
| {field1: {subfield1: foo, subfield2: bar, subfield3: [10, 20, 30, 40]}} |
+-------------------------------------------------------------------------+
1 row(s) fetched.
Elapsed 0.002 seconds.

> select the_column.field1.subfield3[2] from foo;
Internal error: Nested identifiers not yet supported for column foo.the_column.
This was likely caused by a bug in DataFusion's code and we would welcome that you file an bug report in our issue tracker

Describe the solution you'd like

The query should work and return 20

Describe alternatives you've considered

Single level access works fine:

> create or replace view foo as select { 'subfield1': 'foo', subfield2: 'bar', subfield3: [10,20,30,40]} as "field1" ;
0 row(s) fetched.
Elapsed 0.008 seconds.

> select * from foo;
+---------------------------------------------------------------+
| field1                                                        |
+---------------------------------------------------------------+
| {subfield1: foo, subfield2: bar, subfield3: [10, 20, 30, 40]} |
+---------------------------------------------------------------+
1 row(s) fetched.
Elapsed 0.003 seconds.

> select field1.subfield3[2] from foo;
+----------------------------------------------------+
| get_field(foo.field1, Utf8("subfield3"))[Int64(2)] |
+----------------------------------------------------+
| 20                                                 |
+----------------------------------------------------+
1 row(s) fetched.
Elapsed 0.002 seconds.

Additional context

No response

@alamb alamb added the enhancement New feature or request label Jul 12, 2024
@alamb alamb changed the title "Nested identifiers not yet supported" error Implement nested identifier access ( "Nested identifiers not yet supported" ) Jul 12, 2024
@kavirajk
Copy link
Contributor

take

@kavirajk
Copy link
Contributor

@alamb kinda stuck on this. I track down where the error is coming from.

But don't know which datatype (I assume Datatype::Struct?) methods needs to be modified to allow nested access here in getfield.rs. Appreciate any leads/tips? thanks.

@alamb
Copy link
Contributor Author

alamb commented Jul 22, 2024

Sorry @kavirajk I am not super familar with the struct access so I can't offer any code specific suggestions

What I would probably do is remove the check, then write some tests, and then work to get all the tests passing.

@kavirajk
Copy link
Contributor

Thanks @alamb good idea.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants