Skip to content

fix: add support for repeated record query parameters #2698

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 10 commits into from
May 25, 2023
+583 −8
Merged

fix: add support for repeated record query parameters #2698

Changes from 1 commit
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
2c891e7
fix: add support for repeated record query parameters
obada-ab May 15, 2023
eefcdae
fix: set the query parameter type to an empty struct array if values …
obada-ab May 17, 2023
f8e36ce
🦉 Updates from OwlBot post-processor
gcf-owl-bot[bot] May 17, 2023
8deb47a
Merge branch 'main' into struct-array
obada-ab May 17, 2023
f15630a
chore: add test for a real dataset
Neenu1995 May 18, 2023
6fc4085
🦉 Updates from OwlBot post-processor
gcf-owl-bot[bot] May 18, 2023
0353c4e
chore: set up repeated record test to run in service account
obada-ab May 22, 2023
b5ac4af
doc: create and query repeated record sample
obada-ab May 25, 2023
49fa251
🦉 Updates from OwlBot post-processor
gcf-owl-bot[bot] May 25, 2023
746db7a
refactor: reformat repeated record sample
obada-ab May 25, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
55 changes: 55 additions & 0 deletions google-cloud-bigquery/src/test/java/com/google/cloud/bigquery/it/ITBigQueryTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4142,6 +4142,61 @@ public void testUnnestRepeatedRecordNamedQueryParameter() throws InterruptedExce
}
}

@Test
public void testUnnestRepeatedRecordNamedQueryParameterFromDataset() throws InterruptedException {
List<QueryParameterValue> tuples = new ArrayList<>();
QueryParameterValue statusValue = QueryParameterValue.string("single");
QueryParameterValue addressValue = QueryParameterValue.string("123 this lane");
QueryParameterValue cityValue = QueryParameterValue.string("Torono");
QueryParameterValue stateValue = QueryParameterValue.string("ON");
QueryParameterValue zipValue = QueryParameterValue.string("1h2j34");
QueryParameterValue numberOfYearsValue = QueryParameterValue.string("3");

Map<String, QueryParameterValue> struct = new HashMap<>();
Copy link
Contributor Author

@obada-ab obada-ab May 22, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The order of fields inside structs matters when doing comparisons, and HashMap does not maintain insertion order. Because of these reasons, the IN UNNEST(addresses) query doesn't return any results.

I tried replacing HashMap with a LinkedHashMap and using ImmutableMap.of, both options made the IN UNNEST(addresses) query work as intended and return the correct row.

struct.put("statusValue", statusValue);
struct.put("addressValue", addressValue);
struct.put("cityValue", cityValue);
struct.put("stateValue", stateValue);
struct.put("zipValue", zipValue);
struct.put("numberOfYearsValue", numberOfYearsValue);
QueryParameterValue recordValue = QueryParameterValue.struct(struct);
tuples.add(recordValue);

QueryParameterValue repeatedRecord =
QueryParameterValue.array(tuples.toArray(), StandardSQLTypeName.STRUCT);

String unnestRecord = "SELECT TEMP from `neenutestproject.AVRODATASET.NESTEDREPEATEDTABLE`, UNNEST(@repeatedRecord) AS TEMP ";
QueryJobConfiguration queryJobConfiguration = QueryJobConfiguration.newBuilder(unnestRecord)
.setUseLegacySql(false)
.addNamedParameter("repeatedRecord", repeatedRecord)
.build();
TableResult unnestResult = bigquery.query(queryJobConfiguration);
FieldValueList rowValue = unnestResult.getValues().iterator().next();
//Why is this printing the one row 4 times?
unnestResult
.iterateAll()
.forEach(row -> row.forEach(val -> System.out.printf("%s\n", val.toString())));

System.out.println("**************FROM DATASET****************");
// addresses is a REPEATED RECORD field with an array of Structs of the above type.
// @repeatedRecord is array of struct type namedQueryParameter with 1 struct
// the UI does return the correct 1 row output
// this test returns 0 results.
String query = "SELECT * FROM `neenutestproject.AVRODATASET.NESTEDREPEATEDTABLE`, UNNEST(@repeatedRecord) AS TEMP where TEMP IN UNNEST(addresses);";
QueryJobConfiguration queryConfig =
QueryJobConfiguration.newBuilder(query)
.setUseLegacySql(false)
.addNamedParameter("repeatedRecord", repeatedRecord)
.build();
TableResult results = bigquery.query(queryConfig);

System.out.println( Iterables.size(results.getValues()));
results
.iterateAll()
.forEach(row -> row.forEach(val -> System.out.printf("%s", val.toString())));
System.out.println("Query with Array of struct parameter performed successfully.");
}

@Test
public void testEmptyRepeatedRecordNamedQueryParameters() throws InterruptedException {
QueryParameterValue[] tuples = {};
Expand Down