BigQuery: listJobs is throwing IAE

[shevek]

Thanks for stopping by to let us know something could be better!

PLEASE READ: If you have a support contract with Google, please create an issue in the support console instead of filing on GitHub. This will ensure a timely response.

Please run down the following list and make sure you've tried the usual "quick fixes":

• Search the issues already opened: https://github.com/googleapis/google-cloud-java/issues
• Check for answers on StackOverflow: http://stackoverflow.com/questions/tagged/google-cloud-platform

If you are still having issues, please be sure to include as much information as possible:

Environment details

1. Specify the API at the beginning of the title (for example, "BigQuery: ...")
2. OS type and version: Ubuntu 19.04
3. Java version: openjdk version "1.8.0_222"
4. google-cloud-java version(s): com.google.cloud:google-cloud-bigquery:1.96.0

Steps to reproduce

Happens about 100 times while listing 65,000 jobs on a project.

Code example

bq.listJobs(...);
for (Job job : page.getValues()) { // Throws.
}

Stack trace

java.lang.IllegalArgumentException: Provided dataset is null or empty
        at com.google.common.base.Preconditions.checkArgument(Preconditions.java:142)
        at com.google.cloud.bigquery.TableId.<init>(TableId.java:67)
        at com.google.cloud.bigquery.TableId.fromPb(TableId.java:109)
        at com.google.cloud.bigquery.QueryJobConfiguration$Builder.<init>(QueryJobConfiguration.java:178)
        at com.google.cloud.bigquery.QueryJobConfiguration$Builder.<init>(QueryJobConfiguration.java:89)
        at com.google.cloud.bigquery.QueryJobConfiguration.fromPb(QueryJobConfiguration.java:903)
        at com.google.cloud.bigquery.JobConfiguration.fromPb(JobConfiguration.java:128)
        at com.google.cloud.bigquery.JobInfo$BuilderImpl.<init>(JobInfo.java:158)
        at com.google.cloud.bigquery.Job.fromPb(Job.java:485)
        at com.google.cloud.bigquery.BigQueryImpl$32.apply(BigQueryImpl.java:1127)
        at com.google.cloud.bigquery.BigQueryImpl$32.apply(BigQueryImpl.java:1124)
        at com.google.common.collect.Iterators$6.transform(Iterators.java:786)
        at com.google.common.collect.TransformedIterator.next(TransformedIterator.java:47)

External references such as API reference guides used

• ?

Any additional information below

Making sure to follow these steps will guarantee the quickest resolution possible.

Thanks!

[pmakani]

@shevek i tried with below example. i am not able to reproduce that error which you are getting.
can you please share your sample code or let me know if i am missing something ?

public void createQueryJob() {
    String datasetId = "test_list_dataset";
    String tableId = "test_list_table_a";
    String query =
        "SELECT StringField, BooleanField FROM " + TableId.of(datasetId, tableId).getTable();
    QueryJobConfiguration config =
        QueryJobConfiguration.newBuilder(query).setDefaultDataset(datasetId).build();
    Job job = bigQuery.create(JobInfo.of(JobId.of(), config));
    System.out.println(job.getStatus().getError());
  }

  public void listJob() {
    Page<Job> jobs = bigQuery.listJobs();
    for (Job job : jobs.getValues()) {
      System.out.println(job.getJobId());
    }
  }

Output:

JobId{project=xyz-devs, job=e5b3896b-413b-4bfc-912e-1fd88044d401, location=US}
JobId{project=xyz-devs, job=18862a4f-0ee8-4eb2-9313-ac586495ce74, location=US}
JobId{project=xyz-devs, job=1412dcc0-57e7-4648-ac2a-58e5e11ec722, location=US}
JobId{project=xyz-devs, job=3454dea4-bdae-4066-9f6f-d8e629bf281c, location=US}
JobId{project=xyz-devs, job=46a7a8a2-66e5-4c1c-a7af-e093084f9f5c, location=US}
JobId{project=xyz-devs, job=970e32c3-e745-457a-9afb-12d7d6ef4d63, location=US}

[shevek]

Our code is the same. We just ran much larger volume, possibly over a larger period of time, and possibly over a number of backend revisions. Is there any way to have a job without a default dataset? We're listing jobs submitted through the GUI and potentially by other means including unit tests, I don't know how the jobs were submitted, and the ones that throw exceptions I (obviously) can't inspect to figure that out.

[pmakani]

@shevek without a default dataset, it will throw an error from backend.

i.e.

Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 400 Bad Request
{
  "code" : 400,
  "errors" : [ {
    "domain" : "global",
    "message" : "Table name \"test_list_table_a\" missing dataset while no default dataset is set in the request.",
    "reason" : "invalid"
  } ],
  "message" : "Table name \"test_list_table_a\" missing dataset while no default dataset is set in the request.",
  "status" : "INVALID_ARGUMENT"
}

[shevek]

@pmakani Does this imply that it's impossible to listJobs on a job which was submitted without a default dataset?

[pmakani]

Closing this issue. Feel free to reopen if above reply did not help.

[shevek]

Situation still arises, and the above has not helped, because when I call this function, it still throws an exception in code I do not control, but I do not appear to have permission reopen the issue.