[Bug]: snowflake_execute - context deadline exceeded duration

[fabiomx]

Terraform CLI Version

1.10.4

Terraform Provider Version

1.0.0

Company Name

No response

Terraform Configuration

terraform {
  required_providers {
    snowflake = {
      source  = "Snowflake-Labs/snowflake"
      version = "1.0.0"
    }
  }
}

provider "snowflake" {
  organization_name = "xxxx"
  account_name      = "xxxx"
  user              = "username"
  authenticator     = "SNOWFLAKE_JWT"
  private_key = file("~/path.p8")

  role                     = "role_name"
  warehouse                = "wh_name"
  preview_features_enabled = [
    "snowflake_dynamic_table_resource", "snowflake_file_format_resource", "snowflake_function_sql_resource", "snowflake_pipe_resource",
    "snowflake_procedure_python_resource", "snowflake_stage_resource",
    "snowflake_storage_integration_resource", "snowflake_table_resource"
  ]
  request_timeout = 1320 # 22mins # tested with no/other values, same issue
  driver_tracing = "trace"
}

# Context deadline test
resource "snowflake_execute" "context_deadline_test" {
 execute = "CALL SYSTEM$WAIT(22, 'MINUTES');"
 revert = "select 'context_deadline_test revert'"
}

Category

category:resource

Object type(s)

resource:execute

Expected Behavior

The query should be able to run for the necessary time, even if it's longer than 20 minutes.

Actual Behavior

Because of "context deadline exceeded duration 20m0" error (see attached log), the SQL query is canceled after 20 minutes.

I'm not sure if the issue depends on the provider, the Go driver, or even Snowflake itself (I'll comment on that later).

Obviously, the original query is not a SYSTEM$WAIT, but a real query that copies data through an "insert into..from" statement, and can take many minutes. In fact, a few weeks ago, I was able to run the same code on other tables without any problems, despite it taking over 20 minutes (it even reached over 60 minutes without issues).

It may be relevant to share that just last night, the same error appeared in our Grafana instance when connecting to Snowflake, which also uses the Go driver (client: failed to query data: Failed to query data: rpc error: code = DeadlineExceeded desc = context deadline exceeded).

On the other hand, the queries are executed correctly from another client (such as DataGrip via JDBC driver) and the same machine, so it's not an issue with account/user-level parameters, network connectivity, or machine-specific settings, which I've also reviewed.

Due to the uncertainty about the origin of the issue, I opened a ticket with Snowflake support, relating it to Go driver, but since the problem is also related to Terraform, which is not covered, they have dismissed it.

Steps to Reproduce

Add

resource "snowflake_execute" "context_deadline_test" {
 execute = "CALL SYSTEM$WAIT(22, 'MINUTES');"
 revert = "select 'context_deadline_test revert'"
}

and apply the resource

How much impact is this issue causing?

Medium

Logs

https://gist.github.com/fabiomx/dbcc757b635e227498526a50c415069a

Additional Information

No response

Would you like to implement a fix?

• Yeah, I'll take it 😎

[sfc-gh-asawicki]

Hey @fabiomx. Thanks for reaching out to us.

Client timeout is one thing, but we also have to add custom timeouts directly to the resource (https://developer.hashicorp.com/terraform/language/resources/syntax#operation-timeouts).

We should add these to the next release. I will get back to you so you would be able to verify if that solves your problem.

[fabiomx]

Hi @sfc-gh-asawicki , thanks for getting back to me.

Yes, indeed I had noticed that this provider doesn't support adding custom timeouts directly to resources as described in the Terraform documentation. However, I'm still puzzled because, as mentioned before, I hadn't experienced any issues with timeouts until now, at least up to version 0.98.0.

Where are these 20-minute timeouts set? I couldn't find any reference to them in the provider's settings or Snowflake's parameters.

Also, do you have an estimated release date for the next version that will include the custom timeouts? I'd be happy to verify if it solves my problem.

Thank you

[sfc-gh-asawicki]

Hey @fabiomx.

These are the defaults: https://developer.hashicorp.com/terraform/plugin/sdkv2/resources/retries-and-customizable-timeouts#default-timeouts-and-deadline-exceeded-errors.

We should have the next patch on Thursday next week (16 Jan).

[sfc-gh-asawicki]

Hey @fabiomx. The release got postponed to Monday (20 Jan); We are sorry for the inconvenience.