v1.43.0

1.43.0

New Features and Improvements

• Added state upgrader to remove max_clusters_per_user and ebs volume attributes set to zero (#3551).
• Robust retries for workspace get-status (#3550).
• Updated data_aws_crossaccount_policy to format resource string with account and region (#3544).
• Fixed Issue: The field node_type_id cannot be supplied when an instance pool ID is provided (#3549)

Documentation Changes

• Fixed error message typo in mounts.go (#3552).
• State the options for run_if (#3548).

Exporter

• Don't emit all UC objects when handling dependencies (#3556).
• Track removed objects during the Emit phase (#3554).
• Optimize generation of databricks_group_member resource (#3559).
• Included some UC objects into default listing (#3565)

Internal Changes

• Jobs GoSDK schema migration (#3532).
• Stabilize integration tests (#3542).

v1.42.0

New Features and Improvements

• Allow changes to display_name for databricks_service_principal (#3523).
• Modify databricks_repo to support Git folders in the workspace (#3447).
• Retry requests in the Databricks Go SDK until context timeout (#3537).
• Set ForceSendFields for boolean/integer values explicitly set to false/0 (#3385).

Bug fixes

• Added ConflictsWith to legacy tasks in databricks_job to avoid errors when job parameters are used (#3500).
• Fixed spn secret permanent diff (#3525).
• Introduced common.NoClientData helper function and use it in databricks_aws_crossaccount_policy data source to allow its use with workspace- or account-level providers (#3508).
• Set id attribute of databricks_metastore data source (#3503).

Documentation Changes

• Added documentation for compute options for tasks in databricks_job resource (#3504).
• Added warning about depends_on order (#3489).
• Fixed various doc typos (#3512).
• Fixed databricks_current_metastore data source names in docs examples (#3515).
• Improved Job documentation structure (#3511).
• Removed confusing mention of cloud types for databricks_clusters and databricks_sql_warehouses data sources (#3518).

Exporter

• Added UC support for DLT pipelines (#3494).
• Ignored instance pools with the empty name (#3522).

Internal Changes

• Added DATABRICKS_USER_AGENT_EXTRA (#3520).
• Adjusted list of supported platforms in godownloader-databricks-provider.sh (#3527).
• Switched to Go SDK for databricks_directory resource and data source (#3509).
• Fixed codecov for repository (#3530).

Dependency updates:

• Bump databricks-sdk-go to v0.40.0 (#3534).

v1.41.0

1.41.0

New Features and Improvements

• Added resources for configuring Serverless network connectivity (#3402).
• Data source databricks_current_metastore should not error out for non-UC workspaces (#3497).

Exporter

• Adjust retry error on deadline exceeded (#3491).

Internal Changes

• Added account workspace setting type (#3415).
• Added test config (#3487).
• Refactor CustomizeSchema for ResourceProvider to skip unfeasible customizations (#3481).
• Bump golang.org/x/net from 0.22.0 to 0.23.0 (#3493).

v1.40.0

New Features and Improvements

• Added enable_predictive_optimization parameter for databricks_catalog and databricks_schema resources. (#3416).
• Added deployment field to the databricks_pipeline resource (#3428).
• Added lookup by name or region to databricks_metastore data source (#3419).
• Added zone_id to gcp_attributes of databricks_instance_pool (#3426).
• Added support for SQL Notebooks execution in databricks_job (#3420).
• Support adding and removing columns in databricks_sql_table (#3381).
• Fixed databricks_entitlement Update & Delete (#3434).
• Fixed databricks_mws_private_access_settings creation (#3439).
• Fixed databricks_user and databricks_service_principal integration tests (#3469).
• Fixed suppress_diff for webhook notifications at the task level in databricks_job (#3441).
• Only suppress diff for run_if when it is ALL_SUCCESS (#3454).
• Used Account client in the databricks_aws_crossaccount_policy data source (#3423).
• Added data sources databricks_external_locations and databricks_external_location (#3464).
• Added environment and environment_key to job settings (#3461).
• Added managed properties (#3465).

Documentation Changes

• Use correct provider alias in AWS E2 provisioning guide (#3425).
• Document pricing_tier of databricks_mws_workspaces resource (#3433).
• Updated documentation for Delta Sharing resources; refactoring relevant resources & data sources (#3466).
• Added description for new fields in databricks_lakehouse_monitor
• Update about using Databricks-managed service principals in all clouds (#3482)

Exporter

• Decrease a need for get-status calls for directories when listing workspace objects (#3470).
• Don't emit permissions and grants if object is going to be ignored (#3417).
• Don't generate references to ignored resources (#3418).
• Generate resource data from the object status obtained during the listing operation (#3460).
• Improve emitting of directories (#3462).
• Skip listing of jobs deployed with DABs (#3445).

Internal Changes

• Migrated databricks_cluster resource to Go SDK (#3376).
• Used ClusterSpec in databricks_cluster resource (#3459).
• Bump Go SDK to v0.38.0 (#3458).
• Bump golang.org/x/mod from 0.16.0 to 0.17.0 (#3435).
• Clean up integration tests and expose helper functions for identifying cloud (#3438).
• Fix integration tests for databricks_entitlements (#3467).
• Switch to use structs from Go SDK for databricks_recipient resource (#3400).
• Make libraries in resource_job slice instead of set (#3398).
• Fixed TestUcAccJobRunAsMutations
• Ensure tests are only run in the selected environment (#3476)
• Use fixture cluster for databricks_sql_permissions integration test (#3483)
• Improve integration tests for databricks_model_serving (#3479)

v1.39.0

New Features and Improvements

• Add databricks_online_table resource (#3352).
• Add resource provider registry (#3347).
• Add support for Vector Search Indexes (#3266).
• Add table update trigger in terraform API (#3401).
• Added owner attribute to databricks_registered_model (#3366).
• Added support for case insensitivity for id fields in UC resources (#3164).
• Allow changes to display_name of databricks_group resource without recreating a group (#3334).
• Delete Vector Search Endpoint when we get an error while waiting for creation (#3348).
• Fix Read operation in databricks_catalog_workspace_binding so we can perform import (#3276).
• Fix sending scale_to_zero_enabled for served_entities in databricks_model_serving resource (#3387).
• Support persisting empty fields (#3362).
• [MLOPS-596] Add Lakehouse Monitor Resource (#3238).
• Update databricks_model_serving resource to match latest API spec/SDK (#3093).

Documentation Changes

• Clarify disable as user deletion (#3360).
• Add MODEL to documentation about data_object_type in databricks_share resource (#3380).
• Fix broken links in metastore-related docs (#3356).
• Update documentation about databricks-managed service principals (#3350).
• Fix code example for databricks_storage_credentials data source (#3327).
• Fix storage credential doc guide (#3340).
• Improve description of depends_on block (#3390).
• Improve job library documentation (#3399).
• [DOCS] change master to main & GH image raw links (#3358).
• [DOC] Removing NOTEBOOK_FILE from databricks_share (#3353).
• [MLOPS-717] Rename Amazon Bedrock (#3403).

Exporter

• Don't generate resources with empty names (#3345).
• Remove too aggressive omitting for volume_type attribute of databricks_volume resource (#3328).
• Add an option for exporting secret values into terraform.tfvars (#3372).
• Add support for generation of dependencies between resources (#3377).
• Change listing behavior for metastore and storage credentials (#3367).
• Modify grants for some UC objects to allow handling of other owners (#3357).

Internal Changes

• Add integration test for ForEachTask (#3283).
• Bump github.com/hashicorp/hcl/v2 from 2.20.0 to 2.20.1 (#3405).
• Bump github.com/zclconf/go-cty from 1.14.3 to 1.14.4 (#3393).
• Bump go-sdk version to 0.36.0 (#3391).
• Bump go-sdk version to 0.37.0 (#3410).
• Bump golang.org/x/mod from 0.15.0 to 0.16.0 (#3329).
• Bump google.golang.org/protobuf from 1.32.0 to 1.33.0 (#3368).
• Disable Lakehouse Monitoring integration tests on GCP (#3355).
• Increase default timeout for databricks_vector_search_endpoint (#3332).
• Make StructToSchema work for recursive data types (#3302).
• Port databricks_secret_scope, databricks_secret and databricks_secret_acl to Go SDK (#3373).
• SCIM patch to remove resources must not contain a value field (#3374).
• Update Aliases function for ResourceProvider to handle recursive structures more gracefully (#3338).
• Update Go SDK to latest + Fix backwards incompatible change (#3333).
• Update master to main in diff schema (#3331).
• improve databricks_aws_crossaccount_policy data source (#3343).
• lock metastore assignment test (#3386).
• Added support for updating columns in databricks_sql_table resource (#3359).
• Fix integration test for sql column update (#3384).

v1.38.0

New Features and Improvements

• Use workspace-level default for enable_serverless_compute if not specified in databricks_sql_global_config (#3279). Previously, this resource would disable serverless unless you set the deprecated enable_serverless_compute attribute to true. From this release, this resource will not modify enable_serverless_compute, leaving it as its default value (enabled) if unspecified.
• Fixed webhook_notifications diff suppression for databricks_job (#3309).
• Perform case-insensitive error message checking when using force in databricks_user (#3295).
• Use stable IDs for settings resources (#3314).

Documentation Changes

• Improved documentation for INTERNAL and INTERNAL_AND_EXTERNAL delta sharing scope in terraform (#3287).
• Added user guide for new behaviours with UC by default (#3318).
• Expanded the note about private link in the troubleshooting guide (#3319).
• Document storage_location attribute of the databricks_registered_model (#3249).
• Various documentation improvements (#3288).
• Changed accounts_id to required (#3244).

Exporter

• Improved support for Unity Catalog (#3281).
• Added generation of import blocks for easier importing existing resources in Terraform 1.5+ (#3293).
• Added support for databricks_file resource (#3301).

Internal Changes

• Set Computed field to true for common.Resource account_id attribute (#3310).
• Removed unnecessary alias from cluster resource (#3298).
• Bump Go SDK (#3296).
• Bump github.com/hashicorp/hcl/v2 from 2.19.1 to 2.20.0 (#3317).
• Bump github.com/hashicorp/terraform-plugin-sdk/v2 from 2.32.0 to 2.33.0 (#3300).
• Bump github.com/stretchr/testify from 1.8.4 to 1.9.0 (#3323).
• Bump github.com/zclconf/go-cty from 1.14.2 to 1.14.3 (#3322).

v1.37.1

New Features and Improvements

• Removed CustomizeDiff and Client Side Validation for databricks_grants (#3290).
• Added terraform support for restrict ws admins setting (#3243).

Internal Changes

• Migrated databricks_global_init_script to Go SDK (#2036).
• Bump github.com/hashicorp/terraform-plugin-sdk/v2 from 2.31.0 to 2.32.0 (#3177).

v1.37.0

1.37.0

New Features and Improvements

• Add databricks_file resource (#3265).
• Add databricks_storage_credential and databricks_storage_credentials data sources (#3254).
• Add source attribute to dbt_task and sql_task.file tasks to support files from workspace (#3208).
• Add computed volume_path attribute to databricks_volume resource (#3272).
• Add support for Vector Search Endpoints (#3191).
• [JOBS-16324] Terraform support for Foreach tasks (private preview) (#3252).
• fix: properly propagate auth_type to the databricks client (#3273).

Documentation Changes

• Fix images and add note on setting multiple authorizations for workspace setup (#3259).
• Remove data_object_type=TABLE only restriction in databricks_share (#3108).
• Remove legacy guides (#3282).
• Update for_each_task docs. (#3271).

Exporter

• Support for some Unity Catalog resources (#3242).
• Rework handling of listings and interactive prompting (#3241).
• UC exporter databricks storage credential feature (#3219).

Internal Changes

• Add CustomDiffFunc for health in sql_endpoint resources (#3227).
• Bump github.com/databricks/databricks-sdk-go 0.33.0 (#3275).
• Suppress diff on whitespace change for resources that often use HERE-docs (#3251).

v1.36.3

New Features and Improvements

• Explicitly set securable field when reading databricks_grants or databricks_grant (#3246).

Documentation Changes

• Added information on id and other exposed attributes where appropriate (#3237).
• Fixed docs in metastore databricks_grants example (#3239).

Exporter

• Detect & handle deleted workspace objects (notebooks/files/directories) when running in incremental mode (#3225).

Internal Changes

• Make IterFields take in aliases (#3207).

v1.36.2

1.36.2

New Features and Improvements

• Added databricks_aws_unity_catalog_policy data source (#2483).
• Removed omitempty in destination fields in clustes_api.go (#3232), to address (#3231)

Exporter

• Omitted git_provider only for well-known Git URLs (#3216).

Internal Changes

• Bumped github.com/zclconf/go-cty from 1.14.1 to 1.14.2 (#3144).
• Bumped golang.org/x/mod from 0.14.0 to 0.15.0 (#3229).