Permission Denied When Looking up Entry[BUG]

[atuma-tds]

What happened:
Tag template uploaded, service account has all required permission to data catalog and big query, but unable to push tags: "WARNING:root:Permission denied when looking up Entry for //bigquery.googleapis.com/projects/test****/datasets/us_state_sales/tables/us_state_salesregions. The resource will be skipped."

What you expected to happen:
Tags to be attached to specified datasets/tables.

How to reproduce it (as minimally and precisely as possible):
Just following the tutorial line by line.

Anything else we need to know?:
I can manually add the tags to dataset/table using the UI after the tag template is created using the tutorial, but still unable to attach tags using the tutorial.

[atuma-tds]

I am wondering if this is because since this repo was last updated, data catalog and dataplex have merged together, so maybe there is a different method for attaching tags to datasets while creating the tag templates themselves remained the same?

[mesmacosta]

Hi @atuma-tds, the error most likely is related to the Dataplex changes and newest versions of the google-cloud-datacatalog library.

Which version of the google-cloud-datacatalog library are you using in the same environment running datacatalog-util ?

For reference I just ran datacatalog-util today with this set of libraries.

output of pip list:

Package                              Version
------------------------------------ -----------
cachetools                           5.3.3
certifi                              2024.2.2
charset-normalizer                   3.3.2
datacatalog-fileset-enricher         1.2.0
datacatalog-fileset-exporter         0.2.3
datacatalog-fileset-processor        0.1.4
datacatalog-object-storage-processor 0.1.2
datacatalog-tag-exporter             0.3.2
datacatalog-tag-manager              2.0.5
datacatalog-tag-template-exporter    0.1.2
datacatalog-tag-template-processor   0.1.2
datacatalog-util                     0.11.6
google-api-core                      2.19.0
google-auth                          2.29.0
google-cloud-core                    2.4.1
google-cloud-datacatalog             1.0.2
google-cloud-storage                 2.16.0
google-crc32c                        1.5.0
google-resumable-media               2.7.0
googleapis-common-protos             1.63.0
grpc-google-iam-v1                   0.12.7
grpcio                               1.63.0
grpcio-status                        1.48.2
idna                                 3.7
numpy                                1.24.4
pandas                               2.0.3
pip                                  24.0
proto-plus                           1.23.0
protobuf                             3.20.3
pyasn1                               0.6.0
pyasn1_modules                       0.4.0
python-dateutil                      2.9.0.post0
pytz                                 2024.1
requests                             2.31.0
rsa                                  4.9
setuptools                           69.5.1
six                                  1.16.0
tabulate                             0.9.0
tzdata                               2024.1
urllib3                              2.2.1
wheel                                0.43.0

And successfully ran it attaching a tag:

INFO:root:
INFO:root:===> Upsert Tags from CSV [STARTED]
INFO:root:
INFO:root:Reading CSV file: sample-input/create-tags/tags-opt-1-all-metadata.csv...
INFO:root:
INFO:root:Upserting the Tags...
INFO:root:
INFO:root:LOOKUP Entry: //bigquery.googleapis.com/projects/**-redacted-**/datasets/data_profiles_dataset/tables/profile_results_latest_v1
INFO:root:--------------------------------------------------
INFO:root:     Found!
INFO:root:
INFO:root:GET Tag Template: projects/**-redacted-**/locations/us-central1/tagTemplates/aproved_template
INFO:root:--------------------------------------------------
INFO:root:     Found!
INFO:root:
INFO:root:UPDATE Tag: projects/**-redacted-**/locations/us/entryGroups/@bigquery/entries/**-redacted-**/tags/CZ6y_qyBY0Ilv
INFO:root:--------------------------------------------------
INFO:root:
INFO:root:==== Upsert Tags from CSV [FINISHED] =============

Then with some small changes in the codebase I ran using google-cloud-datacatalog==3.19.0 and I got the same error from this thread. To actually upgrade this library and libraries that are used by this one would require some effort. So do let me know if you still would use this with the version you are using, or if anyone else is following this issue and facing similar issues.