Hi there 👋 Thanks for your contribution!

The OpenMetadata team will review the PR shortly! Once it has been labeled as safe to test, the CI workflows
will start executing and we'll be able to make sure everything is working as expected.

Let us know if you need any help!

Hi there 👋 Thanks for your contribution!

The OpenMetadata team will review the PR shortly! Once it has been labeled as safe to test, the CI workflows
will start executing and we'll be able to make sure everything is working as expected.

Let us know if you need any help!

@aballiet I've couple of questions regarding this pr:

1. I see that you are trying for find the project name and then finding the bigquery service in OpenMetadata related to that project, why do we need this approach? for airbyte we expect the name of them service in OpenMetadata would be same as the destination connection name in Airbyte, can you not maintain this consistent naming for this?
2. Also there is a fallback applied when source table involved in the table is not found, this might be applicable to your use case but other users might not expect such lineage any particular reason why the source table would not be found?

@aballiet I've couple of questions regarding this pr:

1. I see that you are trying for find the project name and then finding the bigquery service in OpenMetadata related to that project, why do we need this approach? for airbyte we expect the name of them service in OpenMetadata would be same as the destination connection name in Airbyte, can you not maintain this consistent naming for this?
2. Also there is a fallback applied when source table involved in the table is not found, this might be applicable to your use case but other users might not expect such lineage any particular reason why the source table would not be found?

1. I think the best approach to encourage adoption of the tool is to reduce the amount of naming convention. When setting up OpenMetadata, you may already have a lot of tools setup in your stack so better use truth from config instead of user entry to perform matching.

It's just a better design approach IMO to use Google BigQuery config to match with the correct database in OMD instead of a name in Airbyte that can change at anytime. Enforcing new naming convention is a blocker for us, we have 100s of Airbyte connection and will not change the names just because we have OMD.

2. It's not a fallback, it's the default behavior you have to support other destination using name to match (Snowflake, or any other available Airbyte destination). I just made the integration for BigQuery, ideally we would do it for every major destination supported in Airbyte.

Hope it makes more sense. In any case, we cannot merge this right now, as we need to wait OMD 1.5 to support lineage for pipelines.