-
Notifications
You must be signed in to change notification settings - Fork 244
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Exposing SubQuery in Table Level Lineage #192
Comments
I see why you would want this feature, but unfortunately this would need a lot of changes to current code base. To make it happen, we need to refactor the code not limited to the following aspect:
In conclusion, thanks for reporting this, but this may take time. |
Because complex buniess login need many steps:temp table/subquery/with. |
Thanks for the example. I'll prioritize this issue. |
The first step is done after we close #196 |
what's the tech stack / sql dialect you're using? Looks like PostgreSQL. |
do you have a wechat or other social account?maybe communicate needs more convenient |
I'd prefer to keep all the discussion here so that later people know how decision is made along the way. But feel free to email me if that's convenient for you. |
how about this issue going? |
Not so fast. Right now I'm still focusing on fixing various edge case bugs for column level lineage. Might need another 2 or 3 minor version release before things are stable at column level. |
column lineage can show subquery , then table lineage contain subquery perhaps easier, for example directly from column lineage is one way. |
how about this issue going? |
We're prioritizing #302, changing the underlying parser to support more SQL dialects. Every other issue is put on hold for the moment. |
how about this issue going? |
Can you share the use case? I really didn't anticipate this feature should attract so many attentions? |
I'm now using MySQL. When doing data analysis, we use a lot of subquery. When a problem occur, I use this library to analyze the SQL so that we trace the origin of each field. However, when meeting subqueries, it's not so useful because we can only see that the data starts from a subquery. |
@BrandenXia it feels to me that your problem is mostly related with incomplete column lineage. Can you check #303 to see if that's the same with your issue? Or you can share some queries that you don't get what you expect with column lineage. |
Yeah, that's basically same with the problem I mentioned. |
with c1 as (select * from ttt),
c2 as(select * from c1)
select * from (select * from c2 as aa)b
I want the table lineage inclued subquery name, this example need output:
ttt -> c1
c1 -> c2
c2 -> b
The text was updated successfully, but these errors were encountered: