Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

DataflowPlan updates for group by metrics #1147

Closed
wants to merge 37 commits into from
Closed

DataflowPlan updates for group by metrics #1147

wants to merge 37 commits into from

Commits on Apr 15, 2024

  1. Add helper method SemanticModelJoinPath.from_single_element()

    @courtneyholcomb
    courtneyholcomb committed Apr 15, 2024
    Configuration menu
    Copy the full SHA
    0a1d91b View commit details
    Browse the repository at this point in the history

  2. Add MetricSubqueryJoinPath class

    @courtneyholcomb
    courtneyholcomb committed Apr 15, 2024
    Configuration menu
    Copy the full SHA
    050ee32 View commit details
    Browse the repository at this point in the history

  3. Rename method to specify 'valid' join (as opposed to fan-out join)

    @courtneyholcomb
    courtneyholcomb committed Apr 15, 2024
    Configuration menu
    Copy the full SHA
    11186c8 View commit details
    Browse the repository at this point in the history

  4. Update LinkableMetric to use MetricSubqueryJoinPath and remove redund… 

    …ancies
    @courtneyholcomb
    courtneyholcomb committed Apr 15, 2024
    Configuration menu
    Copy the full SHA
    98c0fec View commit details
    Browse the repository at this point in the history

  5. Map joinable metrics to entities instead of semantic models for more … 

    …specific group by resolution
    @courtneyholcomb
    courtneyholcomb committed Apr 15, 2024
    Configuration menu
    Copy the full SHA
    735a451 View commit details
    Browse the repository at this point in the history

  6. Use helper properties

    @courtneyholcomb
    courtneyholcomb committed Apr 15, 2024
    Configuration menu
    Copy the full SHA
    7d7d163 View commit details
    Browse the repository at this point in the history

  7. Get more specific about avoiding cycle creation

    @courtneyholcomb
    courtneyholcomb committed Apr 15, 2024
    Configuration menu
    Copy the full SHA
    0608c91 View commit details
    Browse the repository at this point in the history

Commits on Apr 17, 2024

  1. Types cleanup

    @courtneyholcomb
    courtneyholcomb committed Apr 17, 2024
    Configuration menu
    Copy the full SHA
    b5bf038 View commit details
    Browse the repository at this point in the history

Commits on Apr 18, 2024

  1. Fix bug in snapshot generation

    @courtneyholcomb
    courtneyholcomb committed Apr 18, 2024
    Configuration menu
    Copy the full SHA
    268cf40 View commit details
    Browse the repository at this point in the history

  2. Changelog

    @courtneyholcomb
    courtneyholcomb committed Apr 18, 2024
    Configuration menu
    Copy the full SHA
    324f64a View commit details
    Browse the repository at this point in the history

  3. Move setup dict to avoid circular logic error

    @courtneyholcomb
    courtneyholcomb committed Apr 18, 2024
    Configuration menu
    Copy the full SHA
    f05970e View commit details
    Browse the repository at this point in the history

  4. Privatize method

    @courtneyholcomb
    courtneyholcomb committed Apr 18, 2024
    Configuration menu
    Copy the full SHA
    23f976c View commit details
    Browse the repository at this point in the history

  5. Set JOINED and MULTI_HOP properties from join_path length (cleanup)

    @courtneyholcomb
    courtneyholcomb committed Apr 18, 2024
    Configuration menu
    Copy the full SHA
    ecbb5e9 View commit details
    Browse the repository at this point in the history

  6. Move logic for directly linkable metrics to _get_joined_elements (sem… 

    …antics)
    @courtneyholcomb
    courtneyholcomb committed Apr 18, 2024
    Configuration menu
    Copy the full SHA
    5b9210d View commit details
    Browse the repository at this point in the history

  7. Use new logic for finding linkable metrics (from previous commit) in … 

    …create_linkable_element_set_from_join_path
    @courtneyholcomb
    courtneyholcomb committed Apr 18, 2024
    Configuration menu
    Copy the full SHA
    b3d54a8 View commit details
    Browse the repository at this point in the history

  8. Update snapshots

    @courtneyholcomb
    courtneyholcomb committed Apr 18, 2024
    Configuration menu
    Copy the full SHA
    ed59e3d View commit details
    Browse the repository at this point in the history

  9. Add test with multi-hop join manifest

    @courtneyholcomb
    courtneyholcomb committed Apr 18, 2024
    Configuration menu
    Copy the full SHA
    4f4c45a View commit details
    Browse the repository at this point in the history

  10. Update documentation

    @courtneyholcomb
    courtneyholcomb committed Apr 18, 2024
    Configuration menu
    Copy the full SHA
    054e26f View commit details
    Browse the repository at this point in the history

Commits on Apr 23, 2024

  1. Update changelog to be more specific

    @courtneyholcomb
    courtneyholcomb committed Apr 23, 2024
    Configuration menu
    Copy the full SHA
    c4487f4 View commit details
    Browse the repository at this point in the history

  2. Add test for filtering by the same metric you're querying

    @courtneyholcomb
    courtneyholcomb committed Apr 23, 2024
    Configuration menu
    Copy the full SHA
    96408a8 View commit details
    Browse the repository at this point in the history

  3. Update SQL engine snapshots

    @courtneyholcomb
    courtneyholcomb committed Apr 23, 2024
    Configuration menu
    Copy the full SHA
    468a7d9 View commit details
    Browse the repository at this point in the history

  4. Update method to handle group by metrics

    @courtneyholcomb
    courtneyholcomb committed Apr 23, 2024
    Configuration menu
    Copy the full SHA
    a769baa View commit details
    Browse the repository at this point in the history

  5. Write test for 2-hop metric subquery join

    @courtneyholcomb
    courtneyholcomb committed Apr 23, 2024
    Configuration menu
    Copy the full SHA
    dab28d7 View commit details
    Browse the repository at this point in the history

  6. Use query parser to build group by metric source node 

    Needed to resolve the appropriate join path for multi-hop joins. Moved source node-building logic to query parser class to avoid cirular import errors. This change also required adding the query parser as a dependency for the dataflow plan builder.
    @courtneyholcomb
    courtneyholcomb committed Apr 23, 2024
    Configuration menu
    Copy the full SHA
    08d8c0f View commit details
    Browse the repository at this point in the history

  7. Parse entity links in Metric.group_by param

    @courtneyholcomb
    courtneyholcomb committed Apr 23, 2024
    Configuration menu
    Copy the full SHA
    448c5dd View commit details
    Browse the repository at this point in the history

  8. Add separate multi-hop test without explicity entity link

    @courtneyholcomb
    courtneyholcomb committed Apr 23, 2024
    Configuration menu
    Copy the full SHA
    0e6830a View commit details
    Browse the repository at this point in the history

  9. Handle group by metrics where missed

    @courtneyholcomb
    courtneyholcomb committed Apr 23, 2024
    Configuration menu
    Copy the full SHA
    1ffc496 View commit details
    Browse the repository at this point in the history

Commits on Apr 24, 2024

  1. Update LinkableMetrics & related classes to track metric_to_entity_jo… 

    …in_path

LinkableMetrics are extra complicated because they involve multiple join paths: 1) the join path needed to join the metric subquery to the outer query, and 2) the join path used in the subquery to join the metric to the group by entity. So far we have only been tracking join path 1. This commit adds tracking for join path 2.
    @courtneyholcomb
    courtneyholcomb committed Apr 24, 2024
    Configuration menu
    Copy the full SHA
    da0c8d0 View commit details
    Browse the repository at this point in the history

  2. Add metric_subquery_entity_links to GroupByMetricSpec 

    Needed when building source node for group by metric.
    @courtneyholcomb
    courtneyholcomb committed Apr 24, 2024
    Configuration menu
    Copy the full SHA
    c9a4c19 View commit details
    Browse the repository at this point in the history

Commits on Apr 25, 2024

  1. Update DFP join logic to account for goup by metric source nodes

    @courtneyholcomb
    courtneyholcomb committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    c7681a3 View commit details
    Browse the repository at this point in the history

  2. Capture metric_specs missed in ToElementNameSet transform function

    @courtneyholcomb
    courtneyholcomb committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    a11cf18 View commit details
    Browse the repository at this point in the history

  3. Add multi-hop integration test

    @courtneyholcomb
    courtneyholcomb committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    5befd3e View commit details
    Browse the repository at this point in the history

  4. Update AddGroupByMetrics to be singular AddGroupByMetric - should onl… 

    …y ever accept one group by metric
    @courtneyholcomb
    courtneyholcomb committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    f42b345 View commit details
    Browse the repository at this point in the history

  5. Remove cumulative and offset metrics from linkable metric options

    @courtneyholcomb
    courtneyholcomb committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    51a4a78 View commit details
    Browse the repository at this point in the history

  6. Delete unused expression

    @courtneyholcomb
    courtneyholcomb committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    5416b30 View commit details
    Browse the repository at this point in the history

  7. Test that all expected single-hop filters can successfully build data… 

    …flow plans
    @courtneyholcomb
    courtneyholcomb committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    48404d7 View commit details
    Browse the repository at this point in the history

  8. WIP - Test all expected metric filters work

    @courtneyholcomb
    courtneyholcomb committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    40b80da View commit details
    Browse the repository at this point in the history