Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[docs] document LIR attribution #30899

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

mgree
Copy link
Contributor

@mgree mgree commented Dec 23, 2024

Documents the LIR mapping introspection source (#29848).

Preview at https://preview.materialize.com/materialize/30899/transform-data/troubleshooting/.

Motivation

Checklist

  • This PR has adequate test coverage / QA involvement has been duly considered. (trigger-ci for additional test/nightly runs)
  • This PR has an associated up-to-date design doc, is a design doc (template), or is sufficiently small to not require a design.
  • If this PR evolves an existing $T ⇔ Proto$T mapping (possibly in a backwards-incompatible way), then it is tagged with a T-proto label.
  • If this PR will require changes to cloud orchestration or tests, there is a companion cloud PR to account for those changes that is tagged with the release-blocker label (example).
  • If this PR includes major user-facing behavior changes, I have pinged the relevant PM to schedule a changelog post.

@mgree mgree added A-docs Area: documentation A-optimization Area: query optimization and transformation T-observability labels Dec 23, 2024
@mgree mgree requested a review from ala2134 December 23, 2024 20:35
@mgree mgree requested a review from a team as a code owner December 23, 2024 20:35
Copy link
Contributor

@kay-kim kay-kim left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for this! Left some trivial suggestions (feel free to ignore).
I can pick up after vacation w.r.t. table rendering.

`winning_bids` view, while `u149` is the `wins_by_item` index.

The `operator` is indented using [`REPEAT`](/sql/functions/#repeat) and
`mz_lir_mapping.nesting`). The indenting, combined with ordering by
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
`mz_lir_mapping.nesting`). The indenting, combined with ordering by
`mz_lir_mapping.nesting`. The indenting, combined with ordering by


### Attributing memory usage

(If you have not read about [attributing computation time](#attributing-computation-time), please do so first, as it explains some core concepts.)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
(If you have not read about [attributing computation time](#attributing-computation-time), please do so first, as it explains some core concepts.)
{{< tip >}}
If you have not read about [attributing computation time](#attributing-computation-time), please do so first, as it explains some core concepts.
{{< /tip >}}

```sql
SELECT mo.name AS name, global_id, lir_id, parent_lir_id, REPEAT(' ', nesting * 2) || operator AS operator,
SUM(duration_ns)/1000 * '1 microsecond'::INTERVAL AS duration, SUM(count) AS count
FROM mz_introspection.mz_lir_mapping mlm
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Trivial nit (feel free to disregard). Do we want the FROM to either left-align with SELECT or right align with the 'LEFT JOIN'/'JOIN' ?

Have zero opinion as I've seen various alignments when using JOINS and I don't think we have a company style yet. But, this one seems to differ from the others.

ORDER BY global_id, lir_id DESC;
```

Running this query on an auction generator will produce results that look something like the following (though the specific numbers will vary, of course):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Double checking - so, users would just run this query ... and it'd list everything, yes? Like we wouldn't be able to scope this to the auction generator data -- or should we also add mo.id in ('wins_by_item', 'winning_bids') ?

[`mz_catalog.mz_objects`](/sql/system-catalog/mz_catalog/#mz_objects)
to find the actual name corresponding to the `global_id`.

Notice that the index is actually _two_ `global_id`s: `u148` is the
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we put this sentence and the The duration column ... closer? since they have to do with things to observe from the output?

Also, should we be more explicit in what we mean by "the index is actually two global_ids: ... wins_by_item index"? That is, the create index had to execute/"materialize" the view ... or something like that because I think having the word index appear twice might confuse people.

| wins_by_item | u149 | 8 | null | Arrange 7 | 00:00:00.013887 | 9347 |
| wins_by_item | u149 | 7 | 8 | Get::PassArrangements u148 | null | null |

The `duration` column lets us see that the `TopK` operator is where we
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
The `duration` column lets us see that the `TopK` operator is where we
The `duration` column shows that the `TopK` operator is where we

| wins_by_item | u149 | 8 | null | Arrange 7 | null | null | null | null |
| wins_by_item | u149 | 7 | 8 | Get::PassArrangements u148 | null | null | null | null |

Here, the hinted `DISTINCT ON INPUT GROUP SIZE` is `255.0`. We can re-create our view and index using the hint as follows:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Here, the hinted `DISTINCT ON INPUT GROUP SIZE` is `255.0`. We can re-create our view and index using the hint as follows:
Here, the hinted `DISTINCT ON INPUT GROUP SIZE` is `255`. We can re-create our view and index using the hint as follows:

Running this query on an auction generator will produce results that look something like the following (though the specific numbers will vary, of course):


| name | global_id | lir_id | parent_lir_id | operator | duration | count |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, markdown table like this won't preserve the spacing(I know, with all your nice indentation logic ... come on, markdown) :shakes-fist:
https://preview.materialize.com/materialize/30899/transform-data/troubleshooting/#attributing-computation-time

When I get back, I can move separate these into a data file and a table
where in the data file, can use ```mzsql annotation to maintain spacing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-docs Area: documentation A-optimization Area: query optimization and transformation T-observability
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants