[Reporting] support long dashboards via multiple snapshots#248566
[Reporting] support long dashboards via multiple snapshots#248566pmuellr wants to merge 14 commits intoelastic:mainfrom
Conversation
|
Project deployments require a Github label, please add one or more of |
|
Cloud deployments require a Github label, please add |
|
Pinging @elastic/response-ops (Team:ResponseOps) |
…otstrap' locally and then commit the changes and push to your branch
|
Note for anyone looking at the "new" dep From main: |
|
/ci |
|
I'm not having any success generating a moderately long report using the current level of code. Here's a log of what I'm seeing: So the report is 1430 wide by 8881 high. Each pixel will be 4 bytes - RGBA, and we're at zoom:2, which means double the width and height (so 4 times bigger), so 1430 * 8881 * 4 * 4 == 203197280. I originally read that as 20MB, but it's 203,197,280 - 200MB! We allocate space for that right when we log My current guess is that node js has allocated that memory that chromium needed to build the screenshot - it will need a fair amount of memory to get a shot that big as well. I'll try with a smaller tile size, but I'm not hopefult this technique is going to be practical. And it makes me wonder if even a native png tile stitching function would use significantly less memory. Just not a great "job" for a low-end, low-memory non-gpu linux box? |
💔 Build Failed
Failed CI StepsTest Failures
Metrics [docs]
History
|
|
Going to close this issue and fix the "long report" issue with PR #248785 . The "tiling" approach in this PR does work, but requires a lot of memory. It may be useful in the future if we move to some kind of centralized printing service where memory isn't as much of an issue. |
resolves #138812
Summary
When "long" dashboards have reports generated, there are often artifacts, as described in the issue above.
The problem is that Chrome does not support generating a screenshot greater than 16K rows. Because we also use a zoom setting of 2, that only allows dashboards < 8K from not having artifacts.
To fix this, instead of taking a single snapshot of the entire dashboard, we will partition the dashboard into a vertical set of "tiles" of a fixed height, and take a snapshot of each tile. We then stitch them back together with
@pdf-lib/upng.We'll just do this for dashboards that are so long they need to be generated this way.
Note that it would be possible to also sort very WIDE dashboards, with a fair amount of work, and probably extremely slow - it's easy to compose vertical tiles of images, will be harder to compose them vertically as well.
New dependency
@pdf-lib/upngPurpose: What is this dependency used for? Briefly explain its role in your changes.
We extract "tiles" from a large image - basically a subset of the image. Then we combine them to form the full image. To do this, we need code that converts
.pngformatted image to RBGA pixels, and vice versa. This package provides that.Justification: Why is adding this dependency the best approach?
Because this package is already indirectly included in Kibana via
pdf-lib.Alternatives explored: Were other options considered (e.g., using existing internal libraries/utilities, implementing the functionality directly)? If so, why was this dependency chosen over them?
Actually, I'd prefer to use a native solution, like
sharp, but that will no doubt add more size, and I think we're trying to avoid native packages at this point.Existing dependencies: Does Kibana have a dependency providing similar functionality? If so, why is the new one preferred?
Not one that is explicitly depended on. we actually do have
sharpand alsopngjsasdevDependencies, but moving those intodependenciesagain, adds more size and product runtime deps. Feels better to use the existing implicirly depended on, already shipped package.Checklist
Check the PR satisfies following conditions.
Reviewers should verify this PR satisfies this list as well.
backport:*labels.