Reduce lag experienced when expanding doc table rows

[Bargs]

Fixes #7381

There were a couple of issues making row expansion slow in the doc table.

Here's a timeline snapshot before I made any changes:

We see giant forced reflows, courtesy of the ACE editor. ACE is used to display the JSON view of the doc, which we don't even show by default. So commit # 1 prevents rendering the JSON view until the user actually clicks on the JSON tab.

Now the timeline looks like this:

Slightly better, but now fixed_scroll.js is causing more forced reflows. This is because it's calculating DOM element widths during the digest cycle.

The purpose of the fixed scroll directive is to add a horizontal scroll bar to the bottom of the window which proxies scroll events to the target element's scroll bar. This is useful in cases like the doc table, where the real container's scroll bar might be waaay at the bottom of the page (and in fact inaccessible due to infinite scroll loading).

I thought I might be able to give the doc table container a height equal to the remaining space on the page using flexbox, pulling the real horizontal scrollbar up to the bottom of the window. It worked, but when I switched everything to flex the performance was even worse.

So I landed on the solution in commit # 2, replacing the angular watcher with a timeout that checks the element's width at regular intervals. Timeouts are ugly, but the existing code was basically already on a timeout, and this new solution has the benefit of not being tied to the digest cycle where it'll slow down the page during user interactions.

The final timeline:

Total time to handle the click event went from ~700ms to ~150ms.

I'd be open to other solutions, but in the interest of time I figured this was an ok compromise and an improvement over what we already have.

[Bargs]

Hmmm, probably a timing issue in the tests, I'll check that out tomorrow.

[Bargs]

Tests should be good now

[lukasolson]

Thanks for the clarifying comment. 👍

[lukasolson]

Hmm, is there anything we can listen to (like window resize) to invoke this, or do we really have to do it every half-second?

[Bargs]

I thought about window resize, but there are other situations where the container width might change, like adding/removing columns from the doc table. I also tried the scroll event, thinking it might fire when the container changes size since the scroll bar would also be changing size, but no dice :(

afaik there is no event that'll tell us when the container has changed size 100% of the time

[Bargs]

On the plus side, as long as the content of the page hasn't changed the timeout function is pretty lightweight. Definitely still open to other ideas though.

[lukasolson]

Blah, I really like the idea of just removing the fixedScroll directive altogether. I really don't see its utility. You can scroll horizontally without having to use the scroll bar (using a trackpad, or shift+mouse wheel). I think the only reason I ever want to use the scroll bar is when I need to scroll a really far distance quickly, and that isn't a valid use case here. I think this code adds unnecessary complexity and performance issues, and sometimes it behaves kinda unexpectedly. I'm definitely +1 on removing this code altogether. Thoughts @Bargs @weltenwort @spalger @epixa?

[Bargs]

I'd really love to just rip it out, but I do worry about less savvy users not knowing about shift+scroll :/

[weltenwort]

Are there any situations where we actually want to horizontal scrolling to happen? Maybe it would be more beneficial to invest some effort into avoiding that in general (e.g. by performing more wrapping, adjusting styles via media queries, etc...)?

[Bargs]

I tried to come up with some creative ideas today but nothing really stuck. I also played around with flexbox some more but to no avail. I wish CSS grid was available in browsers already.

Open to suggestions.

[weltenwort]

So if my understanding is correct, the fixedScroll thing is needed because we perform horizontal scrolling on the doctable container but vertical scrolling on the body. Using flexbox you attempted to move the vertical scrolling to the doctable container as well?

Doing that sounds very reasonable and might be a good idea for all columns in the discover view. I'm always annoyed when the field selector column or even the time picker scroll out at the top.

I wouldn't be surprised if the bad flexbox performance stems from the inefficient dom manipulation that is being done in some places inside the doc table or from unclean javascript code that triggers reflows/layouts by performing measurements. Maybe css tables (display: table) would be worth investigating while we're waiting for grid layouts?

[Bargs]

It's certainly possible that our code or markup is interacting poorly with flexbox, but I think getting it to work well, if possible, is high fruit. It also has some other issues as well. Getting the doc table container to fill the height of its parent without growing to the size of its content requires specific styles be applied up the DOM hierarchy all the way to the body. The more I play with it the more I feel flexbox isn't good for such high level layout.

I don't think tables will help us here because there's no way to get a table row to fill its container without growing with the size of its content, which is what we need the doc table to do so that it fills all the vertical space in the window while remaining scrollable if the content is larger than the window. Let me know if I'm misunderstanding what you meant though.

As it stands, I think this PR improves performance without making fixedScroll any worse than it already is. If you guys agree, I'd like to get this merged and discuss larger rewrites of the doc table as a separate issue. It sounds like we all would like to see it cleaned up, but I think that work needs to stand on its own merits because this small issue doesn't warrant such an effort.

[Bargs]

I just noticed this PR while reading my github notifications. I'm gonna test it tomorrow to see if I can use it to get rid of the timeout.

[Bargs]

@lukasolson @weltenwort turns out ResizeObserver only emits events for changes to content area, not scroll area. The content width of the doc table wrapper doesn't change since it's scrollable. So implementing a solution with ResizeObserver would add significant complexity to the directive and I'm not confident it would work correctly in all cases. You'd have to know for a fact which element is going to overflow the container, apply the observer to that element, and somehow let the container know to update its scroll bar. But this would fail if any other child element happened to overflow, and it would also fail if the window changes size while the content remains the same.

So... I guess I'm back to thinking my previous implementation is the best we can do at the moment. I'd like to hear your feedback on my previous comment!

[weltenwort]

Too bad about the ResizeObserver.

After reading a bit more about it, I have to agree that display: table probably won't help us here. This sounds like the exact use case for flexbox with everything except for the discover table having a fixed height while the table being flex: 1 0 0. Looking at the DOM, the body > .content > .app-wrapper > .app-wrapper-panel > .application hierarchy already is made of flexboxes that fill the full height. Too bad that extending the flexbox model through the rest of the .app-container would have such negative performance impact.

Regarding the timeout solution: It is indeed probably not worse than before. Another option would be to _.debounce the watcher in the previous implementation.

[Bargs]

@weltenwort can you elaborate on what you mean by debouncing the watcher? Are you referring to the callback, or the functions that return the watched values?

The watched functions are the cause of the forced layouts since they check the width of the element. If I debounce those on the trailing edge they'll return out of date values. If I debounce them on the leading edge the forced layout will still occur during the digest cycle. Maybe I'm misunderstanding something though.

[weltenwort]

True, it might be problematic to debounce the functions that return the watched values. I was thinking of a compromise in which we perform the comparison inside the checkWidth function the way you implemented it in this PR so far, but debounce it and trigger it unconditionally during the digest cycle. That way the recalculation would still be bound to changes in the displayed content instead of constant polling. Not sure if the noop of the debounced function is still too costly to execute on every digest though.

[Bargs]

@weltenwort just pushed a commit using debounce. Is this what you were thinking of? Seems to work well, I'm good with that implementation if you are.

[weltenwort]

Yes, that's what I meant. Sorry for not being clear enough.

[weltenwort]

Does this still work when using _.debounce instead of $timeout?

[Bargs]

Our angular friendly version of debounce actually uses $timeout under the hood so I still need to flush those timeouts to get the tests to work.

[Bargs]

I've removed the timeout flushing and mocked debounce instead so that the tests don't need to know anything about debounce's implementation.

[weltenwort]

TIL we have a custom debounce. Thanks!

[weltenwort]

For slightly more performance optimization these reassignments could go into the if block. I'm not in favour of those state variables in general, but can't think of a better way. 😉

[Bargs]

Done

[Bargs]

@weltenwort @lukasolson so, how are we looking?

[weltenwort]

LGTM, good improvement for such small code changes

[lukasolson]

LGTM!