Performance 2020

[foxdavidj]

Part II Chapter 9: Performance

Content team

Authors	Reviewers	Analysts	Draft	Queries	Results
@thefoxis	@dimension85 @borisschapira @estelle @zeman @rviscomi @OBTo @noamr @ashrith-kulai @Zizzamia @exterkamp	@max-ostapenko @dooman87	Doc	*.sql	Sheet

Content team lead: @thefoxis

Welcome chapter contributors! You'll be using this issue throughout the chapter lifecycle to coordinate on the content planning, analysis, and writing stages.

The content team is made up of the following contributors:

• Authors
• Reviewers
• Analysts

New contributors: If you're interested in joining the content team for this chapter, just leave a comment below and the content team lead will loop you in.

Note: To ensure that you get notifications when tagged, you must be "watching" this repository.

Milestones

0. Form the content team

• Jul 6th: Project owners have selected an author to be the content team lead
• Jul 13th: The content team has at least one author, reviewer, and analyst (minimally viable team formed)

1. Plan content

• Jul 20th: The content team has completed the chapter outline in the draft doc
• Jul 27th: Analysts have triaged the feasibility of all proposed metrics

2. Gather data

• Jul 27th: Analysts have added all necessary custom metrics and drafted a PR to track query progress

• Aug 1 - 31: August crawl

• Sep 7th: Analysts have queried all metrics and saved the output to the results sheet

3. Validate results

• Sep 14th: The content team has reviewed the results sheet

4. Draft content

• Nov 12th: Authors have completed the first draft in the doc
• Nov 26th: The content team has prototyped all data visualizations

5. Publication

• Nov 26th: The content team has reviewed the final draft, converted to markdown, and filed a PR to add it to the 2020 content directory

• Dec 9th: Target launch date

[dimension85]

I am interested in the reviewers role for this topic - Phil

[borisschapira]

I'm afraid I'm not fluent enough in English to create the content. I can help reviewing, though.

[rviscomi]

@thefoxis thank you for agreeing to be the lead author for the Performance chapter! As the lead, you'll be responsible for driving the content planning and writing phases in collaboration with your content team, which will consist of yourself as lead, any coauthors you choose as needed, peer reviewers, and data analysts.

The immediate next steps for this chapter are:

1. Establish the rest of your content team. Several other people were interested or nominated (see below), so that's a great place to start. The larger the scope of the chapter, the more people you'll want to have on board.
2. Start sketching out ideas in your draft doc.
3. Catch up on last year's chapter and the project methodology to get a sense for what's possible.

There's a ton of info in the top comment, so check that out and feel free to ping myself or @OBTo with any questions!

To everyone else who has been nominated:

@logicalphase
@Zizzamia
@noamr
@dimension85

we'd still love to have you contribute as a peer reviewer or coauthor as needed. Let us know if you're still interested!

[rviscomi]

@dimension85 @borisschapira thank you! I've added you both as reviewers.

[estelle]

I can also help out.

[rviscomi]

Thanks Estelle! I've also added you as a reviewer.

[dimension85]

Excellent, thanks for adding me to the team - this is the first time I have been involved with this project and am looking forward to contributing.

[logicalphase]

Would love to co-author this section. Please let me know if I can help.

[zeman]

Happy to review again this year!

[foxdavidj]

An all-star team again this year. Really excited for this 🎉

[noamr]

On Wed, 1 Jul 2020 at 21:52 Rick Viscomi ***@***.***> wrote: @thefoxis <https://github.com/thefoxis> thank you for agreeing to be the lead author for the Performance chapter! As the lead, you'll be responsible for driving the content planning and writing phases in collaboration with your content team, which will consist of yourself as lead, any coauthors you choose as needed, peer reviewers, and data analysts. The immediate next steps for this chapter are: 1. Establish the rest of your content team. Several other people were interested or nominated (see below), so that's a great place to start. The larger the scope of the chapter, the more people you'll want to have on board. 2. Start sketching out ideas in your draft doc <https://docs.google.com/document/d/1EeUJ88PS8Ms9XUrNIpM2tpm5Ad_K682ZMVfc24TM02w/edit?usp=sharing> . 3. Catch up on last year's chapter <https://almanac.httparchive.org/en/2019/performance> and the project methodology <https://almanac.httparchive.org/en/2019/methodology> to get a sense for what's possible. There's a ton of info in the top comment, so check that out and feel free to ping myself or @OBTo <https://github.com/obto> with any questions! To everyone else who has been nominated: @HyperPress @Zizzamia <https://github.com/Zizzamia> @noamr <https://github.com/noamr> @dimension85 <https://github.com/dimension85> we'd still love to have you contribute as a peer reviewer or coauthor as needed. Let us know if you're still interested!

Great! Would love to review.

…

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#905 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAVT5QSPOAQUHH4N2SRL4TRZOAYLANCNFSM4OJ2CP2A> .

[ashrith-kulai]

I would like to be a reviewer for this topic.

[Zizzamia]

Happy to join as a reviewer, looking forward to collaborate with the all Content team. 🌲 🚀 🌕

[thefoxis]

👋 hi everyone!

thanks again @rviscomi and @OBTo for selecting me to lead this effort. looks like we have a solid team of reviewers but we're short on analysts. should we be looking for people swapping areas of responsibility or look for new people to join?

I'm going to re-read last years chapter and reflect on what has happened in perf within the year. I think it'd be nice to speak to some new developments in the space, for example core web vitals or the shift in performance score algorithm. but it also depends on what sort of trends/data we're able to discover via the archive. I'll jot some notes down in the google doc within the next couple days. JavaScript is also always good to cover since it affects perf/UX greatly, so we could be looking at TBT & TTI. none of these were really covered last year so there would be no duplication (the previous report mentioned FID though, which is only relevant to people using RUM solutions/not Lighthouse).

if anyone has any ideas/suggestions, I'm all 👂

[Zizzamia]

@thefoxis yeah I agree 100% ! 🔥

Other points on top of my mind are:

• Network Information API: how we use effectiveType to detect 2/3g connections.
• Device Memory 1 & Hardware Concurrency: how we use deviceMemory in conjunction with hardwareConcurrency to understand the experience between LowEnd vs HighEnd devices.
• Element Timing API: the upcoming element option in the Performance Observer is something interesting for potential custom metrics.

What else? 🤔
One thing I am kind of curious to reflect is the different implications of TBT and LCP performance between SPA and SSR, and how websites are handling those performance trade-offs.

That's it for now, but I am sure the rest of the team has more angles we could deep dive 🌲🚀🌕

[thefoxis]

@OBTo thank you for letting me know! I didn't see it. I doubt I can do this by Nov 9, I've been sick for the past week and still I am not well (fortunately, it's not covid 🤪 ) so it delays my ability to write a bit. that being said, I shouldn't need a lot of time to produce a draft, I'll post here once it's ready 👍

[rviscomi]

Sounds great, hope you feel better!

[max-ostapenko]

@thefoxis get well soon!
We are looking forward to receiving your feedback again.

[thefoxis]

@OBTo @rviscomi I'll have a complete draft ready sometime next week. there's a bit more to be written than I initially expected. of course, you can keep an eye on the draft in the doc, as I'm writing directly there.

@max-ostapenko @dooman87 I'd appreciate some help with the charts, since some are only representing a single metric versus complete readings and my spreadsheet skills turn out to fall a bit short here 😅 I'm going off the 2019 almanac, so the charts for each metric are:

• metric by device
• metric by geo
• metric by effective connection type

The connection type chart wasn't portraying quite what I wanted (as per 2019 chapter), I tried playing with it, but didn't go far. This is what I'd love to have for each metric:

I'm not entirely sure how to generate the charts for geo + device for other metrics than LCP so if you could point me in the right direction or help out here, that'd be great 😸

[rviscomi]

Hi @thefoxis I'm excited to see this coming along! I'm also happy to help with the data viz, feel free to @ me on whichever charts you'd like help with.

[max-ostapenko]

@thefoxis As per your question: please check the chart here, if I understood you correctly: https://docs.google.com/spreadsheets/d/164FVuCQ7gPhTWUXJl1av5_hBxjncNi0TK8RnNseNPJQ/edit#gid=306222260
(I changed columns to the corresponding ones in a data range)

I'll appreciate if we can jump to sheet comments for exported results discussion. Please add comments on what needs adjustments and if any visualisations are missing.

[thefoxis]

@rviscomi can you point me to the CrUX's categorisations of good|needs improvement|poor? I know where those ranges live for LH/perf score calculation but I'm not quite sure if it's identical in CrUX's dataset 🤔 I just want to be able to reference it for each metric since you can't tell from the charts.

[rviscomi]

@thefoxis the thresholds used by CrUX for the CWV are defined at https://web.dev/vitals/#core-web-vitals

We could also edit the legends of the charts to say something like "Good (<100ms)" and "Poor (>=300ms)".

[thefoxis]

@rviscomi gotcha! that's exactly what I had in mind :) what about FCP and TTFB values? where do those come from?

FYI I only have the geo sections (pending charts) + conclusion left, so if you want to read the draft as is and leave comments, you're more than welcome to! cc @OBTo

[rviscomi]

FCP and TTFB are more subjective and don't have official documentation. For reference the thresholds used by CrUX for fast/slow are:

FCP: 1.5s, 2.5s
TTFB: 0.5s, 1.5s

[borisschapira]

A quick word to say that I'm starting the review and posting comments on the doc.
I am French 🇫🇷 , and our culture teaches us to be quite direct. So I'll try to formulate things in a way that is appropriate, but I may not do it well. But I don’t mean to be rude. Even if I comment some things as food for thoughts, it's only out of constructive criticism. Because I find that the work that has been done in the data analysis and writing is already exceptional.

[thefoxis]

@borisschapira no sweat 🙂 English is my second language too and the Poles are apparently also very direct so I know what you mean. any feedback appreciated! I already responded / acted on some of your comments. all good finds 👍 thank you! ⚡

[exterkamp]

Thanks for looking at my comments @thefoxis! I had a small nitpick with CLS, and ended up reading the whole chapter because it was such a good read. Really well written all round 💯

[rviscomi]

Thanks for the review @exterkamp. Could you add yourself to the contributors list? (alphabetical by first name)

[noamr]

A great read! I'm reviewing the material now, prepare for some comments :)

[thefoxis]

thanks @noamr and @exterkamp! good comments all around, I’m addressing it along @rviscomi’s feedback :) a few more bits to go, but I can see the end now 🎉 glad you enjoyed reading so far!

[thefoxis]

@OBTo @rviscomi I believe I addressed all existing feedback and added a conclusion. FYI I removed captions since I figured with the markdown format I'll just submit them with the PR; let me know if that's suitable. feeling pretty good about it! let me know if there's anything else.

@dimension85 @borisschapira @estelle @zeman @rviscomi @OBTo @noamr @ashrith-kulai @Zizzamia @exterkamp if you'd like to add any more/first thoughts to the draft, you're welcome to. I'm not sure how much ability I will have to address big shifts in content, but fixes / clarifications / smaller suggestions shouldn't be an issue with the schedule I reckon 😅

[foxdavidj]

@thefoxis Sounds like a solid plan to me. I'll look at it again this evening to see if I pick up on anything before we start the editing process :)

[rviscomi]

@thefoxis I'm excited to see this moving along! Thank you for your hard work getting it to this point. Can you open a PR to submit the markdown version of the draft? (the final milestone in the initial checklist)

I've stubbed out the markdown file already with some early metadata and you can see the 2019 version for reference. I'm happy to help convert the data viz to the correct figure format, if you leave them as TODOs/placeholders in the markdown file. Please also update the chapter metadata to remove anyone who hasn't contributed, add your bio, add features quote/stats, etc.

[thefoxis]

@rviscomi all done in #1608 👍

[foxdavidj]

@thefoxis Just finished reading the chapter and quite enjoyed it. Great job. Just had a few small questions and suggestions I left in the Google Doc :)