The Vue core PR queue

[skirtles-code]

I'm concerned about the health of the Vue core repo. I'm hoping raising awareness of the problem might help.

I know it's annoying when people claim that Vue is dead or dying. It clearly isn't dying. But the state of the PR queue is a cause for concern.

---

TLDR: I think the underlying problem is that the merge process is overly reliant on Evan. Isn't it possible for the 'simple' fixes to be merged by other members of the team?

---

Some analysis

The PR queue currently contains just under 500 open PRs. But that number lacks context.

For a while now, I've felt that PRs aren't being merged reliably. Issues get opened, PRs get submitted, but they then just sit around unmerged. In some cases the changes are big and complicated, but a lot of them are small, simple changes that seem pretty easy to review and merge.

Rather than just relying on my gut feeling, I decided to do some analysis.

I thought it might be interesting to plot some charts that show the proportion of PRs that have been merged for each month. The pictures below show the charts for Vite, Svelte, React and Vue.

To learn more about what exactly those charts are showing, see https://gist.github.com/skirtles-code/41af2c457c51b6d09348f128c89ac42a.

The quick summary is:

• Only open and merged PRs are considered. The charts ignore closed and draft PRs.
• Vite and Svelte have really good merge rates for PRs. They tend to be over 90%.
• React isn't doing quite as well, but it still tends to be over 70%.
• Vue hasn't reached 70% in 18 months and is often below 40%.

Those projects aren't necessarily comparable and PRs can vary significantly, but I think these charts do seem to confirm what my instincts were telling me. Vue has a long-term problem with keeping on top of the PR queue.

Is this really a problem?

I would say so.

Vue's community can solve a lot of problems by itself. We can help each other on Discord, write blog posts, add libraries to the ecosystem, create RFCs, and open PRs. But merging PRs into Vue's core is one of the few bottlenecks that normal members of the community are powerless to solve.

This leads to a lot of frustration, not just for those encountering bugs, but also for contributors. My personal experience has been that it's quite demotivating to fix problems and then see those fixes sit around unmerged for 10 months. There are several other bugs that I'd like to fix, but it seems like a waste of effort if they won't be merged. I also worry that the bugs I'm fixing are already fixed by somebody else, but it's very difficult to find duplicates when there are so many open PRs.

The large number of PRs also leads to a lot of merge conflicts, which creates even more work.

Obviously people need to be patient about PRs being merged, but how long is too long? Many of us are trying to encourage companies to use Vue for their projects. As an independent framework, one of Vue's selling points should be that it is able to move quickly and get fixes merged, without any corporate bureaucracy holding it back. In practice, even if you do all the work required to create a fix, it's likely that it'll sit around unmerged indefinitely for no apparent reason.

I've seen concerns like these raised before. e.g.:

• 300 PRs #7050
• 400 PRs #8560

I'm aware of some of the reasons why the Vue core repo sometimes goes quiet: finite resources, the cost of task switching, etc. But those reasons always seem based on the premise that Evan is the only person who can merge the PRs.

Why is there such a heavy reliance on Evan to do all the merging? Can't this responsibility be shared to allow bugfixes to continue when Evan is busy with other things?

I think a lot of us are excited about the ongoing work on Vite, and we all admire Evan's ability to create so much of the software we love. I feel like everybody wins if Evan is free to pursue working on bigger things, while other people hold down the fort and do the running repairs to Vue.

If the core team needs more help from the community then I think there'd be plenty of people willing to put in the time and effort to help out. But currently it isn't clear how the community can help.

Having more people doing reviews doesn't seem like it'd make much difference. There are already people doing reviews. The PRs just aren't getting merged.

Maybe all the 'easy wins' are being merged?

One possible explanation for the merge rates seen in the chart above is that the 'easy wins' are being merged and only the more difficult PRs are being left open. I've spent a while looking through the open PRs and I don't think that's what's happening.

I ran some scripts to find 'small' PRs. Being small doesn't necessarily mean that they're easy to merge, but they do tend to be easier to review than larger changes.

• 20 PRs that only change the tests
• 44 PRs making small changes, plus tests
• 60 PRs making small changes, but without tests

From looking through these PRs myself, it's clear that some of them are junk and should just be closed. Even that would help make the queue a bit more manageable. But there are plenty of others in those lists that look like good changes with relatively low risk.

Some PRs need a new minor version, but most can just go in a patch release. It appears that work on 3.4 has begun, and that's great, but personally I think the large backlog of PRs for small bugs and DX improvements is more important.

Maybe the old PRs are no longer relevant?

Another explanation could be that those old PRs aren't really relevant and it'd just be a lot of unnecessary work to close them. Again, I don't think this is an accurate assessment. It's true that many of the old PRs will now have merge conflicts, but in many cases the problems they're fixing still exist and the proposed fix is still worth pursuing.

---

Thanks for reading.

[daniluk4000]

Thank you for this hard work! The effort you put in this discussion is amazing.

I still believe that as PRs count are growing, it comes to nearly an impossible stage of reviewing them all. Community voting system (prioritizing of most commented/reacted issues) could help in some way, as well as solutions you offered above - junk and small PRs that can easy be closed/merged. All of those solutions require constant and regular work with PRs/Issues.

I've stated in my 400prs discussion will not create any "500 PRs" this time as I really did hope since "300 PRs" discussion almost a year ago that it will never come to this. Glad to see someone created this and did great work in summarizing the problem. Bug fixes, perfomance and DX improvements... All of them are waiting for their time.

I have also taken a look on approved PRs and 94 PRs are approved now. I really do hope that it could mean that those PRs will be merged in next release, as it is like 20% of total count. Some of them were approved more than a half year ago though...

[yyx990803]

Thanks for the analysis. I very much agree the concern is justified - and I agree that the bottleneck is on me.

Not trying to make an excuse, but I've been spending the past few months working on something substantial but not directly Vue related. During this period I have been trying to get the team to help review PRs (hence the many "ready to merge" ones), but I haven't been able to properly sit down and do a merge session for a while.

The necessary change is to establish a process where core team members get the permission and confidence to merge PRs without me being the bottleneck - we've already started preparing the preliminary measures for that: ecosystem-ci, a documented and more detailed PR review process, the "Ready to merge" label... what's missing is the go-ahead for team members to do merges. I just discussed this with @sodatea yesterday - I will be working with @sodatea , @sxzz, @pikax to get us there.

[pikax]

I agree with the sentiment and already voiced my concerns, I'll be tacking a more active role at the least until we get this situation under control (hopefully 🤞)

I'm going (it will take some more time, still in page 3 or 4) through all nearly 500 PR and add them to the new project "Vue 3.4" (the project title might be misleading, not every single PR will land in 3.4), this will allow us (team members and users alike) to have a better view of the state of Vue PR. I believe this admin work is very useful with a gigantic amount of PR that we need to handle.

The rules (roughly):

• If a PR has conflicts with main branch is assigned to Stale column
• If it has not addressed comments is assigned Waiting
• Needs review for unreviewed PRs

I'll like to get a closer view at the Slate PRs and then I can bring them up to the team to discuss we we close or work on them, since most of them are fixing issues.

This still does not solve the bottleneck, but this might make PRs a bit more "chewable", also a higher view of Vue current state.

Hopefully just by getting the PRs merged should close most of the issues (I still need to go through them afterwards), then we can focus on getting the issues under control.

Apologies for just bringing "admin work" updates.

[jacekkarczmarczyk]

add them to the new project "Vue 3.4" (the project title might be misleading, not every single PR will land in 3.4)

Can you clarify what do you mean by that? I understand that some PRs might be rejected after the review, some may be postponed to 3.5 or later, but does this project contain also PRs that are intended to be merged in 3.3.x?

[pikax]

Sure... currently I'm adding all of the PRs to the project for visibility, not discerning if the PR is minor, major, etc.

Meaning, even if the PR is "Ready to merge" it might not be going in in 3.4.x.