Improve ranking system

[web-dev-sam]

Describe the bug
I was interested in how the rank was calculated and wanted to know how to rank up. I found out that you use a function called normalcdf to normalize the calculated score. Its basically impossible to get a rank below A+ since normalcdf(0, 231, 7) * 100 returns 51.55 which is in the A+ area.

The code can be found here.

TODOS (Added by @rickstaa)

• Decide on a new grading system, Change the ranking/level system #2265.
  • Create PR that adds this new grading system.
• Merge Missing term in calculation of TOTAL_VALUES. #2116.
• Merge Ranking System v2 #1186.
• Create PR that shows the top user percentage on hover (see Grade description in the report card. #2157).

[nrchandan]

Created a PR that demonstrates this: #459

[MarvinJWendt]

This is still very relevant

/cc @anuraghazra

[SuperFola]

Commenting to get this issue fixed

/cc @anuraghazra

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[SuperFola]

This is still an issue

[magneticflux-]

I think it might be broken in more ways than one.

• The implementation doesn't seem to work right: https://stackoverflow.com/questions/5259421/cumulative-distribution-function-in-javascript/5263759#comment114047028_5263759
• The CDF used is for the normal distribution, which I doubt GitHub stats adhere to. Perhaps a power-law distribution function would be a better fit? https://en.wikipedia.org/wiki/Power_law#Power-law_probability_distributions

Here's another example of the weird ranking behavior: #330 (comment)

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[magneticflux-]

Still an issue

[anuraghazra]

I think it might be broken in more ways than one.

• The implementation doesn't seem to work right: https://stackoverflow.com/questions/5259421/cumulative-distribution-function-in-javascript/5263759#comment114047028_5263759
• The CDF used is for the normal distribution, which I doubt GitHub stats adhere to. Perhaps a power-law distribution function would be a better fit? https://en.wikipedia.org/wiki/Power_law#Power-law_probability_distributions

Here's another example of the weird ranking behavior: #330 (comment)

I agree. rank evaluation is not correct and too biased.
Sorry but I have no good ideas on how to make it less inaccurate and improve it (i suck at maths)
Would love to hear any opinions and possible implementation details.

cc @magneticflux-

[CrafterKolyan]

@anuraghazra I've made an implementation of one of my ideas in #960 with an explanation inside PR

[rickstaa]

@LucBerge and @ZjzMisaka, I agree with you that the ranking needs to be improved. There have been multiple proposals and PRs open that try to improve the ranking:

• Change the ranking/level system #2265
• Yet Another Proposal for Calculate Ranks #1560
• Suggestion for more ranks or a different raking system. #1579
• Improve ranking system #455
• Ranking System v1 #960
• Ranking System v2 #1186

I think three things can be improved about the ranking:

• Switch to a more commonly used grading system (i.e. A-F, A+ to F or % based).
• Show top-users percentage on hover (see Grade description in the report card. #2157).
• Improve the rank calculation (see Missing term in calculation of TOTAL_VALUES. #2116 and Ranking System v2 #1186).

I asked @anuraghazra for his opinion at #2265.

[rickstaa]

TODOS

• Decide on a new grading system, Change the ranking/level system #2265.
  • Create PR that adds this new grading system.
• Merge Missing term in calculation of TOTAL_VALUES. #2116.
• Merge Ranking System v2 #1186.
• Create PR that shows the top user percentage on hover (see Grade description in the report card. #2157).

I also added this list to the top of this issue.

[ZjzMisaka]

Hi @rickstaa,
I understand your caution in modifying the ranking system, but I believe that making changes is always good.
Can some temporary modifications based on existing ranking system be made before receiving a response from @anuraghazra, at least to change the situation where the ranking is stuck in A+?
I think ranking can reflect the changes made by user from the initial state, rather than the gap between users.
Therefore, I don't care if my ranking will change to F after the ranking system changes, I prefer to see my progress dynamically.
After all, modifying a ranking system cannot satisfy everyone. If you insist on giving everyone a "beautiful" ranking, the system will ultimately become non referential.
Thanks.

[rickstaa]

Hi @rickstaa, I understand your caution in modifying the ranking system, but I believe that making changes is always good. Can some temporary modifications based on existing ranking system be made before receiving a response from @anuraghazra, at least to change the situation where the ranking is stuck in A+? I think ranking can reflect the changes made by user from the initial state, rather than the gap between users. Therefore, I don't care if my ranking will change to F after the ranking system changes, I prefer to see my progress dynamically. After all, modifying a ranking system cannot satisfy everyone. If you insist on giving everyone a "beautiful" ranking, the system will ultimately become non referential. Thanks.

I agree with you. I already started working on this last week. Not only that, but I will merge #1186 (comment) as soon as @francois-rozet has resolved some small issues I had with that pull requests 👍🏻. @anuraghazra can always change it to his liking later.

[rickstaa]

Just a heads-up. I just merged #1186 which, in my opinion, is a great improvement compared to the old ranking. You are welcome to open pull requests if you think the weights can be further improved 👍🏻. There is also a proposal open to change the rank levels. You are welcome to put your ideas in #2265.

[rickstaa]

Closing this as completed.

[rickstaa]

Re-opened so that people can provide feedback on the new ranking system and create improvements.

[lorezyra]

Would be good to update the readme for this project to include an explanation on the ranking system and meaning of the letters.

[rickstaa]

Would be good to update the readme for this project to include an explanation on the ranking system and meaning of the letters.

You're right! The documentation will be updated when #2749 is merged 👍🏻. You are welcome to suggest any improvements.

[objektwerks]

So with the new ranking algo, a former ranking of 'S' for 12K plus annualized commits earns a modest ranking of 'B'. My ego may never recover. ;)

[rickstaa]

So with the new ranking algo, a former ranking of 'S' for 12K plus annualized commits earns a modest ranking of 'B'. My ego may never recover. ;)

I'm afraid you are an outlier 😮. You made as many commits in one year as I did in my whole GitHub career 🔥. In my opinion, PRs are preferred over small commits since it improves code collaboration 🤔. I do understand that this is opinioned and that in some cases it is better to work with commits. Can you maybe check if @francois-rozet new ranking mechanism improves your rank (see #2762)?

[rickstaa]

@objektwerks I checked and #2762 will improve the behaviour. Also by making some pull requests you can increase your rank rapidly.

Old ranks

Available ranks are S+ (top 2.5%), S (top 10%), A+ (top 25%), A (top 50%), B+ (top 75%) and B (everyone). This ranking scheme is based on the Japanese Academic Grading system. It is calculated as a weighted sum of ranks for each statistic (number of commits, pull requests, issues, stars and followers) based on the cumulative distribution function of an exponential distribution. The implementation can be investigated at src/calculateRank.js. The circle around the rank shows the rank completion percentage compared to the maximum rank.

Ranks (after #2762)

S, A+, A, A-, B+, B, B-, C+ and C

[objektwerks]

@rickstaa Thanks for the explanation and tip!

I tried the #2762 option, which did reflect the ranking circle above -- with one minor difference in total commits. It now reads: Total Commits: 29.4k So the count goes back further than 1 year.

You all might consider 2 rankings -- 1) pr-driven and 2) commit-driven. Just a thought. :) Cheers!

[francois-rozet]

@rickstaa Thanks for the explanation and tip! You all might consider 2 rankings -- 1) pr-driven and 2) commit-driven. Just a thought. :) Cheers!

Hello, I think making a lot of commits should not be a goal. Having 10000 commits (especially if named "fix") that change one or two lines, is a bad development practice. PRs are much better indicators of meaningful contributions.

[objektwerks]

@francois-rozet Everyone has different development styles. Hence the multi-style ranking suggestion. :)

[rickstaa]

@francois-rozet Everyone has different development styles. Hence the multi-style ranking suggestion. :)

I agree with @francois-rozet on this one 😅. Although everybody has a different development style, from experience I noticed that these small commits hurt maintainability when working on projects with other people. Nevertheless, since this is an OS (community based) project, if enough people support your idea we might look at it (see #1935 for the most popular feature requests) 👍🏻.

[objektwerks]

@rickstaa

I work alone on multiple projects. And small units of work make sense for my workflow. Not everyone is an OSS warrior. ;)

On a commercial or open source project, with several contributors, developers ideally work in a branch and submit PRs as required. So commit size or frequency is irrelevant. It's the PR process that impacts a project.

Ok, I've said way too much. :) Thanks for the conversation! And all the best with your project. Cheers!

[rickstaa]

@rickstaa

I work alone on multiple projects. And small units of work make sense for my workflow. Not everyone is an OSS warrior. ;)

On a commercial or open source project, with several contributors, developers ideally work in a branch and submit PRs as required. So commit size or frequency is irrelevant. It's the PR process that impacts a project.

Ok, I've said way too much. :) Thanks for the conversation! And all the best with your project. Cheers!

No, you exactly hit the nail on the head. For personal projects, it doesn't really matter and in commercial projects or big OS projects, mostly PR with squashed commits are used 👍🏻. It's not that I have anything against people that add many commits to the master branch. It is only that me and the other collaborators/contributors are trying to promote PR-based contributions so that people will have less friction onboarding to bigger projects.

Having that said, this is a community project, so all input is valued equally. Furthermore, it's easy to increase the weights for the commits on your own fork 👍🏻. I think these ranks should be used as a fun motivational tool, not as a single source of truth. If you get more motivated to strengthen your coding skills and OS contributions by having the rank rise yearly by the number of commits, that's also great ❤️‍🔥.

github-readme-stats/src/calculateRank.js

Lines 27 to 28 in 1a2a82c

	const COMMITS_MEAN = all_commits ? 1000 : 250,
	COMMITS_WEIGHT = 2;

[rickstaa]

Closing as this feature was completed in #2762. You are welcome to open a new issue or pull request if you guys have more ideas to improve the ranking. One idea, for example, I think would be great to have is the following:

Maybe we can give the user the ability to set the rank_type to number or letter? We could use the inverse of the number that is used to calculate the rank 🤔.

Originally posted by @rickstaa in #2265 (comment)

[rickstaa]

Additionally, I now created #2859, which gives users the ability to show the percentile:

[![Anurag's GitHub stats](https://github-readme-stats-git-addpercentilerankicon-rickstaa.vercel.app/api?username=mindlabordev&rank_icon=percentile)](https://github.com/anuraghazra/github-readme-stats)

[rickstaa]

@silverwind, we decided to replace progress with percentile since it is unclear to users what progress means (see #2859 (review)). Therefore, changing progress to percentile in your README would be best.