Top Languages Card not working properly #136

fluorspar20 · 2020-07-21T17:07:41Z

Describe the bug
I have quite a few repos with the top language being javascript. However, the top languages card doesn't show javascript at all in the list.

Expected behavior
It should show js as one of the top languages.

Screenshots / Live demo link

rjoydip-zz · 2020-07-21T17:26:41Z

Same with my profile.

github-readme-stats/top-langs not showing all languages as well as percentages.

But github-profile-languages is more appropriate.

filiptronicek · 2020-07-21T17:38:49Z

Sure, but I think the GitHub GraphQl API just takes the raw LOC count from all your repos and does the analysis from that. Not 100% sure tho

filiptronicek · 2020-07-21T17:43:27Z

Ok, I was mistaken, it should just fetch the top language of the repo. @anuraghazra WDYT about this?

github-readme-stats/src/fetchTopLanguages.js

Lines 5 to 25 in dc3e9a5

    
           const fetcher = (variables, token) => { 
        
             return request( 
        
               { 
        
                 query: ` 
        
                 query userInfo($login: String!) { 
        
                   user(login: $login) { 
        
                     repositories(isFork: false, first: 100) { 
        
                       nodes { 
        
                         languages(first: 1) { 
        
                           edges { 
        
                             size 
        
                             node { 
        
                               color 
        
                               name 
        
                             } 
        
                           } 
        
                         } 
        
                       } 
        
                     } 
        
                   } 
        
                 }

anuraghazra · 2020-07-21T17:52:28Z

Same with my profile.

github-readme-stats/top-langs not showing all languages as well as percentages.

But github-profile-languages is more appropriate.

Working on my side, which browser you are using?

live demo

rjoydip-zz · 2020-07-21T17:54:23Z

Working on my side, which browser you are using?

Chrome. But below code is showing rust & go lang as well.

query {
  user(login: "rjoydip") {
    repositories(isFork: false, first: 100) {
      nodes {
        languages(first: 1) {
          edges {
            size
            node {
              color
              name
            }
          }
        }
      }
    }
  }
}

anuraghazra · 2020-07-21T17:56:14Z

Ok, I was mistaken, it should just fetch the top language of the repo. @anuraghazra WDYT about this?

Yup it should fetch the correct langs.

filiptronicek · 2020-07-21T17:59:25Z

Mine is also weird 😕 @anuraghazra

Actual (pie chart)	Our implementation

anuraghazra · 2020-07-21T18:01:36Z

NOTE: Consider the 100 max repos & also it get's the totalSize (in bytes) to calculate how many bytes you have written with the language.

filiptronicek · 2020-07-21T18:02:48Z

Is that how we want it to be? Is there not a better implementation?

NOTE: Consider the 100 max repos & also it get's the totalSize (in bytes) to calculate how many bytes you have written with the language.

anuraghazra · 2020-07-21T18:05:43Z

Is that how we want it to be? Is there not a better implementation?

NOTE: Consider the 100 max repos & also it get's the totalSize (in bytes) to calculate how many bytes you have written with the language.

That's how github calculates and it's all fetched from github's api so no way the data is wrong, maybe the data processing is wrong from my side. have to do some experiments.

anuraghazra · 2020-07-21T18:08:53Z

I'll look into this tomorrow.

anuraghazra · 2020-07-22T07:34:40Z

Working on my side, which browser you are using?

Chrome. But below code is showing rust & go lang as well.

query {
  user(login: "rjoydip") {
    repositories(isFork: false, first: 100) {
      nodes {
        languages(first: 1) {
          edges {
            size
            node {
              color
              name
            }
          }
        }
      }
    }
  }
}

Hi @rjoydip yes, but as you can see

              "edges": [
                {
                  "size": 196,
                  "node": {
                    "color": "#dea584",
                    "name": "Rust"
                  }
                }
              ]

There is only one rust lang in those 100 results, and the size is 196bytes it's i think this is why it's not showing

anuraghazra · 2020-07-22T07:37:00Z

Maybe if i change the gql query to fetch 5 langs from a certain repo then it would be better because for now i'm just only selecting one language from each repo.

user(login: "rjoydip") {
    repositories(isFork: false, first: 100) {
      nodes {
        languages(first: 5) {
          edges {
            size
            node {
              color
              name
            }
          }
        }
      }
    }
  }

rjoydip-zz · 2020-07-22T08:25:34Z

Maybe if i change the gql query to fetch 5 langs from a certain repo then it would be better because for now i'm just only selecting one language from each repo.

user(login: "rjoydip") {
    repositories(isFork: false, first: 100) {
      nodes {
        languages(first: 5) {
          edges {
            size
            node {
              color
              name
            }
          }
        }
      }
    }
  }

@anuraghazra Yes, I saw the same thing. It'll be better to make dynamic isFork and languages as variable.

anuraghazra · 2020-07-22T09:15:37Z

Not dynamic, making it max 5 or 10 would do the job, a repo can't have too much languages anyways.

and isFork should always be false, don't want to count forked repos. for example if anyone forked reactjs then they would have lot of js code

filiptronicek · 2020-07-22T16:02:31Z

I am of the opinion, that we should count forks too, it's something, that GitHub also does, and forks exist also because people have projects they make on their own.

I am not sure if GitHub provides this, I am not exactly an expert on their v4 API, but the extensions that provide the same solution must be querying it somehow, I'll look into that.

filiptronicek · 2020-07-22T16:08:32Z

Just a link for some info on another solution:
https://github.com/freyamade/github-user-languages

@anuraghazra

rjoydip-zz · 2020-07-22T18:38:54Z

FYI...

{
  user(login: "rjoydip") {
    repositories(isFork: false, first: 100, orderBy: {field: UPDATED_AT, direction: DESC}) {
      nodes {
        name
        updatedAt
        languages(first: 5, orderBy: {field: SIZE, direction: DESC}) {
          nodes {
            name
          }
        }
        primaryLanguage {
          name
        }
      }
    }
  }
}

filiptronicek · 2020-07-22T18:51:29Z

Useful, thanks, for this, I think it would be enough to not use the languages, just the primary one. / cc: @anuraghazra @rjoydip

{
  user(login: "rjoydip") {
    repositories(isFork: false, first: 100, orderBy: {field: UPDATED_AT, direction: DESC}) {
      nodes {
        primaryLanguage {
          name
        }
      }
    }
  }
}

That gives out something like this:

{
  "data": {
    "user": {
      "repositories": {
        "nodes": [
          {
            "primaryLanguage": {
              "name": "TypeScript"
            }
          },
          {
            "primaryLanguage": {
              "name": "TypeScript"
            }
          },
          {
            "primaryLanguage": {
              "name": "Java"
            }
          },
          {
            "primaryLanguage": null
          }
    ]
}

stemount · 2020-07-22T20:23:22Z

Useful, thanks, for this, I think it would be enough to not use the languages, just the primary one. / cc: @anuraghazra @rjoydip

{
  user(login: "rjoydip") {
    repositories(isFork: false, first: 100, orderBy: {field: UPDATED_AT, direction: DESC}) {
      nodes {
        primaryLanguage {
          name
        }
      }
    }
  }
}

That gives out something like this:

{
  "data": {
    "user": {
      "repositories": {
        "nodes": [
          {
            "primaryLanguage": {
              "name": "TypeScript"
            }
          },
          {
            "primaryLanguage": {
              "name": "TypeScript"
            }
          },
          {
            "primaryLanguage": {
              "name": "Java"
            }
          },
          {
            "primaryLanguage": null
          }
    ]
}

I think this is would be good for "most used language widget" as it is currently an approximation of many repos.

for example I could one repo that is just an express app just serving mostly HTML, but it would say 100% typescript.

filiptronicek · 2020-07-23T08:26:43Z

Useful, thanks, for this, I think it would be enough to not use the languages, just the primary one. / cc: @anuraghazra @rjoydip

I think this is would be good for "most used language widget" as it is currently an approximation of many repos.

for example I could one repo that is just an express app just serving mostly HTML, but it would say 100% typescript.

Does this mean, that HTML isn't considered a language in this analysis? I am a bit confused. Can you give me an example repo?

anuraghazra · 2020-07-23T08:39:47Z

I don't think we can effectively do language analysis, for example take a scenario if someone uploaded node_modules to their github then their javascript would be 100% no matter what. same as #153

filiptronicek · 2020-07-23T13:47:46Z

I don't think we can effectively do language analysis, for example take a scenario if someone uploaded node_modules to their github then their javascript would be 100% no matter what. same as #153

True, but nobody should ever do that (upload their node_modules), if they do, they cannot be then angry at our code, which considers Industry best practices and it also affects GitHub's own analysis.

anuraghazra · 2020-07-23T14:31:34Z

I don't think we can effectively do language analysis, for example take a scenario if someone uploaded node_modules to their github then their javascript would be 100% no matter what. same as #153

True, but nobody should ever do that (upload their node_modules), if they do, they cannot be then angry at our code, which considers Industry best practices and it also affects GitHub's own analysis.

Yup, but that's not my point, there are lot of scenarios where we cannot evaluate code correctly and there is no perfect way to do that, lets just take an example of my website's github repo which has a develop branch and a master branch and master branch holds all the static html code which is generated by gatsby, there are huge amounts of meta data, JSON data & javascript files.

https://github.com/anuraghazra/anuraghazra.github.io/tree/master

tobiasvl · 2020-07-23T21:04:48Z

I also came here because I saw discrepancies between this tool and my Chrome extension-generated pie chart (linked above).

As an example, this tool says my top language is JavaScript with 46.21% and Lua is second with 29.29%, while the pie chart says I have 5 JavaScript repos and 15 Lua repos. However, if I do a search for JavaScript repos, I only get 1. Not sure what to make of that; presumably this tool counts LOC while the pie chart counts top language per repo, but perhaps the pie chart counts forks too, since it comes up with 5 and not 1?

By the way, not sure if it's relevant, but organization profile pages (like https://github.com/github) actually list the top 5 languages in the org's repos (without bars or percentages or anything fancy). It looks like those are just top languages, not LOC. I might be mistaken though.

anuraghazra · 2020-07-28T16:49:14Z

And i also like the suggestion of @stemount

I think this is would be good for "most used language widget" as it is currently an approximation of many repos.

I think the "Top Languages" labeling is misleading, it should be "Most used languages"

anuraghazra · 2020-07-28T16:55:33Z

@NikhilCodes i don't no whats wrong with your profile but i've checked with other user's stats with the fix i'm working on and they are all fine expect yours.

Btw i've checked the graphql request and seems like you do have a very very very large python repo, and this repo is so huge in bytes its straight up kicking your dart stats, so i think the stats are totally fine.

{
            "nameWithOwner": "NikhilCodes/VirtualBLU",
            "isFork": false,
            "languages": {
              "edges": [
                {
                  "size": 78537442,
                  "node": {
                    "color": "#3572A5",
                    "name": "Python"
                  }
                }
              ]
            }
          },

THE UPDATED GQL QUERY LOOKS LIKE THIS

user(login: "NikhilCodes") {
    repositories(ownerAffiliations: OWNER, isFork: false, first: 100) {
      nodes {
        nameWithOwner
        isFork
        languages(first: 10, orderBy: {field: SIZE, direction: DESC}) {
          edges {
            size
            node {
              color
              name
            }
          }
        }
      }
    }
  }

tobiasvl · 2020-07-28T17:06:50Z

@tobiasvl i was experimenting with the api & the code, Are you satisfied with these stats?

Much better! Thank you.

However, I believe there's something wrong with your "tie-break" algorithm, so to speak. The Chrome extension linked earlier lists these as my top languages:

Lua (15 repos)
Python (13 repos)
JavaScript (5 repos)
Assembly (5 repos)
HTML (5 repos)
CSS (5 repos)
Ruby (4 repos)
C (4 repos)

And then a bunch of languages, including Java, with 1 repo each.

As you'll notice, your new code lists Assembly as my third repo and then C below that. It seems that it's simply ignoring my HTML and CSS repos, because I have as many of them as I have assembly repos! That would also explain why Java is on the list at all even though I only have one Java repo – it's ignoring my 4 Ruby repos because I already have 4 C repos on the list.

So, to sum it up, I think your patched code is getting there, but if there are several languages with the same repo count, it only displays one of them and ignores the rest.

anuraghazra · 2020-07-28T17:11:26Z

@tobiasvl It is not about how many repos you have, you could have 100 Js repos with 10bytes of code and you can have 1 Python repo with 20000bytes of code, Python would be at top in this case.

So, to sum it up, I think your patched code is getting there, but if there are several languages with the same repo count, it only displays one of them and ignores the rest.

As you can see i changed the gql query to fetch 10 languages in every repo & i'm calculating all of them.

tobiasvl · 2020-07-28T17:13:11Z

Hmm, OK, thanks. If it's intentional then I'm definitely fine with this. I don't want HTML and CSS on my list anyway 😅

anuraghazra · 2020-07-28T17:14:51Z

Also @tobiasvl

I believe there's something wrong with your "tie-break" algorithm

Actually there is no special algorithm in play here, i'm just sorting/manipulating the data coming from Github's API and picking the Top languages which have the most size.

tobiasvl · 2020-07-28T17:21:01Z

I see, thanks. Ship it!

anuraghazra · 2020-07-28T17:27:13Z

Oh my godh.
I just realized this, why even everyone was comparing the stats with http://ionicabizau.github.io/github-profile-languages/ ?????

because i just checked their source code and they are calculating "How MANY languages a user has in their profile" & github-readme-stats is calculating "MOST used languages in user's profile"

And they have a package called gh-polygot which is doing this :-
https://github.com/IonicaBizau/node-gh-polyglot/blob/master/lib/index.js#L102-L106

VictorNS69 · 2020-08-03T11:45:19Z

Hi! First of all, thanks for the job you are doing with this repo!

I have a question, that I think fits here.

I have like 15 Python repositories, but I have no stats in "most used languages"
Here you can see the live card:

Also the static card:

Maybe Python is not my "Top language with the most size", but seems strange to not have any %.

anuraghazra · 2020-08-11T14:25:22Z

Maybe Python is not my "Top language with the most size", but seems strange to not have any %

@VictorNS69
You have so much code in java & ASP that python did not made it to the list.

tobiasvl · 2020-08-11T18:52:20Z

Yes, I think it's pertinent to point out that strangely, the percentages seem to be within the top 5. You don't have 1.69% TeX in all your repos, but it makes up 1.69% of all the code within your top 5 languages. (Unless I've misunderstood.)

anuraghazra · 2020-08-11T19:22:13Z

Yes, I think it's pertinent to point out that strangely, the percentages seem to be within the top 5. You don't have 1.69% TeX in all your repos, but it makes up 1.69% of all the code within your top 5 languages. (Unless I've misunderstood.)

He has one repo with a good amount of TeX https://github.com/VictorNS69/Apuntes-Ciber

abdullasirajudeen · 2020-09-21T05:29:49Z

I my Profile Not Show Top Language, All are Empty
Github Stat Link

anuraghazra · 2020-09-21T07:09:58Z

@abdullasirajudeen because you don't have anything, you have 5 repos 4 of them are forks which won't be counted and one is your readme repo which does not have any code.

tobiasvl · 2020-09-21T07:15:00Z

@anuraghazra I think you mean @abdullasirajudeen? (I have lots of repos)

abdullasirajudeen · 2020-09-21T07:29:26Z

@tobiasvl because you don't have anything, you have 5 repos 4 of them are forks which won't be counted and one is your readme repo which does not have any code.

private repository not counted here.

anuraghazra · 2020-09-21T07:48:01Z

@anuraghazra I think you mean @abdullasirajudeen? (I have lots of repos)

oh yeahh, sorry for the wrong mention.

apoorvpandey0 · 2021-05-22T14:05:43Z

Mine is also working incorrectly,
It's showing top language as JS but that's not the case I use Python

aelbenney · 2021-09-01T13:28:50Z

Is there any stats for the languages you have contributed in? (Company repositories)

kavinvalli · 2021-09-16T13:57:44Z

Mine is also working incorrectly,
It's showing top language as JS but that's not the case I use Python

Same here. I haven't used python in very long but it still shows up, and doesn't show JS and TS which i use mostly

ramonpaolo · 2022-01-06T16:42:24Z

same here

dancarlton · 2023-02-01T00:27:22Z

mine isn't showing any languages at all in my readme, but it is updated in one of my repos. am I sourcing it incorrectly?

fluorspar20 changed the title ~~Top Languages not working properly~~ Top Languages Card not working properly Jul 21, 2020

anuraghazra added the bug Something isn't working. label Jul 21, 2020

anuraghazra added the lang-card Issues related to the language card. label Jul 23, 2020

anuraghazra mentioned this issue Jul 24, 2020

Not showing dart in Top Languages card #167

Closed

anuraghazra added the help wanted Extra attention is needed. label Jul 24, 2020

anuraghazra mentioned this issue Jul 28, 2020

feat: improved top lang fetching & changed title #240

Merged

anuraghazra closed this as completed in #240 Jul 28, 2020

anuraghazra mentioned this issue Jul 29, 2020

fix: fetch top lang in each repository #204

Merged

anuraghazra mentioned this issue Aug 11, 2020

Top Langs incorrect #270

Closed

anuraghazra mentioned this issue Aug 13, 2020

Top language card not showing Python #358

Closed

anuraghazra mentioned this issue Sep 20, 2020

[BREAKING CHANGE] New Top Language Detection Method #481

Closed

kitswas mentioned this issue Feb 18, 2022

[Proposal] New Top Language Detection Algorithm #1600

Closed

pandasoli mentioned this issue Apr 24, 2023

Cached language stats #2638

Closed

Top Languages Card not working properly #136

Top Languages Card not working properly #136

Comments

fluorspar20 commented Jul 21, 2020 • edited Loading

rjoydip-zz commented Jul 21, 2020

filiptronicek commented Jul 21, 2020

filiptronicek commented Jul 21, 2020

anuraghazra commented Jul 21, 2020 • edited Loading

rjoydip-zz commented Jul 21, 2020 • edited Loading

anuraghazra commented Jul 21, 2020 • edited Loading

filiptronicek commented Jul 21, 2020

anuraghazra commented Jul 21, 2020

filiptronicek commented Jul 21, 2020

anuraghazra commented Jul 21, 2020

anuraghazra commented Jul 21, 2020

anuraghazra commented Jul 22, 2020

anuraghazra commented Jul 22, 2020

rjoydip-zz commented Jul 22, 2020

anuraghazra commented Jul 22, 2020

filiptronicek commented Jul 22, 2020

filiptronicek commented Jul 22, 2020

rjoydip-zz commented Jul 22, 2020

filiptronicek commented Jul 22, 2020

stemount commented Jul 22, 2020

filiptronicek commented Jul 23, 2020 • edited by anuraghazra Loading

anuraghazra commented Jul 23, 2020

filiptronicek commented Jul 23, 2020

anuraghazra commented Jul 23, 2020

tobiasvl commented Jul 23, 2020

anuraghazra commented Jul 28, 2020

anuraghazra commented Jul 28, 2020

tobiasvl commented Jul 28, 2020

anuraghazra commented Jul 28, 2020 • edited Loading

tobiasvl commented Jul 28, 2020

anuraghazra commented Jul 28, 2020

tobiasvl commented Jul 28, 2020

anuraghazra commented Jul 28, 2020 • edited Loading

VictorNS69 commented Aug 3, 2020

anuraghazra commented Aug 11, 2020 • edited Loading

tobiasvl commented Aug 11, 2020

anuraghazra commented Aug 11, 2020

abdullasirajudeen commented Sep 21, 2020 • edited Loading

anuraghazra commented Sep 21, 2020 • edited Loading

tobiasvl commented Sep 21, 2020

abdullasirajudeen commented Sep 21, 2020

anuraghazra commented Sep 21, 2020

apoorvpandey0 commented May 22, 2021

aelbenney commented Sep 1, 2021

kavinvalli commented Sep 16, 2021

ramonpaolo commented Jan 6, 2022

dancarlton commented Feb 1, 2023

fluorspar20 commented Jul 21, 2020 •

edited

Loading

anuraghazra commented Jul 21, 2020 •

edited

Loading

rjoydip-zz commented Jul 21, 2020 •

edited

Loading

anuraghazra commented Jul 21, 2020 •

edited

Loading

filiptronicek commented Jul 23, 2020 •

edited by anuraghazra

Loading

anuraghazra commented Jul 28, 2020 •

edited

Loading

anuraghazra commented Jul 28, 2020 •

edited

Loading

anuraghazra commented Aug 11, 2020 •

edited

Loading

abdullasirajudeen commented Sep 21, 2020 •

edited

Loading

anuraghazra commented Sep 21, 2020 •

edited

Loading