Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: way to avoid github API limit (add tests + docs) #123

Merged
merged 10 commits into from
Dec 28, 2018
Merged
Show file tree
Hide file tree
Changes from 9 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 12 additions & 1 deletion .all-contributorsrc
Original file line number Diff line number Diff line change
Expand Up @@ -232,7 +232,7 @@
"avatar_url": "https://avatars2.githubusercontent.com/u/8073251?v=4",
"profile": "https://github.com/xuchaoying",
"contributions": [
"code",
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was causing all-contributors to crash hard with a useless error message Cannot read property 'then' of null I'm going to propose we improve the parsing of the config file, to alert on issues like this

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"code"
]
},
{
Expand All @@ -243,6 +243,17 @@
"contributions": [
"doc"
]
},
{
"login": "jakebolam",
"name": "Jake Bolam",
"avatar_url": "https://avatars2.githubusercontent.com/u/3534236?v=4",
"profile": "https://jakebolam.com",
"contributions": [
"code",
"doc",
"test"
]
}
]
}
15 changes: 9 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
[![version][version-badge]][package] [![downloads][downloads-badge]][downloads]
[![MIT License][license-badge]][license]

[![All Contributors](https://img.shields.io/badge/all_contributors-24-orange.svg?style=flat-square)](#contributors)
[![All Contributors](https://img.shields.io/badge/all_contributors-26-orange.svg?style=flat-square)](#contributors)
[![PRs Welcome][prs-badge]][prs] [![Code of Conduct][coc-badge]][coc]
[![Watch on GitHub][github-watch-badge]][github-watch]
[![Star on GitHub][github-star-badge]][github-star]
Expand Down Expand Up @@ -140,7 +140,7 @@ Where `username` is the user's GitHub or Gitlab username, and `contribution` is
- tutorial: [✅](# 'Tutorials')
- video: [📹](# 'Videos')

Please note that if you are using a self-hosted gitlab instance, before adding
Please note that if you are using a self-hosted gitlab instance, before adding
contributor, you need to set an environment variable named `PRIVATE_TOKEN` first.

> Private token is the personal access token to authenticate with the GitLab API.
Expand All @@ -163,7 +163,7 @@ all-contributors check
```

> Due to GitHub API restrictions, this command only works for projects with less
> than 500 contributors.
> than 500 contributors. (Unless you set a PRIVATE_TOKEN) as mentioned below

## Configuration

Expand Down Expand Up @@ -210,6 +210,10 @@ These are the keys you can specify:
}
```

In some cases you may see the error message 'GitHub API rate limit exceeded for xxx'. You may need to set an environment variable named `PRIVATE_TOKEN` in order to circumvent this rate-limiting.

> Private token is your personal access token to authenticate with the GitHub API.

## Contributors

Thanks goes to these wonderful people
Expand All @@ -221,9 +225,8 @@ Thanks goes to these wonderful people
| :---: | :---: | :---: | :---: | :---: | :---: |
| [<img src="https://avatars3.githubusercontent.com/u/8212?v=3" width="100px;"/><br /><sub><b>Jerod Santo</b></sub>](https://jerodsanto.net)<br />[💻](https://github.com/jfmengels/all-contributors-cli/commits?author=jerodsanto "Code") | [<img src="https://avatars1.githubusercontent.com/u/574871?v=3" width="100px;"/><br /><sub><b>Kevin Jalbert</b></sub>](https://github.com/kevinjalbert)<br />[💻](https://github.com/jfmengels/all-contributors-cli/commits?author=kevinjalbert "Code") | [<img src="https://avatars3.githubusercontent.com/u/5038030?v=4" width="100px;"/><br /><sub><b>tunnckoCore</b></sub>](https://i.am.charlike.online)<br />[🔧](#tool-charlike "Tools") | [<img src="https://avatars2.githubusercontent.com/u/304450?v=4" width="100px;"/><br /><sub><b>Mehdi Achour</b></sub>](https://machour.idk.tn/)<br />[💻](https://github.com/jfmengels/all-contributors-cli/commits?author=machour "Code") | [<img src="https://avatars1.githubusercontent.com/u/8344688?v=4" width="100px;"/><br /><sub><b>Roy Revelt</b></sub>](https://codsen.com)<br />[🐛](https://github.com/jfmengels/all-contributors-cli/issues?q=author%3Arevelt "Bug reports") | [<img src="https://avatars1.githubusercontent.com/u/422331?v=4" width="100px;"/><br /><sub><b>Chris Vickery</b></sub>](https://github.com/chrisinajar)<br />[💻](https://github.com/jfmengels/all-contributors-cli/commits?author=chrisinajar "Code") |
| [<img src="https://avatars2.githubusercontent.com/u/1026002?v=4" width="100px;"/><br /><sub><b>Bryce Reynolds</b></sub>](https://github.com/brycereynolds)<br />[💻](https://github.com/jfmengels/all-contributors-cli/commits?author=brycereynolds "Code") | [<img src="https://avatars3.githubusercontent.com/u/2322305?v=4" width="100px;"/><br /><sub><b>James, please</b></sub>](http://www.jmeas.com)<br />[🤔](#ideas-jmeas "Ideas, Planning, & Feedback") [💻](https://github.com/jfmengels/all-contributors-cli/commits?author=jmeas "Code") | [<img src="https://avatars3.githubusercontent.com/u/1057324?v=4" width="100px;"/><br /><sub><b>Spyros Ioakeimidis</b></sub>](http://www.spyros.io)<br />[💻](https://github.com/jfmengels/all-contributors-cli/commits?author=spirosikmd "Code") | [<img src="https://avatars3.githubusercontent.com/u/12335761?v=4" width="100px;"/><br /><sub><b>Fernando Costa</b></sub>](https://github.com/fadc80)<br />[💻](https://github.com/jfmengels/all-contributors-cli/commits?author=fadc80 "Code") | [<img src="https://avatars0.githubusercontent.com/u/197404?v=4" width="100px;"/><br /><sub><b>snipe</b></sub>](https://snipe.net)<br />[📖](https://github.com/jfmengels/all-contributors-cli/commits?author=snipe "Documentation") | [<img src="https://avatars0.githubusercontent.com/u/997157?v=4" width="100px;"/><br /><sub><b>Gant Laborde</b></sub>](http://gantlaborde.com/)<br />[💻](https://github.com/jfmengels/all-contributors-cli/commits?author=GantMan "Code") |
| [<img src="https://avatars2.githubusercontent.com/u/17708702?v=4" width="100px;"/><br /><sub><b>Md Zubair Ahmed</b></sub>](https://in.linkedin.com/in/mzubairahmed)<br />[📖](https://github.com/jfmengels/all-contributors-cli/commits?author=M-ZubairAhmed "Documentation") [🐛](https://github.com/jfmengels/all-contributors-cli/issues?q=author%3AM-ZubairAhmed "Bug reports") [💻](https://github.com/jfmengels/all-contributors-cli/commits?author=M-ZubairAhmed "Code") [⚠️](https://github.com/jfmengels/all-contributors-cli/commits?author=M-ZubairAhmed "Tests") | [<img src="https://avatars3.githubusercontent.com/u/6177621?v=4" width="100px;"/><br /><sub><b>Divjot Singh</b></sub>](http://bogas04.github.io)<br />[📖](https://github.com/jfmengels/all-contributors-cli/commits?author=bogas04 "Documentation") | [<img src="https://avatars0.githubusercontent.com/u/15315098?v=4" width="100px;"/><br /><sub><b>João Marques</b></sub>](https://github.com/tigermarques)<br />[💻](https://github.com/jfmengels/all-contributors-cli/commits?author=tigermarques "Code") [📖](https://github.com/jfmengels/all-contributors-cli/commits?author=tigermarques "Documentation") [🤔](#ideas-tigermarques "Ideas, Planning, & Feedback") | [<img src="https://avatars3.githubusercontent.com/u/1192452?v=4" width="100px;"/><br /><sub><b>Andrew Lisowski</b></sub>](http://hipstersmoothie.com)<br />[💻](https://github.com/jfmengels/all-contributors-cli/commits?author=hipstersmoothie "Code") [📖](https://github.com/jfmengels/all-contributors-cli/commits?author=hipstersmoothie "Documentation") [⚠️](https://github.com/jfmengels/all-contributors-cli/commits?author=hipstersmoothie "Tests") | [<img src="https://avatars3.githubusercontent.com/u/1736154?v=4" width="100px;"/><br /><sub><b>Xianming Zhong</b></sub>](https://github.com/chinesedfan)<br />[📖](https://github.com/jfmengels/all-contributors-cli/commits?author=chinesedfan "Documentation") | [<img src="https://avatars3.githubusercontent.com/u/3680914?v=4" width="100px;"/><br /><sub><b>Dura</b></sub>](https://github.com/chris-dura)<br />[📖](https://github.com/jfmengels/all-contributors-cli/commits?author=chris-dura "Documentation") | | [<img src="https://avatars2.githubusercontent.com/u/8073251?v=4" width="100px;"/><br /><sub><b>C.Y.Xu</b></sub>](https://github.com/xuchaoying)<br />[💻](https://github.com/jfmengels/all-contributors-cli/commits?author=xuchaoying "Code") [📖](https://github.com/jfmengels/all-contributors-cli/commits?author=xuchaoying "Documentation") |
| [<img src="https://avatars2.githubusercontent.com/u/8073251?v=4" width="100px;"/><br /><sub><b>C.Y.Xu</b></sub>](https://github.com/xuchaoying)<br />[💻](https://github.com/jfmengels/all-contributors-cli/commits?author=xuchaoying "Code") [📖](https://github.com/jfmengels/all-contributors-cli/commits?author=xuchaoying "Documentation") |

| [<img src="https://avatars2.githubusercontent.com/u/17708702?v=4" width="100px;"/><br /><sub><b>Md Zubair Ahmed</b></sub>](https://in.linkedin.com/in/mzubairahmed)<br />[📖](https://github.com/jfmengels/all-contributors-cli/commits?author=M-ZubairAhmed "Documentation") [🐛](https://github.com/jfmengels/all-contributors-cli/issues?q=author%3AM-ZubairAhmed "Bug reports") [💻](https://github.com/jfmengels/all-contributors-cli/commits?author=M-ZubairAhmed "Code") [⚠️](https://github.com/jfmengels/all-contributors-cli/commits?author=M-ZubairAhmed "Tests") | [<img src="https://avatars3.githubusercontent.com/u/6177621?v=4" width="100px;"/><br /><sub><b>Divjot Singh</b></sub>](http://bogas04.github.io)<br />[📖](https://github.com/jfmengels/all-contributors-cli/commits?author=bogas04 "Documentation") | [<img src="https://avatars0.githubusercontent.com/u/15315098?v=4" width="100px;"/><br /><sub><b>João Marques</b></sub>](https://github.com/tigermarques)<br />[💻](https://github.com/jfmengels/all-contributors-cli/commits?author=tigermarques "Code") [📖](https://github.com/jfmengels/all-contributors-cli/commits?author=tigermarques "Documentation") [🤔](#ideas-tigermarques "Ideas, Planning, & Feedback") | [<img src="https://avatars3.githubusercontent.com/u/1192452?v=4" width="100px;"/><br /><sub><b>Andrew Lisowski</b></sub>](http://hipstersmoothie.com)<br />[💻](https://github.com/jfmengels/all-contributors-cli/commits?author=hipstersmoothie "Code") [📖](https://github.com/jfmengels/all-contributors-cli/commits?author=hipstersmoothie "Documentation") [⚠️](https://github.com/jfmengels/all-contributors-cli/commits?author=hipstersmoothie "Tests") | [<img src="https://avatars3.githubusercontent.com/u/1736154?v=4" width="100px;"/><br /><sub><b>Xianming Zhong</b></sub>](https://github.com/chinesedfan)<br />[📖](https://github.com/jfmengels/all-contributors-cli/commits?author=chinesedfan "Documentation") | [<img src="https://avatars2.githubusercontent.com/u/8073251?v=4" width="100px;"/><br /><sub><b>C.Y.Xu</b></sub>](https://github.com/xuchaoying)<br />[💻](https://github.com/jfmengels/all-contributors-cli/commits?author=xuchaoying "Code") |
| [<img src="https://avatars3.githubusercontent.com/u/3680914?v=4" width="100px;"/><br /><sub><b>Dura</b></sub>](https://github.com/chris-dura)<br />[📖](https://github.com/jfmengels/all-contributors-cli/commits?author=chris-dura "Documentation") | [<img src="https://avatars2.githubusercontent.com/u/3534236?v=4" width="100px;"/><br /><sub><b>Jake Bolam</b></sub>](https://jakebolam.com)<br />[💻](https://github.com/jfmengels/all-contributors-cli/commits?author=jakebolam "Code") [📖](https://github.com/jfmengels/all-contributors-cli/commits?author=jakebolam "Documentation") [⚠️](https://github.com/jfmengels/all-contributors-cli/commits?author=jakebolam "Tests") |
<!-- ALL-CONTRIBUTORS-LIST:END -->

This project follows the
Expand Down
34 changes: 34 additions & 0 deletions src/repo/__tests__/github.js
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,40 @@ test('fill in the name when null is returned', async () => {
expect(info.name).toBe('nodisplayname')
})

test('attaches token when supplied', async () => {
const mockAuthToken = 'myMock-token-adaskjda'
nock('https://api.github.com')
.matchHeader('authorization', `token ${mockAuthToken}`)
.get('/users/test-token')
.reply(200, {
html_url: 'test-token',
})

await(getUserInfo('test-token', 'https://github.com', mockAuthToken))
})

test('attaches no token when supplied empty', async () => {
nock('https://api.github.com')
.matchHeader('authorization', '')
.get('/users/test-token')
.reply(200, {
html_url: 'test-token',
})

await(getUserInfo('test-token', 'https://github.com', ''))
})

test('attaches no token when not supplied', async () => {
nock('https://api.github.com')
.matchHeader('authorization', '')
.get('/users/test-token')
.reply(200, {
html_url: 'test-token',
})

await(getUserInfo('test-token'))
})

test('fill in the name when an empty string is returned', async () => {
nock('https://api.github.com')
.get('/users/nodisplayname')
Expand Down
21 changes: 10 additions & 11 deletions src/repo/github.js
Original file line number Diff line number Diff line change
@@ -1,14 +1,13 @@
const pify = require('pify')
const request = pify(require('request'))

function getRequestHeaders() {
function getRequestHeaders(optionalPrivateToken = '') {
const requestHeaders = {
'User-Agent': 'request',
}

const optionalAuthToken = process.env.GITHUB_TOKEN
if (optionalAuthToken) {
requestHeaders.Authorization = `token ${optionalAuthToken}`

if (optionalPrivateToken && optionalPrivateToken.length > 0) {
requestHeaders.Authorization = `token ${optionalPrivateToken}`
}

return requestHeaders
Expand All @@ -28,11 +27,11 @@ function getNextLink(link) {
return nextLink.split(';')[0].slice(1, -1)
}

function getContributorsPage(url) {
function getContributorsPage(url, optionalPrivateToken) {
return request
.get({
url,
headers: getRequestHeaders(),
headers: getRequestHeaders(optionalPrivateToken),
})
.then(res => {
const body = JSON.parse(res.body)
Expand All @@ -55,7 +54,7 @@ function getContributorsPage(url) {
})
}

const getUserInfo = function(username, hostname) {
const getUserInfo = function(username, hostname, optionalPrivateToken) {
/* eslint-disable complexity */
if (!hostname) {
hostname = 'https://github.com'
Expand All @@ -65,7 +64,7 @@ const getUserInfo = function(username, hostname) {
return request
.get({
url: `${root}/users/${username}`,
headers: getRequestHeaders(),
headers: getRequestHeaders(optionalPrivateToken),
})
.then(res => {
const body = JSON.parse(res.body)
Expand All @@ -87,14 +86,14 @@ const getUserInfo = function(username, hostname) {
})
}

const getContributors = function(owner, name, hostname) {
const getContributors = function(owner, name, hostname, optionalPrivateToken) {
if (!hostname) {
hostname = 'https://github.com'
}

const root = hostname.replace(/:\/\//, '://api.')
const url = `${root}/repos/${owner}/${name}/contributors?per_page=100`
return getContributorsPage(url)
return getContributorsPage(url, optionalPrivateToken)
}

module.exports = {
Expand Down