Skip to content
This repository was archived by the owner on Apr 30, 2019. It is now read-only.

Support an API token for rate limit release #74

Closed
vvakame opened this issue Mar 11, 2014 · 13 comments
Closed

Support an API token for rate limit release #74

vvakame opened this issue Mar 11, 2014 · 13 comments

Comments

@vvakame
Copy link
Member

vvakame commented Mar 11, 2014

Please support an API token for rate limit release.
https://github.com/DefinitelyTyped/tsd#github-rate-limit
This problem happened in the office that there is me.

with grunt-tsd

Running "tsd:client" (tsd) task
-> config: ./tsd.json
error!
Error: unexpected status code: 403 on: https://api.github.com/repos/borisyankov/DefinitelyTyped/branches/master
    at Request.<anonymous> (/Users/vvakame/work/server/node_modules/grunt-tsd/node_modules/tsd/build/api.js:4442:34)
    at Request.EventEmitter.emit (events.js:95:17)
    at Request.onResponse (/Users/vvakame/work/server/node_modules/grunt-tsd/node_modules/tsd/node_modules/request/request.js:827:10)
    at ClientRequest.g (events.js:180:16)
    at ClientRequest.EventEmitter.emit (events.js:95:17)
    at HTTPParser.parserOnIncomingClient [as onIncoming] (http.js:1688:21)
    at HTTPParser.parserOnHeadersComplete [as onHeadersComplete] (http.js:121:23)
    at CleartextStream.socketOnData [as ondata] (http.js:1583:20)
    at CleartextStream.read [as read] (tls.js:511:12)
    at CleartextStream.Readable.read (_stream_readable.js:320:10)
    at EncryptedStream.write [as _write] (tls.js:366:25)
    at doWrite (_stream_writable.js:226:10)
    at writeOrBuffer (_stream_writable.js:216:5)
    at EncryptedStream.Writable.write (_stream_writable.js:183:11)
    at write (_stream_readable.js:583:24)
    at flow (_stream_readable.js:592:7)
    at Socket.pipeOnReadable (_stream_readable.js:624:5)
    at Socket.EventEmitter.emit (events.js:92:17)    at emitReadable (_stream_readable.js:408:10)
    at emitReadable (_stream_readable.js:404:5)
    at readableAddChunk (_stream_readable.js:165:9)
    at Socket.Readable.push (_stream_readable.js:127:10)
    at TCP.onread (net.js:528:21)
Warning: Task "tsd:client" failed. Use --force to continue.

Check for a response from github.

$ curl  -i https://api.github.com/users/whatever
HTTP/1.1 403 Forbidden
Server: GitHub.com  
Date: Tue, 11 Mar 2014 04:57:12 GMT
Content-Type: application/json; charset=utf-8
Status: 403 Forbidden
X-RateLimit-Limit: 60
X-RateLimit-Remaining: 0
X-RateLimit-Reset: 1394517224
X-GitHub-Media-Type: github.beta
X-Content-Type-Options: nosniff
Content-Length: 258 
Access-Control-Allow-Credentials: true
Access-Control-Expose-Headers: ETag, Link, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval
Access-Control-Allow-Origin: *
X-GitHub-Request-Id: DEE5E3DF:7B0C:A8C572:531E97A7

{
  "message": "API rate limit exceeded for 222.229.227.223. (But here's the good news: Authenticated requests get a higher rate limit. Check out the documentation for more details.)",
  "documentation_url": "http://developer.github.com/v3/#rate-limiting"
}

and my tsd.json

$ cat tsd.json
{
    "version": "v4",
    "repo": "borisyankov/DefinitelyTyped",
    "ref": "master",
    "path": "src/main/typescript/typings",
    "bundle": "src/main/typescript/typings/bundle.d.ts",
    "installed": {
        "jquery/jquery.d.ts": {
            "commit": "0854df3b0f7187ae445453aaad2563a97b70df08"
        },
        "angularjs/angular.d.ts": {
            "commit": "0854df3b0f7187ae445453aaad2563a97b70df08"
        },
        "angularjs/angular-mocks.d.ts": {
            "commit": "0854df3b0f7187ae445453aaad2563a97b70df08"
        },
        "angularjs/angular-route.d.ts": {
            "commit": "0854df3b0f7187ae445453aaad2563a97b70df08"
        },
        "mocha/mocha.d.ts": {
            "commit": "0854df3b0f7187ae445453aaad2563a97b70df08"
        },
        "chai/chai.d.ts": {
            "commit": "0854df3b0f7187ae445453aaad2563a97b70df08"
        }
    }
}

my Gruntfile.js

/// Gruntfile.json
                tsd: {
                        client: {
                                options: {
                                        // execute a command
                                        command: 'reinstall',

                                        //optional: always get from HEAD
                                        latest: false,

                                        // optional: specify config file
                                        config: './tsd.json'
                                }
                        }
                },

homebrew's solution
http://stackoverflow.com/questions/20130681/setting-github-api-token-for-homebrew

It may be another problem, but it is willing cache is not enabled?
not exists 0854df3b0f7187ae445453aaad2563a97b70df08

$ find ~/.tsd-cache
/Users/vvakame/.tsd-cache
/Users/vvakame/.tsd-cache/borisyankov-DefinitelyTyped
/Users/vvakame/.tsd-cache/borisyankov-DefinitelyTyped/git-api-v3.0.0-fmt1.0
/Users/vvakame/.tsd-cache/borisyankov-DefinitelyTyped/git-api-v3.0.0-fmt1.0/51
/Users/vvakame/.tsd-cache/borisyankov-DefinitelyTyped/git-api-v3.0.0-fmt1.0/83
/Users/vvakame/.tsd-cache/borisyankov-DefinitelyTyped/git-api-v3.0.0-fmt1.0/83/83b1e137418a79e2e25c07eac78a6ffa1f3727f2.json
/Users/vvakame/.tsd-cache/borisyankov-DefinitelyTyped/git-api-v3.0.0-fmt1.0/83/83b1e137418a79e2e25c07eac78a6ffa1f3727f2.raw
/Users/vvakame/.tsd-cache/borisyankov-DefinitelyTyped/git-api-v3.0.0-fmt1.0/_info.json
/Users/vvakame/.tsd-cache/borisyankov-DefinitelyTyped/git-api-v3.0.0-fmt1.0/b5
/Users/vvakame/.tsd-cache/borisyankov-DefinitelyTyped/git-raw-fmt1.0
/Users/vvakame/.tsd-cache/borisyankov-DefinitelyTyped/git-raw-fmt1.0/19
/Users/vvakame/.tsd-cache/borisyankov-DefinitelyTyped/git-raw-fmt1.0/19/1971aa9c880b2211f1fdc822f83a5a4395f3f23a.json
/Users/vvakame/.tsd-cache/borisyankov-DefinitelyTyped/git-raw-fmt1.0/19/1971aa9c880b2211f1fdc822f83a5a4395f3f23a.raw
/Users/vvakame/.tsd-cache/borisyankov-DefinitelyTyped/git-raw-fmt1.0/_info.json
/Users/vvakame/.tsd-cache/borisyankov-DefinitelyTyped/git-raw-fmt1.0/be
/Users/vvakame/.tsd-cache/borisyankov-DefinitelyTyped/git-raw-fmt1.0/be/be6cc5afc78efdc17f57a5aa3b1caac2e9e435b4.json
/Users/vvakame/.tsd-cache/borisyankov-DefinitelyTyped/git-raw-fmt1.0/be/be6cc5afc78efdc17f57a5aa3b1caac2e9e435b4.raw
/Users/vvakame/.tsd-cache/borisyankov-DefinitelyTyped/git-raw-fmt1.0/cb
/Users/vvakame/.tsd-cache/borisyankov-DefinitelyTyped/git-raw-fmt1.0/cb/cb06387a0110f30d73b5f464bb3c1b9c148f8763.json
/Users/vvakame/.tsd-cache/borisyankov-DefinitelyTyped/git-raw-fmt1.0/cb/cb06387a0110f30d73b5f464bb3c1b9c148f8763.raw
/Users/vvakame/.tsd-cache/borisyankov-DefinitelyTyped/git-raw-fmt1.0/d5
/Users/vvakame/.tsd-cache/borisyankov-DefinitelyTyped/git-raw-fmt1.0/d5/d5a3a92bfabd1164b0f5b9c924a45f4085d102ab.json
/Users/vvakame/.tsd-cache/borisyankov-DefinitelyTyped/git-raw-fmt1.0/d5/d5a3a92bfabd1164b0f5b9c924a45f4085d102ab.raw
/Users/vvakame/.tsd-cache/borisyankov-DefinitelyTyped/git-raw-fmt1.0/d8
/Users/vvakame/.tsd-cache/borisyankov-DefinitelyTyped/git-raw-fmt1.0/d8/d86bd51b08cd92414fc1cc7b28a9355d1f293c6d.json
/Users/vvakame/.tsd-cache/borisyankov-DefinitelyTyped/git-raw-fmt1.0/d8/d86bd51b08cd92414fc1cc7b28a9355d1f293c6d.raw
/Users/vvakame/.tsd-cache/borisyankov-DefinitelyTyped/git-raw-fmt1.0/e5
/Users/vvakame/.tsd-cache/borisyankov-DefinitelyTyped/git-raw-fmt1.0/e5/e55ce68613cf6993929335d66a61d0b6e94d177d.json
/Users/vvakame/.tsd-cache/borisyankov-DefinitelyTyped/git-raw-fmt1.0/e5/e55ce68613cf6993929335d66a61d0b6e94d177d.raw
@Bartvds Bartvds mentioned this issue Mar 11, 2014
Open
@Bartvds
Copy link
Collaborator

Bartvds commented Mar 11, 2014

Yea, this was bound to happen one day. I'm looking into adding an OAuth token.

We have a .tsdrc support (for #71) I could easily add a field to save a token.

For this I need to find the correct OAuth scope that only boosts the public rate limit without giving access, because the .tsdrc is not secure (I have question pending with Github support for this).

Also note: there was also a issue with the cache not keeping as long as expected, this is fixed for 0.5.7, (which also supports standard http proxy).

@Bartvds
Copy link
Collaborator

Bartvds commented Mar 11, 2014

Update: Github has the limited OAuth scope I needed and I have a basic test working here. Need a little to make it nice but is definitely in next version.

@Bartvds
Copy link
Collaborator

Bartvds commented Mar 12, 2014

Preview landed: https://github.com/DefinitelyTyped/tsd/releases/tag/0.5.7-alpha

See the readme for info. Does this work for you?

@vvakame
Copy link
Member Author

vvakame commented Mar 17, 2014

@Bartvds Great! 👍

$ tsd rate

>> tsd 0.5.7-alpha

 > rate-limit 4989 / 5000 -> 0:44:26

It seems to works fine!

@Bartvds
Copy link
Collaborator

Bartvds commented Mar 17, 2014

Good to hear. I'll try to push 0.5.7 to npm soonish.

Side note: I'm almost though with converting all the code to modules (import/require). (in a branch so 0.5.7 is still old style)

@vvakame
Copy link
Member Author

vvakame commented Mar 17, 2014

@Bartvds good idea.
tsd work only on node.js.

@vvakame
Copy link
Member Author

vvakame commented Mar 28, 2014

I also got rate limit again 😖
I'm waiting for tsd 0.5.7! (and grunt-tsd)

@Bartvds
Copy link
Collaborator

Bartvds commented Mar 28, 2014

Sorry, my bad:

I was making 0.5.7 ready earlier but got side tracked on a node 0.11 bug (I needed node 0.11 here for other things).

I try to make it next day, it is time.

@Bartvds
Copy link
Collaborator

Bartvds commented Mar 28, 2014

@vvakame I just published 0.5.7 to npm

@vvakame
Copy link
Member Author

vvakame commented Mar 29, 2014

😸 that's great!

@vvakame vvakame closed this as completed Mar 29, 2014
@minrk
Copy link

minrk commented May 25, 2015

For build environments like Travis, it would be great if the token could be retrieved from an environment variable instead of needing to be in a file. If GitHub finds a token in plaintext committed to a repo, the token will be revoked.

@timoweiss
Copy link

👍 For environment variable. Almost every third build crashes due to 403.

@jkremser jkremser mentioned this issue Jul 21, 2015
Merged
@Strate
Copy link

Strate commented Nov 9, 2015

You can ever use standard GITHUB_TOKEN variable, which used by official client https://github.com/github/hub

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@vvakame @minrk @Strate @Bartvds @timoweiss