Twitter badges failing Twitter API closed down?

[svrooij]

Are you experiencing an issue with...

shields.io

🐞 Description

As of today I noticed that several of my twitter badges are failing (yours on the home pages as well!)

Twitter closed down its api for 3rd party apps very recently, so I suspect this has something to do with it.

🔗 Link to the badge

HTTPS://shields.io and scroll all the way down.

💡 Possible Solution

Suggestion is to remove all twitter things, since there probably won’t be any progression on the API part.

Or replace with: for now and as fallback when json not working.

https://img.shields.io/badge/follow-%40{username}-1DA1F2?logo=twitter&style={style}

[calebcartwright]

Thanks for reaching out

Twitter closed down its api for 3rd party apps very recently, so I suspect this has something to do with it.

Do you have any additional info on this?

I had to dig back pretty deep into the project history to find the motivation, but turns out we are using a private/unsupported endpoint (http://cdn.syndication.twimg.com/widgets/followbutton/info.json?) to avoid the rate limit and auth requirements that are often deal breakers for the scale and model at which Shields.io operates

[regginator]

Twitter closed down its api for 3rd party apps very recently, so I suspect this has something to do with it.

Do you have any additional info on this?

Yeah, Twitter made an official announcement on this yesterday; they're completely closing down their developer API for all 3rd party clients, sadly.

Here's an article published by The Verge with more information.

turns out we are using a private/unsupported endpoint (http://cdn.syndication.twimg.com/widgets/followbutton/info.json?)

The service that's providing that endpoint in use may just have had its' Twitter-Developer-API access revoked due to this recent change.

[abstractalgo]

Just wanted to confirm that, indeed, Twitter badges are not working, and also found that fetch that was mentioned:

shields/services/twitter/twitter.service.js

Lines 102 to 103 in ac5fdbb

	url: 'http://cdn.syndication.twimg.com/widgets/followbutton/info.json',
	options: { searchParams: { screen_names: user } },

Even after a bit more digging around Internet, I haven't been able to find a different endpoint than that one that can retrieve this kind of data without any auth. 😕 Unfortunate.

[ghost]

• https://img.shields.io/twitter/follow/twitter?style=social
  • 

[Theta-Dev]

I guess the only way to obtain follower counts is the official Twitter API. And they have a fairly strict rate limit of max. 300 user lookups per 15 minutes.

https://developer.twitter.com/en/docs/twitter-api/rate-limits

I guess that shields.io is getting a lot more traffic, even if the like counts would be cached for 24h or so.

Removing the Twitter badge entirely would break existing sites, so my suggestion would be to remove the like counter and have a static badge (similar to the official one).

https://publish.twitter.com/?buttonType=FollowButton&query=%40TwitterDev&widget=Button

[chris48s]

In the last 15 mins we've had over 3,000 requests on the twitter badge at the origin, (i.e: excluding those we served from downstream cache) and it is not exactly peak usage right now. We could probably cache for longer but I think we'd really struggle to stay under the rate limit for a single API key. Given the situation upstream, I'm not sure I have the inclination to invest in a token pooling solution.

I think given the twitter badge consistently ranks inside the top 10 highest traffic badges on the service, I'd agree that instead of following our usual process for deprecating/removing, we could usefully replace it with something like

as a fallback. It is somewhat unusual, but so is retiring one of the highest traffic badges on the service.

I'll try and get a PR in for this tomorrow.

[svrooij]

@chris48s you're right about that. Just plain removing one of the highest ranked badges might not be the best plan.

Maybe in a last ditch effort we should send Elon a message what we want and if he can facilitate?

[calebcartwright]

Maybe in a last ditch effort we should send Elon a message what we want and if he can facilitate?

😄

We could attempt to reach out to see if they'd be willing to support some kind of increased rate limit for us, but I'm rather skeptical anything would come out of it. There's obviously... quite a bit happening at Twitter these days and with their reported focus on financials I'd be surprised if they'd be willing to do so instead of pushing us to pay for some level of access (which we don't have the spare funds for)

[VictorioMolina]

same error

[rickstaa]

In the last 15 mins we've had over 3,000 requests on the twitter badge at the origin, (i.e: excluding those we served from downstream cache) and it is not exactly peak usage right now. We could probably cache for longer but I think we'd really struggle to stay under the rate limit for a single API key. Given the situation upstream, I'm not sure I have the inclination to invest in a token pooling solution.

I think given the Twitter badge consistently ranks inside the top 10 highest traffic badges on the service, I'd agree that instead of following our usual process for deprecating/removing, we could usefully replace it with something like

as a fallback. It is somewhat unusual, but so is retiring one of the highest traffic badges on the service.

I'll try and get a PR in for this tomorrow.

I think asking Twitter to increase the limit for shields.io is the right way. I, however, am surprised if they grant you that request.😅 We asked the same from github-readme-stats for the GraphQL API, but in the end, we have to rely on donated PAT to keep the Public Vercel instance from being Rate Limited. We loop through these PATs and move to the next one when one is rate limited in the hour (see https://github.com/anuraghazra/github-readme-stats/blob/4b173001264e9774a2179915519e6129af689625/src/common/retryer.js#L18-L61). Not the most pretty solution but it works. 👍

[chris48s]

Just one extra note here: This endpoint has previously broken then come back a couple of days later e.g: #7698 Given the timing, I expect this is unlikely but it is also possible it could come back online on the other side of the weekend.

[NEMSLinux]

In the last 15 mins we've had over 3,000 requests on the twitter badge at the origin, (i.e: excluding those we served from downstream cache) and it is not exactly peak usage right now. We could probably cache for longer but I think we'd really struggle to stay under the rate limit for a single API key. Given the situation upstream, I'm not sure I have the inclination to invest in a token pooling solution.

I think given the twitter badge consistently ranks inside the top 10 highest traffic badges on the service, I'd agree that instead of following our usual process for deprecating/removing, we could usefully replace it with something like

as a fallback. It is somewhat unusual, but so is retiring one of the highest traffic badges on the service.

I'll try and get a PR in for this tomorrow.

This would be significantly better than the error message. Perhaps this could be a failover output in event that the endpoint / JSON response are bad? I'd be happy with this. +1

[svrooij]

Maybe respond with

https://img.shields.io/badge/follow-%40{username}-1DA1F2?logo=twitter&style={style}

For now, ao all the badges at least show something better then json could nog be parsed.

And then maybe find a way to fix the badge after that.

[eghysut]

is work thanks you

[![Twitter Follow](https://img.shields.io/badge/follow-%40RexosP-1DA1F2?logo=twitter&style=social)](https://twitter.com/RexosP)

[rickstaa]

Maybe respond with

https://img.shields.io/badge/follow-%40{username}-1DA1F2?logo=twitter&style={style}

For now, ao all the badges at least show something better then json could nog be parsed.

And then maybe find a way to fix the badge after that.

Incorrect bot action, or was the badge changed to the suggestion above🤔?

[svrooij]

Maybe respond with

https://img.shields.io/badge/follow-%40{username}-1DA1F2?logo=twitter&style={style}

For now, ao all the badges at least show something better then json could nog be parsed.

And then maybe find a way to fix the badge after that.

Incorrect bot action, or was the badge changed to the suggestion above🤔?

They used my suggestion in a temp fix. Interesting discussion on wether or not to redirect the twitter badge. #8842