Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support mirror CDN for package and shared profile downloads #960

Merged
merged 1 commit into from
Dec 19, 2023

Conversation

anttimaki
Copy link
Collaborator

API calls can indicate that mirror CDN is preferred by including a query paramater flag in the request. Currently this is supported by the API endpoints that provide download URLs for packages and shared mod manager profiles.

If the backend receives the query parameter flag and is configured to use MirroredS3Storage, it will try to return URLs that point to the first mirror CDN.

This is intended to solve the small but growing number of incidents where users for some reason can't access Thunderstore's main CDN.

Refs TS-2003

Copy link

codecov bot commented Dec 14, 2023

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (7ba4403) 92.93% compared to head (6e1659b) 92.94%.

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #960      +/-   ##
==========================================
+ Coverage   92.93%   92.94%   +0.01%     
==========================================
  Files         286      286              
  Lines        8488     8502      +14     
  Branches      788      790       +2     
==========================================
+ Hits         7888     7902      +14     
  Misses        496      496              
  Partials      104      104              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@MythicManiac
Copy link
Member

These shouldn't be tied to the mirror storage definitions at all, as those might have nothing to do with public-facing CDN routing.

If we want to validate the domains (which is not a bad idea), add a new setting for it and go through that list instead.

@anttimaki
Copy link
Collaborator Author

These shouldn't be tied to the mirror storage definitions at all, as those might have nothing to do with public-facing CDN routing.

If we want to validate the domains (which is not a bad idea), add a new setting for it and go through that list instead.

Reimplemented as:

  1. preferred CDN domain is received as query parameter
  2. Config includes allowlist for legit CDNs
  3. If preferred CDN is allowed, replace the domain and pray the CDN actually has the file at the same path

Copy link
Member

@MythicManiac MythicManiac left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Few questions but looks good overall 👍

Feel free to merge as is or adjust if you think it makes sense to

django/thunderstore/core/tests/test_utils.py Outdated Show resolved Hide resolved
.env.template Outdated Show resolved Hide resolved
API calls can indicate that a mirror CDN is preferred by including a
query parameter in the request. The feature will be supported by the
API endpoints that provide download URLs for packages and shared mod
manager profiles.

The backend will return URL to the preferred CDN only if the CDN is
allowlisted in the config. This is a plain string replacement operation
and the backend doesn't validate the URL it's about to return in any
way.

This is intended to solve the small but growing number of incidents
where users for some reason can't access Thunderstore's main CDN.

Refs TS-2003
@anttimaki anttimaki merged commit e5abf29 into master Dec 19, 2023
25 checks passed
@anttimaki anttimaki deleted the api-mirror-cdn branch December 19, 2023 10:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants