Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Exclude the private mirrors from the mirrors list in the case of fallback behavior #560

Closed
soksanichenko opened this issue Jun 24, 2022 · 3 comments
Closed

Exclude the private mirrors from the mirrors list in the case of fallback behavior #560

soksanichenko opened this issue Jun 24, 2022 · 3 comments
Assignees
@soksanichenko

Comments

@soksanichenko
Copy link
Member

Change https://github.com/AlmaLinux/mirrors/blob/mirrors_service/src/backend/api/handlers.py#L187-L190 to make the call with without_private_mirrors=True
@soksanichenko soksanichenko self-assigned this Jun 24, 2022
@PhirePhly
Copy link
Contributor

To expand on the context here, I noted that when a client doesn't match on the network service cone (subnets and/or ASNs) of any mirrors, it falls back to rely on geo data to find near-by mirrors.

The get_mirrors_list()→ _get_nearest_mirrors(without_private_mirrors=False)→_get_nearest_mirrors_by_geo_data() call chain then fails to account for private mirrors only servicing their specified network cones.

The failure modes here are:

  1. A client which fails geo match lookup receives a full list of mirrors, including private mirrors.
  2. A private mirror accidentally incudes geo data in their mirror.d configuration file, and starts matching as an option for other near-by clients outside their network service cone.

soksanichenko pushed a commit that referenced this issue Jul 1, 2022
Issues #562,#561,#560:
dfb92ae 
- No source of entropy in _get_nearest_mirrors_by_network_data()
- _get_nearest_mirrors_by_network_data() fails to exclude near-by private mirrors for extra options.
- Exclude the private mirrors from the mirrors list in the case of fallback behavior
soksanichenko pushed a commit that referenced this issue Jul 1, 2022
Issues #562,#561,#560:
1f9f85f 
- No source of entropy in _get_nearest_mirrors_by_network_data()
- _get_nearest_mirrors_by_network_data() fails to exclude near-by private mirrors for extra options.
- Exclude the private mirrors from the mirrors list in the case of fallback behavior

The Azure mirrors have allowed list of arches
soksanichenko pushed a commit that referenced this issue Jul 1, 2022
Issues #562,#561,#560:
b715adb 
- No source of entropy in _get_nearest_mirrors_by_network_data()
- _get_nearest_mirrors_by_network_data() fails to exclude near-by private mirrors for extra options.
- Exclude the private mirrors from the mirrors list in the case of fallback behavior

The Azure mirrors have allowed list of arches
soksanichenko pushed a commit that referenced this issue Jul 1, 2022
Issues #562,#561,#560:
c56893c 
- No source of entropy in _get_nearest_mirrors_by_network_data()
- _get_nearest_mirrors_by_network_data() fails to exclude near-by private mirrors for extra options.
- Exclude the private mirrors from the mirrors list in the case of fallback behavior

- The Azure mirrors have allowed list of arches
- Decrease level of logging messages in some cases
- Cache subnets of Azure/AWS cloud
soksanichenko pushed a commit that referenced this issue Jul 1, 2022
Issues #562,#561,#560:
3a17b0f 
- No source of entropy in _get_nearest_mirrors_by_network_data()
- _get_nearest_mirrors_by_network_data() fails to exclude near-by private mirrors for extra options.
- Exclude the private mirrors from the mirrors list in the case of fallback behavior

- The Azure mirrors have allowed list of arches
- Decrease level of logging messages in some cases
- Cache subnets of Azure/AWS cloud
soksanichenko pushed a commit that referenced this issue Jul 1, 2022
Issues #562,#561,#560:
032b008 
- No source of entropy in _get_nearest_mirrors_by_network_data()
- _get_nearest_mirrors_by_network_data() fails to exclude near-by private mirrors for extra options.
- Exclude the private mirrors from the mirrors list in the case of fallback behavior

- The Azure mirrors have allowed list of arches
- Decrease level of logging messages in some cases
- Cache subnets of Azure/AWS cloud
soksanichenko pushed a commit that referenced this issue Jul 1, 2022
Issues #562,#561,#560:
b813bd5 
- No source of entropy in _get_nearest_mirrors_by_network_data()
- _get_nearest_mirrors_by_network_data() fails to exclude near-by private mirrors for extra options.
- Exclude the private mirrors from the mirrors list in the case of fallback behavior

- The Azure mirrors have allowed list of arches
- Decrease level of logging messages in some cases
- Cache subnets of Azure/AWS cloud
soksanichenko pushed a commit that referenced this issue Jul 1, 2022
Issues #562,#561,#560:
1471030 
- No source of entropy in _get_nearest_mirrors_by_network_data()
- _get_nearest_mirrors_by_network_data() fails to exclude near-by private mirrors for extra options.
- Exclude the private mirrors from the mirrors list in the case of fallback behavior

- The Azure mirrors have allowed list of arches
- Decrease level of logging messages in some cases
- Cache subnets of Azure/AWS cloud
@soksanichenko
Copy link
Member Author

soksanichenko commented Jul 4, 2022 •
edited
Loading

Result from dev machine

[ec2-user@ip-172-31-90-25 ~]$ curl -H "X-Forwarded-For: 127.0.0.1" http://localhost/mirrorlist/8/appstream 2>&1 | grep "centos.corp.cloudlinux.com"
[ec2-user@ip-172-31-90-25 ~]$

For 127.0.0.1 the service should return full list and centos.corp.cloudlinux.com is private mirror

soksanichenko pushed a commit that referenced this issue Jul 4, 2022
Issues #562,#561,#560:
6802d4b 
- No source of entropy in _get_nearest_mirrors_by_network_data()
- _get_nearest_mirrors_by_network_data() fails to exclude near-by private mirrors for extra options.
- Exclude the private mirrors from the mirrors list in the case of fallback behavior

- The Azure mirrors have allowed list of arches
- Decrease level of logging messages in some cases
- Cache subnets of Azure/AWS cloud
soksanichenko pushed a commit that referenced this issue Jul 7, 2022
Merge pull request #570 from AlmaLinux/issues_562,561,560
9e59f9b 
Issues #562,#561,#560:
soksanichenko pushed a commit that referenced this issue Jul 7, 2022
Merge pull request #571 from AlmaLinux/issues_562,561,560_yaml_snippets
83aaa3a 
Issues #562,#561,#560:
@soksanichenko
Copy link
Member Author

The patch is deployed to production

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@soksanichenko soksanichenko

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@PhirePhly @soksanichenko