Skip to content

feat(http): add URL replacement feature for HTTP requests#6207

Merged
jdx merged 12 commits intojdx:mainfrom
ThomasSteinbach:main
Sep 6, 2025
Merged

feat(http): add URL replacement feature for HTTP requests#6207
jdx merged 12 commits intojdx:mainfrom
ThomasSteinbach:main

Conversation

@ThomasSteinbach
Copy link
Contributor

  • Support string and regex-based URL replacements via url_replacements setting
  • Enable protocol, hostname, and path modifications for HTTP(S) downloads
  • Configure replacements in mise.toml or globally in settings

What problem does this feature solve?

  • mise has no inbuilt registry it downloads artifacts from.
  • Instead it reads remote registry manifests.
  • Those manifests contain URLs with where to download tools from.
  • In some environments (enterprise, DMZs, ...) those URLs are not directly accessible but mostly by a proxy.
  • mise currently has no configuration for a general URL redirection to that proxy.
  • This is exactly what this feature provides.
  • It is a mechanism where you can tell mise to replace/modify any URL request as desired.

Solves #5660


Full Documentation

... as from the update mise docs:

url_replacements

  • Type: object (optional)
  • Env: MISE_URL_REPLACEMENTS
  • Default: None

Map of URL patterns to replacement URLs. This feature supports both simple hostname replacements
and advanced regex-based URL transformations for download mirroring and custom registries.

Configuration Examples

Environment variable (JSON format):

# Simple hostname replacement
export MISE_URL_REPLACEMENTS='
{
  "github.com": "myregistry.net", 
  "releases.hashicorp.com": "mirror.example.com"
}'

# Regex pattern (note the escaped backslashes in JSON)
export MISE_URL_REPLACEMENTS='
{
  "regex:https://github\.com/([^/]+)/([^/]+)/releases/download/(.+)":
  "https://mirror.corp.com/github/$1/$2/$3"
}'

In mise.toml:

[settings]
# Simple hostname replacement
url_replacements = { 
  "github.com" = "myregistry.net", 
  "releases.hashicorp.com" = "mirror.example.com" 
}

# Regex patterns
url_replacements = { 
  "regex:^http://(.+)" = "https://$1",
  "regex:https://github\\.com/([^/]+)/([^/]+)/releases/download/(.+)" = 
    "https://mirror.corp.com/github/$1/$2/$3"
}

Simple Hostname Replacement

For simple hostname-based mirroring, the key is the original hostname/domain to replace,
and the value is the replacement string. The replacement happens by searching and replacing
the pattern anywhere in the full URL string (including protocol, hostname, path, and query parameters).

Examples:

  • github.com -> myregistry.net replaces GitHub hostnames
  • https://github.com -> https://myregistry.net with protocol excludes e.g. 'api.github.com'
  • https://github.com -> https://proxy.corp.com/github-mirror replaces GitHub with corporate proxy
  • http://host.net -> https://host.net replaces protocol from HTTP to HTTPS

Advanced Regex Replacement

For more complex URL transformations, you can use regex patterns. When a key starts with regex:,
it is treated as a regular expression pattern that can match and transform any part of the URL.
The value can use capture groups from the regex pattern.

Regex Examples

1. Protocol Conversion (HTTP to HTTPS)

[settings]
url_replacements = { 
  "regex:^http://(.+)" = "https://$1" 
}

This converts any HTTP URL to HTTPS by capturing everything after "http://" and replacing it with "https://".

2. GitHub Release Mirroring with Path Restructuring

[settings]
url_replacements = { 
  "regex:https://github\\.com/([^/]+)/([^/]+)/releases/download/(.+)" = 
    "https://mirror.corp.com/github/$1/$2/$3" 
}

Transforms https://github.com/owner/repo/releases/download/v1.0.0/file.tar.gz
to https://mirror.corp.com/github/owner/repo/v1.0.0/file.tar.gz

3. Subdomain to Path Conversion

[settings]
url_replacements = { 
  "regex:https://([^.]+)\\.cdn\\.example\\.com/(.+)" = 
    "https://unified-cdn.com/$1/$2" 
}

Converts subdomain-based URLs to path-based URLs on a unified CDN.

4. Multiple Replacement Patterns (processed in order)

[settings]
url_replacements = { 
  "regex:https://github\\.com/microsoft/(.+)" = 
    "https://internal-mirror.com/microsoft/$1",
  "regex:https://github\\.com/(.+)" = 
    "https://public-mirror.com/github/$1",
  "releases.hashicorp.com" = "hashicorp-mirror.internal.com"
}

First regex catches Microsoft repositories specifically, second catches all other GitHub URLs,
and the simple replacement handles HashiCorp.

Use Cases

  1. Corporate Mirrors: Replace public download URLs with internal corporate mirrors
  2. Custom Registries: Redirect package downloads to custom or private registries
  3. Geographic Optimization: Route downloads to geographically closer mirrors
  4. Protocol Changes: Convert HTTP URLs to HTTPS or vice versa

Regex Syntax

mise uses Rust regex engine which supports:

  • ^ and $ for anchors (start/end of string)
  • (.+) for capture groups (use $1, $2, etc. in replacement)
  • [^/]+ for character classes (matches any character except /)
  • \\. for escaping special characters (note: double backslash required in TOML)
  • *, +, ? for quantifiers
  • | for alternation

You can check on regex101.com if your regex works (see example).
Full regex syntax documentation: https://docs.rs/regex/latest/regex/#syntax

Precedence and Matching

  • Regex patterns (keys starting with regex:) are processed first, in the order they appear
  • Simple hostname replacements are processed second
  • The first matching pattern is used; subsequent patterns are ignored for that URL
  • If no patterns match, the original URL is used unchanged

Security Considerations

When using regex patterns, ensure your replacement URLs point to trusted sources,
as this feature can redirect tool downloads to arbitrary locations.


Kudos to this project <3

ThomasSteinbach and others added 4 commits September 6, 2025 15:00
- Support string and regex-based URL replacements via url_replacements setting
- Enable protocol, hostname, and path modifications for HTTP downloads
- Configure replacements in mise.toml or globally in settings
@jdx
Copy link
Owner

jdx commented Sep 6, 2025

bugbot run

cursor[bot]

This comment was marked as outdated.

@ThomasSteinbach
Copy link
Contributor Author

bugbot run

@cursor
Copy link

cursor bot commented Sep 6, 2025

Skipping Bugbot: Unable to authenticate your request. Please make sure Bugbot is properly installed and configured for this repository.

@jdx
Copy link
Owner

jdx commented Sep 6, 2025

bugbot run

Copy link

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no bugs!


@jdx jdx merged commit 49c7002 into jdx:main Sep 6, 2025
19 checks passed
netbsd-srcmastr pushed a commit to NetBSD/pkgsrc that referenced this pull request Sep 8, 2025
## [2025.9.5](https://github.com/jdx/mise/compare/v2025.9.4..v2025.9.5) - 2025-09-06

### 🚀 Features

- **(task)** add timeout support for task execution by @jdx in [#6216](jdx/mise#6216)
- **(task)** sub-tasks in run lists by @jdx in [#6212](jdx/mise#6212)

### Chore

- fix npm publish action by @jdx in [14f4b09](jdx/mise@14f4b09)
- fix cloudflare release action by @jdx in [00afa25](jdx/mise@00afa25)
- fix git-cliff for release notes by @jdx in [15a9aed](jdx/mise@15a9aed)

## [2025.9.4](https://github.com/jdx/mise/compare/v2025.9.3..v2025.9.4) - 2025-09-06

### Chore

- fix git-cliff on release by @jdx in [3c388f2](jdx/mise@3c388f2)

## [2025.9.3](https://github.com/jdx/mise/compare/v2025.9.2..v2025.9.3) - 2025-09-06

### 🚀 Features

- **(backend)** improve http error when platform url missing; list available platforms by @jdx in [#6200](jdx/mise#6200)
- **(cli)** support scoped packages for all backend types by @earlgray283 in [#6213](jdx/mise#6213)
- **(http)** add URL replacement feature for HTTP requests by @ThomasSteinbach in [#6207](jdx/mise#6207)

### 🐛 Bug Fixes

- **(backend)** preserve arch underscores in platform keys by @jdx in [#6202](jdx/mise#6202)
- **(task)** resolve hanging issue with multiple depends_post by @jdx in [#6206](jdx/mise#6206)
- couldn't download node binary in Alpine, even if it exists in the mirror url by @Hazer in [#5972](jdx/mise#5972)
- **breaking** use config_root for env._.path by @jdx in [#6204](jdx/mise#6204)
- bugfix for paths that include spaces by @karim-elkholy in [#6210](jdx/mise#6210)

### 📚 Documentation

- improve release notes generation by @jdx in [#6197](jdx/mise#6197)
- fix release changelog contributor reporting by @jdx in [#6201](jdx/mise#6201)

### Chore

- use fine-grained gh token by @jdx in [#6208](jdx/mise#6208)
- use settings.local.json for claude config by @jdx in [fd0fba9](jdx/mise@fd0fba9)

### New Contributors

- @ThomasSteinbach made their first contribution in [#6207](jdx/mise#6207)
- @earlgray283 made their first contribution in [#6213](jdx/mise#6213)
- @karim-elkholy made their first contribution in [#6210](jdx/mise#6210)
- @Hazer made their first contribution in [#5972](jdx/mise#5972)

## [2025.9.2](https://github.com/jdx/mise/compare/v2025.9.1..v2025.9.2) - 2025-09-05

### 🐛 Bug Fixes

- **(ci)** set required environment variables for npm publishing by @jdx in [#6189](jdx/mise#6189)
- **(release)** clean up extra newlines in release notes formatting by @jdx in [#6190](jdx/mise#6190)
- **(release)** add proper newline after New Contributors section in cliff template by @jdx in [#6194](jdx/mise#6194)
- **(release)** fix changelog formatting to remove extra blank lines by @jdx in [#6195](jdx/mise#6195)
- **(release)** restore proper newline after New Contributors section by @jdx in [#6196](jdx/mise#6196)

### 🚜 Refactor

- **(ci)** split release workflow into separate specialized workflows by @jdx in [#6193](jdx/mise#6193)

### Chore

- **(release)** require GitHub Actions environment for release-plz script by @jdx in [#6191](jdx/mise#6191)

## [2025.9.1](https://github.com/jdx/mise/compare/v2025.9.0..v2025.9.1) - 2025-09-05

### 🐛 Bug Fixes

- python nested venv path order by @elvismacak in [#6124](jdx/mise#6124)
- resolve immutable release workflow and VERSION file timing issues by @jdx in [#6187](jdx/mise#6187)

### New Contributors

- @elvismacak made their first contribution in [#6124](jdx/mise#6124)

## [2025.9.0](https://github.com/jdx/mise/compare/v2025.8.21..v2025.9.0) - 2025-09-05

### 🚀 Features

- allow set/unset backend aliases by @roele in [#6172](jdx/mise#6172)

### 🐛 Bug Fixes

- **(aqua)** respect order of asset_strs by @risu729 in [#6143](jdx/mise#6143)
- **(java)** treat freebsd as linux (assuming linux compatability) by @roele in [#6161](jdx/mise#6161)
- **(nushell/windows)** Fix $env.PATH getting converted to a string by @zackyancey in [#6157](jdx/mise#6157)
- **(sync)** create uv_versions_path dir if it doesn't exist by @risu729 in [#6142](jdx/mise#6142)
- **(ubi)** show relevent error messages for v-prefixed tags by @risu729 in [#6183](jdx/mise#6183)
- remove nodejs/golang alias migrate code by @risu729 in [#6141](jdx/mise#6141)
- mise activate not working on powershell v5 by @L0RD-ZER0 in [#6168](jdx/mise#6168)

### 📚 Documentation

- **(task)** remove word "additional" to avoid confusions by @risu729 in [#6159](jdx/mise#6159)

### Chore

- update Cargo.lock by @risu729 in [#6184](jdx/mise#6184)

### New Contributors

- @zackyancey made their first contribution in [#6157](jdx/mise#6157)
jdx pushed a commit that referenced this pull request Sep 11, 2025
…6269)

This is a documentation fix of my recent contribution
#6207

I recognized that the examples in the documentation were no valid toml
syntax. I fixed and tested the examples. This PR contains those updated
examples.
@puskunalis
Copy link

This is a great enhancement and very useful, thank you @ThomasSteinbach!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants