Skip to content

Test suite: switch to httpbingo.julialang, fall back to httpbin.julialang on failure #60834

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
DilumAluthge merged 8 commits into from
Jan 28, 2026
Merged

Test suite: switch to httpbingo.julialang, fall back to httpbin.julialang on failure #60834

DilumAluthge merged 8 commits into from
Jan 28, 2026
+12 −3

Conversation

@rokke-git
Copy link
Contributor

@rokke-git rokke-git commented Jan 27, 2026

we've been failing all the prs for a few days now, is it worth adding a backup for the httpbin mirror?

giordano
giordano reviewed Jan 27, 2026
View reviewed changes
inkydragon
inkydragon reviewed Jan 27, 2026
View reviewed changes
Copy link
Member

@inkydragon inkydragon left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://github.com/postmanlabs/httpbin
Last commit: f8ec666 · 8 years ago
Maybe this project is dead?

https://github.com/mccutchen/go-httpbin
Last commit: 5cc6f1e · 4 days ago
Looks fine.

Or we want to add both site.
And https://httpbingo.julialang.org/

xref #base-ci-failure-help on Slack: https://julialang.slack.com/archives/C02C4RYM66N/p1769464953098339

@gbaraldi
Copy link
Member

gbaraldi commented Jan 27, 2026

Can we go httpbin -> httpbingo (ours) and then fail? I don't want to rely to push a bunch of load to a random user (even though it might be fine but still)

@rokke-git
Copy link
Contributor Author

rokke-git commented Jan 27, 2026

nice, didn't know we had another one. do we want to switch the order on those since the first one is still down? or is httpbingo.julialang.org expected to be down more often in the future?

@DilumAluthge
Copy link
Member

DilumAluthge commented Jan 27, 2026

I don't think the order matters too much, if both are our instances.

Can you print the exception and backtrace to the logs?

@DilumAluthge DilumAluthge changed the title fall back on httpbin when julia's mirror is down Test suite: Try httpbin.julialang, fall back to httpbingo.julialang Jan 27, 2026
@DilumAluthge DilumAluthge assigned DilumAluthge and unassigned vtjnash Jan 27, 2026
DilumAluthge
DilumAluthge reviewed Jan 27, 2026
View reviewed changes
@fredrikekre
Copy link
Member

fredrikekre commented Jan 27, 2026 •
edited
Loading

Let's just move to the https://httpbingo.julialang.org/ version instead of complicating things.

@DilumAluthge
Copy link
Member

DilumAluthge commented Jan 27, 2026

Let's just move to the https://httpbingo.julialang.org/ version instead of complicating things.

Seems like it would be better to have multiple options in the test suite, right? In case one instance goes down again in the future?

DilumAluthge
DilumAluthge reviewed Jan 28, 2026
View reviewed changes
@DilumAluthge
Copy link
Member

DilumAluthge commented Jan 28, 2026

Seems like it would be better to have multiple options in the test suite, right? In case one instance goes down again in the future?

E.g. if httpbingo.julialang starts having problems in the future, we can spin up something working at httpbin.julialang, and Base CI will use it automatically.

@DilumAluthge DilumAluthge added the merge me PR is reviewed. Merge when all tests are passing label Jan 28, 2026
@rokke-git rokke-git changed the title Test suite: Try httpbin.julialang, fall back to httpbingo.julialang Test suite: switch to httpbingo.julialang, fall back to httpbin.julialang on failure Jan 28, 2026
@DilumAluthge
Copy link
Member

DilumAluthge commented Jan 28, 2026

CI isn't done yet, but several jobs have finished and look good. Let's give this a try in an attempt to get master to be less red.

@DilumAluthge DilumAluthge merged commit 5ab8db3 into JuliaLang:master Jan 28, 2026
6 of 10 checks passed
@rokke-git
Copy link
Contributor Author

rokke-git commented Jan 28, 2026

shouldn't that commit message at least be the other way around? if it was a size issue than at least test: add fallback url would actually describe what the change does

@StefanKarpinski
Copy link
Member

StefanKarpinski commented Jan 28, 2026

I don't believe that these services have identical APIs. Similar but not exactly the same. Have you tested to see if all tests pass (especially Downloads) when using httpbin instead of httpbingo?

@DilumAluthge
Copy link
Member

DilumAluthge commented Jan 28, 2026

I don't believe that these services have identical APIs. Similar but not exactly the same. Have you tested to see if all tests pass (especially Downloads) when using httpbin instead of httpbingo?

Indeed, the Downloads.jl tests fail when using httpbin.org (example).

However, this PR only affects the server that is used in in test/download_exec.jl.

Downloads.jl has its own logic for choosing the test server: https://github.com/JuliaLang/Downloads.jl/blob/f72cd4d87b424fc48e1137c1fd1fbcec18f49371/test/setup.jl#L84-L85

Basically, Downloads.jl uses JULIA_TEST_HTTPBINGO_SERVER if it is defined, and if not falls back to httpbingo.julialang.org. So Downloads.jl won't use httpbin in its tests (even after this PR).

@DilumAluthge DilumAluthge removed the merge me PR is reviewed. Merge when all tests are passing label Jan 28, 2026
@DilumAluthge DilumAluthge mentioned this pull request Jan 28, 2026
Draft
@DilumAluthge DilumAluthge added the backport 1.10 Change should be backported to the 1.10 release label Jan 28, 2026
DilumAluthge pushed a commit that referenced this pull request Jan 28, 2026
@rokke-git @DilumAluthge
Merge pull request #60834 from rokke-git/patch-5
f550350 
Test suite: switch to `httpbingo.julialang`, fall back to `httpbin.julialang` on failure

(cherry picked from commit 5ab8db3)
@DilumAluthge DilumAluthge mentioned this pull request Jan 28, 2026
Open
71 tasks
@DilumAluthge DilumAluthge added backport 1.11 Change should be backported to release-1.11 backport 1.12 Change should be backported to release-1.12 backport 1.13 Change should be backported to release-1.13 labels Jan 28, 2026
DilumAluthge pushed a commit that referenced this pull request Jan 28, 2026
@rokke-git @DilumAluthge
Merge pull request #60834 from rokke-git/patch-5
e4cfa46 
Test suite: switch to `httpbingo.julialang`, fall back to `httpbin.julialang` on failure

(cherry picked from commit 5ab8db3)
@DilumAluthge DilumAluthge mentioned this pull request Jan 28, 2026
Merged
1 task
DilumAluthge added a commit that referenced this pull request Jan 29, 2026
@DilumAluthge
Merge pull request #60861 from JuliaLang/backports-release-1.11
b52e3ca 
Backport the `httpbingo` testsuite fix (#60834) to Julia 1.11.9
DilumAluthge pushed a commit that referenced this pull request Jan 29, 2026
@rokke-git @DilumAluthge
Merge pull request #60834 from rokke-git/patch-5
ec3ef8b 
Test suite: switch to `httpbingo.julialang`, fall back to `httpbin.julialang` on failure

(cherry picked from commit 5ab8db3)
KristofferC pushed a commit that referenced this pull request Feb 2, 2026
@rokke-git @KristofferC
Merge pull request #60834 from rokke-git/patch-5
4938a3e 
Test suite: switch to `httpbingo.julialang`, fall back to `httpbin.julialang` on failure

(cherry picked from commit 5ab8db3)
(cherry picked from commit e4cfa46)
KristofferC pushed a commit that referenced this pull request Feb 2, 2026
@rokke-git @KristofferC
Merge pull request #60834 from rokke-git/patch-5
a42e47b 
Test suite: switch to `httpbingo.julialang`, fall back to `httpbin.julialang` on failure

(cherry picked from commit 5ab8db3)
(cherry picked from commit e4cfa46)
@KristofferC
Copy link
Member

KristofferC commented Feb 2, 2026

Please squash merge PRs like this.

@DilumAluthge
Copy link
Member

DilumAluthge commented Feb 2, 2026

I think it was squashed, right? 5ab8db3 only shows one parent.

@DilumAluthge
Copy link
Member

DilumAluthge commented Feb 2, 2026

I think it was squashed, right? 5ab8db3 only shows one parent.

Oh, but the first line of the commit message looks like the message for a merge commit!

@KristofferC
Copy link
Member

KristofferC commented Feb 2, 2026

Ah, ok, I think I only looked at the commit message. Not sure what happened then. 🤷

@KristofferC KristofferC removed the backport 1.12 Change should be backported to release-1.12 label Feb 3, 2026
@KristofferC KristofferC mentioned this pull request Feb 4, 2026
Open
21 tasks
@KristofferC KristofferC removed the backport 1.13 Change should be backported to release-1.13 label Feb 4, 2026
@github-actions github-actions bot removed the backport 1.11 Change should be backported to release-1.11 label Feb 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@giordano giordano giordano left review comments

@inkydragon inkydragon inkydragon left review comments

@DilumAluthge DilumAluthge DilumAluthge approved these changes

Assignees

@DilumAluthge DilumAluthge

Labels

backport 1.10 Change should be backported to the 1.10 release

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

9 participants

@rokke-git @gbaraldi @DilumAluthge @fredrikekre @StefanKarpinski @KristofferC @giordano @inkydragon @vtjnash