build: use mirror.tensorflow.org, now populated
#2245
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Summary:
It’s easier for TensorBoard team members to upload files to the
TensorFlow.org mirror than the Bazel mirror. Many of our files were not
mirrored, and fetches from GitHub can be noticeably slow. (For instance,
the Bazel 0.22.0 binary is 1.8–2.0 seconds from the TF mirror and 8.5–12
seconds from GitHub.) As of this commit, all `mirror.bazel.build` URLs
are replaced with `mirror.tensorflow.org` URLs, and all such URLs
contain valid content.
Along the way, I swapped out the defunct numericjs source to a host that
actually still resolves.
On Travis, getting Bazel and running `bazel fetch //tensorboard/...`
typically takes about 44 seconds. Let’s see if this speeds that up.
Test Plan:
Check that Bazel URLs resolve:
```
$ git grep -Pho '(?<=")https?://mirror\.tensorflow\.org/[^"]*' |
> grep -vF '${BAZEL}' | sort | uniq |
> xargs -n 1 -P 32 -- \
> sh -c 'curl -sfL "$1" >/dev/null && echo OK || echo FAIL' unused |
> sort | uniq -c
231 OK
```
Check that the Travis URL resolves:
```
$ BAZEL=0.22.0
$ curl -sfL "http://mirror.tensorflow.org/github.com/bazelbuild/bazel/releases/download/${BAZEL}/bazel-${BAZEL}-linux-x86_64" |
> shasum -a 256
8474ed28ed4998e2f5671ddf3a9a80ae9e484a5de3b8b70c8b654c017c65d363
```
Check that we caught ’em all:
```
$ git grep mirror.bazel | wc -l
0
```
wchargin-branch: mirror-tensorflow-org
|
Well, fetch down from 44s to 40s on Travis. Sure, okay. |
stephanwlee
reviewed
May 20, 2019
| @@ -74,7 +74,7 @@ | |||
| 'Chrome/62.0.3202.94 ' | |||
| 'Safari/537.36', | |||
| 'HTTP User-Agent header to send to Google Fonts') | |||
| flags.DEFINE_string('mirror', 'https://mirror.bazel.build/', | |||
| flags.DEFINE_string('mirror', 'https://mirror.tensorflow.org/', | |||
There was a problem hiding this comment.
Should this url be http instead of https?
stephanwlee
approved these changes
May 20, 2019
wchargin
commented
May 20, 2019
There was a problem hiding this comment.
Am I correct in guessing that
httpis okay since we have the
checksum?
Yes, exactly. A mitm could eavesdrop or deny service, but can never
corrupt the build because everything has known checksums.
I would have used HTTPS anyway, but the TensorFlow mirror does not
support it; it has a bad certificate.
| @@ -74,7 +74,7 @@ | |||
| 'Chrome/62.0.3202.94 ' | |||
| 'Safari/537.36', | |||
| 'HTTP User-Agent header to send to Google Fonts') | |||
| flags.DEFINE_string('mirror', 'https://mirror.bazel.build/', | |||
| flags.DEFINE_string('mirror', 'https://mirror.tensorflow.org/', | |||
nfelt
approved these changes
May 20, 2019
wchargin-source: 7a5d9d7e62962de5645333721c7dd0ec7a3cb485 wchargin-branch: mirror-tensorflow-org
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary:
It’s easier for TensorBoard team members to upload files to the
TensorFlow.org mirror than the Bazel mirror. Many of our files were not
mirrored, and fetches from GitHub can be noticeably slow. (For instance,
the Bazel 0.22.0 binary is 1.8–2.0 seconds from the TF mirror and 8.5–12
seconds from GitHub.) As of this commit, all
mirror.bazel.buildURLsare replaced with
mirror.tensorflow.orgURLs, and all such URLscontain valid content.
Along the way, I swapped out the defunct numericjs source to a host that
actually still resolves.
On Travis, getting Bazel and running
bazel fetch //tensorboard/...typically takes about 44 seconds. Let’s see if this speeds that up.
Test Plan:
Check that Bazel URLs resolve:
Check that the Travis URL resolves:
Check that we caught ’em all:
wchargin-branch: mirror-tensorflow-org