Skip to content

Commit 1cc39c1

Browse files
jprochazkemilk
andcommitted
Image uploader script (#2164)
<!-- Open the PR up as a draft until you feel it is ready for a proper review. Do not make PR:s from your own `main` branch, as that makes it difficult for reviewers to add their own fixes. Add any improvements to the branch as new commits to make it easier for reviewers to follow the progress. All commits will be squashed to a single commit once the PR is merged into `main`. Make sure you mention any issues that this PR closes in the description, as well as any other related issues. To get an auto-generated PR description you can put "copilot:summary" or "copilot:walkthrough" anywhere. --> Closes #2132 ### What Adds `just upload <image>`, which uploads the image to google cloud using `gsutil`. The resulting URL contains the content hash plus the original file name, which ensures that it is unique. ``` $ just upload image.png https://static.rerun.io/dd31922030a6bf7223d5e3728d8da5407f4d6b1a_image.png ``` The script itself contains instructions for how to setup `gsutil`. It's really just a link to Google's tutorial. Eventually it will be documented separately together with the rest of the new workflow. ### Checklist * [x] I have read and agree to [Contributor Guide](https://github.com/rerun-io/rerun/blob/main/CONTRIBUTING.md) and the [Code of Conduct](https://github.com/rerun-io/rerun/blob/main/CODE_OF_CONDUCT.md) * [ ] I've included a screenshot or gif (if applicable) <!-- This line will get updated when the PR build summary job finishes. --> PR Build Summary: https://build.rerun.io/pr/2164 --------- Co-authored-by: Emil Ernerfeldt <[email protected]>
1 parent e2ee66c commit 1cc39c1

File tree

3 files changed

+62
-2
lines changed

3 files changed

+62
-2
lines changed

justfile

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -27,8 +27,7 @@ py-dev-env:
2727
set -euxo pipefail
2828
python3 -m venv venv
2929
venv/bin/pip install --upgrade pip
30-
venv/bin/pip install -r rerun_py/requirements-build.txt
31-
venv/bin/pip install -r rerun_py/requirements-lint.txt
30+
venv/bin/pip install -r scripts/requirements-dev.txt
3231
echo "Do 'source venv/bin/activate' to use the virtual environment!"
3332

3433
# Run all examples with the specified args
@@ -139,3 +138,6 @@ download-design-tokens:
139138
# Update the results of `insta` snapshot regression tests
140139
update-insta-tests:
141140
cargo test; cargo insta review
141+
142+
upload *ARGS:
143+
python3 "scripts/upload_image.py" {{ARGS}}

scripts/requirements-dev.txt

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
# Pip packages all developers need
2+
3+
-r ../rerun_py/requirements-build.txt
4+
-r ../rerun_py/requirements-doc.txt
5+
-r ../rerun_py/requirements-lint.txt
6+
7+
google-cloud-storage==2.9.0 # for scripts/upload_image.py

scripts/upload_image.py

Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
#!/usr/bin/env python3
2+
3+
"""
4+
Upload an image to Google Cloud.
5+
6+
Requires the following packages:
7+
pip install google-cloud-storage # NOLINT
8+
9+
Before running, you have to authenticate via the Google Cloud CLI:
10+
- Install it (https://cloud.google.com/storage/docs/gsutil_install)
11+
- Set up credentials (https://cloud.google.com/storage/docs/gsutil_install#authenticate)
12+
"""
13+
14+
import argparse
15+
import hashlib
16+
import mimetypes
17+
import os
18+
19+
from google.cloud import storage
20+
21+
22+
def content_hash(path: str) -> str:
23+
h = hashlib.sha1()
24+
b = bytearray(128 * 1024)
25+
mv = memoryview(b)
26+
with open(path, "rb", buffering=0) as f:
27+
while n := f.readinto(mv):
28+
h.update(mv[:n])
29+
return h.hexdigest()
30+
31+
32+
def main() -> None:
33+
parser = argparse.ArgumentParser(description="Upload an image.")
34+
parser.add_argument("path", type=str, help="Path to the image.")
35+
args = parser.parse_args()
36+
37+
hash = content_hash(args.path)
38+
object_name = f"{hash}_{os.path.basename(args.path)}"
39+
40+
gcs = storage.Client()
41+
bucket = gcs.bucket("rerun-static-img")
42+
destination = bucket.blob(object_name)
43+
destination.content_type, destination.content_encoding = mimetypes.guess_type(args.path)
44+
with open(args.path, "rb") as f:
45+
destination.upload_from_file(f)
46+
47+
print(f"https://static.rerun.io/{object_name}")
48+
49+
50+
if __name__ == "__main__":
51+
main()

0 commit comments

Comments
 (0)