Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Image uploader script #2164

Merged
merged 10 commits into from
May 29, 2023
3 changes: 3 additions & 0 deletions justfile
Original file line number Diff line number Diff line change
Expand Up @@ -139,3 +139,6 @@ download-design-tokens:
# Update the results of `insta` snapshot regression tests
update-insta-tests:
cargo test; cargo insta review

upload *ARGS:
python3 "scripts/upload_image.py" {{ARGS}}
50 changes: 50 additions & 0 deletions scripts/upload_image.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
#!/usr/bin/env python3

"""
Upload an image to Google Cloud.

Requires the following packages:
pip install google-cloud-storage # NOLINT

Before running, you have to authenticate via the Google Cloud CLI:
- Install it (https://cloud.google.com/storage/docs/gsutil_install)
- Set up credentials (https://cloud.google.com/storage/docs/gsutil_install#authenticate)
"""

import argparse
import hashlib
import mimetypes

from google.cloud import storage
emilk marked this conversation as resolved.
Show resolved Hide resolved


def content_hash(path: str) -> str:
h = hashlib.sha1()
b = bytearray(128 * 1024)
mv = memoryview(b)
with open(path, "rb", buffering=0) as f:
while n := f.readinto(mv):
h.update(mv[:n])
return h.hexdigest()


def main() -> None:
parser = argparse.ArgumentParser(description="Upload an image.")
parser.add_argument("path", type=str, help="Path to the image.")
args = parser.parse_args()

hash = content_hash(args.path)
object_name = f"{hash}_{args.path}"
Copy link
Member

@emilk emilk May 25, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like if I do scripts/upload_image.py /Users/emilk/image.pg, that will result in a really bad object_name. I think we should just use the filename here, not the whole path


gcs = storage.Client()
bucket = gcs.bucket("rerun-static-img")
destination = bucket.blob(object_name)
destination.content_type, destination.content_encoding = mimetypes.guess_type(args.path)
with open(args.path, "rb") as f:
destination.upload_from_file(f)

print(f"https://static.rerun.io/{object_name}")


if __name__ == "__main__":
main()