Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BUILDPACK_S3_BASE_URL can't be set from app env vars #989

Closed
edmorley opened this issue Jul 7, 2020 · 2 comments
Closed

BUILDPACK_S3_BASE_URL can't be set from app env vars #989

edmorley opened this issue Jul 7, 2020 · 2 comments

Comments

@edmorley
Copy link
Member

edmorley commented Jul 7, 2020

The BUILDPACK_S3_BASE_URL functionality can't currently be used via app env vars:

# Set the base URL for downloading buildpack assets like Python runtimes.
# The user can provide BUILDPACK_S3_BASE_URL to specify a custom target.
# Note: this is designed for non-Heroku use, as it does not use the user-provided
# environment variable mechanism (the ENV_DIR).
if [[ "${STACK}" == "cedar-14" ]]; then
# Since Cedar-14 is EOL on 2020-11-02 it's not worth migrating it to the new bucket.
DEFAULT_S3_BASE_URL='https://lang-python.s3.amazonaws.com'
else
DEFAULT_S3_BASE_URL='https://heroku-buildpack-python.s3.amazonaws.com'
fi
S3_BASE_URL="${BUILDPACK_S3_BASE_URL:-${DEFAULT_S3_BASE_URL}}"

There are cases when people wish to use their own binaries on Heroku, but still use this buildpack. We should decide whether to support reading BUILDPACK_S3_BASE_URL from app config vars (currently users must either fork the buildpack, or come up with inventive ways of setting the env, eg via an inline buildpack run first, that uses the export buildpack API feature).

From:
https://heroku.support/887094

@git2gus
Copy link

git2gus bot commented Aug 5, 2020

This issue has been linked to a new work item: W-7918748

edmorley added a commit that referenced this issue Sep 30, 2020
Previously the buildpack's S3 bucket was defined in two places - once
in `VENDOR_URL` and again during the pip installation step. This
duplication was necessary since `VENDOR_URL` also contained the stack's
name, whereas the pip use-case used a non-stack-specific S3 key prefix.

In order to:
* reduce this duplication
* allow overriding of the URL for the pip use-case
* simplify this buildpack's S3 bucket migration (where we'll soon be
  needing the vary the bucket name and wouldn't want to have to
  duplicate that logic in multiple places)

...the `VENDOR_URL` variable has been replaced with `S3_BASE_URL` which
no longer contains the stack name.

The user-configurable override has similarly been renamed from
`BUILDPACK_VENDOR_URL` to `BUILDPACK_S3_BASE_URL`. Note: As before,
this override cannot be set via standard app variables (see #989).

Closes @W-8142401@.
edmorley added a commit that referenced this issue Oct 1, 2020
Previously the buildpack's S3 bucket was defined in two places - once
in `VENDOR_URL` and again during the pip installation step. This
duplication was necessary since `VENDOR_URL` also contained the stack's
name, whereas the pip use-case used a non-stack-specific S3 key prefix.

In order to:
* reduce this duplication
* simplify this buildpack's S3 bucket migration (where we'll soon be
  needing the vary the bucket name and wouldn't want to have to
  duplicate that logic in multiple places)
* allow overriding of the URL for the pip use-case

...the `VENDOR_URL` variable has been replaced with `S3_BASE_URL` which
no longer contains the stack name.

The user-configurable override has similarly been renamed from
`BUILDPACK_VENDOR_URL` to `BUILDPACK_S3_BASE_URL`. Note: As before,
this override cannot be set via standard app variables (see #989).

The unused `USE_STAGING_BINARIES` environment variable has been
removed, since it's a leftover from the project to stand up a staging S3 bucket.
It's redundant given the `BUILDPACK_S3_BASE_URL` variable.

Closes @W-8142401@.
@edmorley edmorley changed the title BUILDPACK_VENDOR_URL can't be set from app env vars BUILDPACK_S3_BASE_URL can't be set from app env vars Oct 23, 2020
@edmorley
Copy link
Member Author

Wontfixing this, since using custom binaries is an advanced use-case, and making it too easy to set a custom bucket via app env vars could lead to some hard to debug support tickets. For use-cases where custom binaries are required, the env var can still be set by an earlier buildpack (or a fork of this buildpack).

dryan pushed a commit to dryan/heroku-buildpack-python that referenced this issue Nov 19, 2020
Previously the buildpack's S3 bucket was defined in two places - once
in `VENDOR_URL` and again during the pip installation step. This
duplication was necessary since `VENDOR_URL` also contained the stack's
name, whereas the pip use-case used a non-stack-specific S3 key prefix.

In order to:
* reduce this duplication
* simplify this buildpack's S3 bucket migration (where we'll soon be
  needing the vary the bucket name and wouldn't want to have to
  duplicate that logic in multiple places)
* allow overriding of the URL for the pip use-case

...the `VENDOR_URL` variable has been replaced with `S3_BASE_URL` which
no longer contains the stack name.

The user-configurable override has similarly been renamed from
`BUILDPACK_VENDOR_URL` to `BUILDPACK_S3_BASE_URL`. Note: As before,
this override cannot be set via standard app variables (see heroku#989).

The unused `USE_STAGING_BINARIES` environment variable has been
removed, since it's a leftover from the project to stand up a staging S3 bucket.
It's redundant given the `BUILDPACK_S3_BASE_URL` variable.

Closes @W-8142401@.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant