Skip to content

Update example symbolic link creation in Dockerfile.template#76

Merged
dongjoon-hyun merged 1 commit intoapache:masterfrom
yaooqinn:temp
Oct 31, 2024
Merged

Update example symbolic link creation in Dockerfile.template#76
dongjoon-hyun merged 1 commit intoapache:masterfrom
yaooqinn:temp

Conversation

@yaooqinn
Copy link
Member

What changes were proposed in this pull request?

Update example symbolic link creation in Dockerfile.template

Why are the changes needed?

requirement from the docker official community

Does this PR introduce any user-facing change?

no

How was this patch tested?

locally ran tools/template.py

    tar -xf spark.tgz --strip-components=1; \
    chown -R spark:spark .; \
    mv jars /opt/spark/; \
    mv RELEASE /opt/spark/; \
    mv bin /opt/spark/; \
    mv sbin /opt/spark/; \
    mv kubernetes/dockerfiles/spark/decom.sh /opt/; \
    mv examples /opt/spark/; \
    ln -s "$(basename /opt/spark/examples/jars/spark-examples_*.jar)" /opt/spark/examples/jars/spark-examples.jar; \
    mv kubernetes/tests /opt/spark/; \
    mv data /opt/spark/; \
    mv python/pyspark /opt/spark/python/pyspark/; \
    mv python/lib /opt/spark/python/lib/; \
    mv R /opt/spark/; \
    chmod a+x /opt/decom.sh; \
    cd ..; \
    rm -rf "$SPARK_TMP";

@yaooqinn
Copy link
Member Author

cc @dongjoon-hyun

Copy link
Member

@dongjoon-hyun dongjoon-hyun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, LGTM. Thank you, @yaooqinn !

@dongjoon-hyun dongjoon-hyun merged commit 51acf3d into apache:master Oct 31, 2024
dongjoon-hyun added a commit to apache/spark that referenced this pull request Oct 31, 2024
….jar`

### What changes were proposed in this pull request?

This PR aims to simplify symbolic link creation of `spark-examples.jar` according to the downstream `docker-library` and `spark-docker` repository change.

### Why are the changes needed?

- `docker-library`
  - docker-library/official-images#17622 (comment)

- `spark-docker`
  - apache/spark-docker#73
  - apache/spark-docker#74
  - apache/spark-docker#76

### Does this PR introduce _any_ user-facing change?

No behavior change.

### How was this patch tested?

Pass the CIs.

### Was this patch authored or co-authored using generative AI tooling?

No.

Closes #48723 from dongjoon-hyun/SPARK-50192.

Authored-by: Dongjoon Hyun <dongjoon@apache.org>
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

Comments