Skip to content

Commit

Permalink
fix: Enhance artifact visualization. Fixes #8619 (#8655)
Browse files Browse the repository at this point in the history
Signed-off-by: Alex Collins <[email protected]>
  • Loading branch information
alexec authored May 6, 2022
1 parent 16fef4e commit 3fdf30d
Show file tree
Hide file tree
Showing 20 changed files with 398 additions and 114 deletions.
9 changes: 6 additions & 3 deletions docs/artifact-visualization.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,18 +12,21 @@ Use cases:

[![Demo](https://img.youtube.com/vi/whoRfYY9Fhk/0.jpg)](https://youtu.be/whoRfYY9Fhk)

* Artifacts now appear as elements in the workflow DAG that you can click on.
* Artifacts appear as elements in the workflow DAG that you can click on.
* When you click on the artifact, a panel appears.
* The first time this appears explanatory text is shown to help you understand if you might need to change your
workflows to use this new feature.
workflows to use this feature.
* Known file types such as images, text or HTML are shown in an inline-frame (`iframe`).
* Artifacts are sandboxed using a Content-Security-Policy that prevents JavaScript execution.
* JSON is shown with syntax highlighting.

To start, take a look at
a [fully formed example](https://github.com/argoproj/argo-workflows/blob/master/examples/artifacts-workflowtemplate.yaml)
the [example](https://github.com/argoproj/argo-workflows/blob/master/examples/artifacts-workflowtemplate.yaml)
.

![Graph Report](assets/graph-report.png)
![Test Report](assets/test-report.png)

## Artifact Types

An artifact maybe a `.tgz`, file or directory.
Expand Down
Binary file added docs/assets/graph-report.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/test-report.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
48 changes: 46 additions & 2 deletions docs/fields.md
Original file line number Diff line number Diff line change
Expand Up @@ -394,6 +394,8 @@ WorkflowTemplate is the definition of a workflow template resource

- [`buildkit-template.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/buildkit-template.yaml)

- [`ci-workflowtemplate.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/ci-workflowtemplate.yaml)

- [`cron-backfill.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/cron-backfill.yaml)

- [`dag-inline-workflowtemplate.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/dag-inline-workflowtemplate.yaml)
Expand Down Expand Up @@ -445,6 +447,8 @@ WorkflowSpec is the specification of a Workflow.

- [`ci-output-artifact.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/ci-output-artifact.yaml)

- [`ci-workflowtemplate.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/ci-workflowtemplate.yaml)

- [`ci.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/ci.yaml)

- [`cluster-wftmpl-dag.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/cluster-workflow-template/cluster-wftmpl-dag.yaml)
Expand Down Expand Up @@ -867,6 +871,8 @@ CronWorkflowSpec is the specification of a CronWorkflow

- [`ci-output-artifact.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/ci-output-artifact.yaml)

- [`ci-workflowtemplate.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/ci-workflowtemplate.yaml)

- [`ci.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/ci.yaml)

- [`cluster-wftmpl-dag.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/cluster-workflow-template/cluster-wftmpl-dag.yaml)
Expand Down Expand Up @@ -1234,6 +1240,8 @@ Arguments to a template

- [`ci-output-artifact.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/ci-output-artifact.yaml)

- [`ci-workflowtemplate.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/ci-workflowtemplate.yaml)

- [`ci.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/ci.yaml)

- [`cluster-wftmpl-dag.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/cluster-workflow-template/cluster-wftmpl-dag.yaml)
Expand Down Expand Up @@ -1800,6 +1808,8 @@ Outputs hold parameters, artifacts, and results from a step

- [`ci-output-artifact.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/ci-output-artifact.yaml)

- [`ci-workflowtemplate.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/ci-workflowtemplate.yaml)

- [`conditional-artifacts.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/conditional-artifacts.yaml)

- [`conditional-parameters.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/conditional-parameters.yaml)
Expand Down Expand Up @@ -1910,6 +1920,8 @@ Artifact indicates an artifact to place at a specified path

- [`ci-output-artifact.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/ci-output-artifact.yaml)

- [`ci-workflowtemplate.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/ci-workflowtemplate.yaml)

- [`ci.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/ci.yaml)

- [`conditional-artifacts.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/conditional-artifacts.yaml)
Expand Down Expand Up @@ -1999,6 +2011,8 @@ Parameter indicate a passed string parameter to a service template with an optio

- [`ci-output-artifact.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/ci-output-artifact.yaml)

- [`ci-workflowtemplate.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/ci-workflowtemplate.yaml)

- [`ci.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/ci.yaml)

- [`cluster-wftmpl-dag.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/cluster-workflow-template/cluster-wftmpl-dag.yaml)
Expand Down Expand Up @@ -2315,6 +2329,8 @@ _No description available_
<summary>Examples with this field (click to open)</summary>
<br>

- [`artifacts-workflowtemplate.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/artifacts-workflowtemplate.yaml)

- [`graph-workflow.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/container-set-template/graph-workflow.yaml)

- [`outputs-result-workflow.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/container-set-template/outputs-result-workflow.yaml)
Expand Down Expand Up @@ -2343,6 +2359,8 @@ DAGTemplate is a template subtype for directed acyclic graph templates

- [`buildkit-template.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/buildkit-template.yaml)

- [`ci-workflowtemplate.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/ci-workflowtemplate.yaml)

- [`cluster-wftmpl-dag.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/cluster-workflow-template/cluster-wftmpl-dag.yaml)

- [`clustertemplates.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/cluster-workflow-template/clustertemplates.yaml)
Expand Down Expand Up @@ -2552,6 +2570,8 @@ Inputs are the mechanism for passing parameters, artifacts, volumes from one tem

- [`ci-output-artifact.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/ci-output-artifact.yaml)

- [`ci-workflowtemplate.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/ci-workflowtemplate.yaml)

- [`ci.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/ci.yaml)

- [`clustertemplates.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/cluster-workflow-template/clustertemplates.yaml)
Expand Down Expand Up @@ -3149,6 +3169,8 @@ ArchiveStrategy describes how to archive files/directory when saving artifacts

- [`artifacts-workflowtemplate.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/artifacts-workflowtemplate.yaml)

- [`ci-workflowtemplate.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/ci-workflowtemplate.yaml)

- [`map-reduce.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/map-reduce.yaml)

- [`output-artifact-s3.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/output-artifact-s3.yaml)
Expand Down Expand Up @@ -3568,6 +3590,8 @@ _No description available_
<summary>Examples with this field (click to open)</summary>
<br>

- [`artifacts-workflowtemplate.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/artifacts-workflowtemplate.yaml)

- [`graph-workflow.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/container-set-template/graph-workflow.yaml)

- [`outputs-result-workflow.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/container-set-template/outputs-result-workflow.yaml)
Expand Down Expand Up @@ -3657,6 +3681,8 @@ DAGTask represents a node in the graph during DAG execution

- [`buildkit-template.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/buildkit-template.yaml)

- [`ci-workflowtemplate.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/ci-workflowtemplate.yaml)

- [`cluster-wftmpl-dag.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/cluster-workflow-template/cluster-wftmpl-dag.yaml)

- [`clustertemplates.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/cluster-workflow-template/clustertemplates.yaml)
Expand Down Expand Up @@ -4078,6 +4104,8 @@ NoneStrategy indicates to skip tar process and upload the files or directory tre

- [`artifacts-workflowtemplate.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/artifacts-workflowtemplate.yaml)

- [`ci-workflowtemplate.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/ci-workflowtemplate.yaml)

- [`map-reduce.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/map-reduce.yaml)

- [`output-artifact-s3.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/output-artifact-s3.yaml)
Expand Down Expand Up @@ -4294,6 +4322,8 @@ ObjectMeta is metadata that all persisted resources must have, which includes al

- [`ci-output-artifact.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/ci-output-artifact.yaml)

- [`ci-workflowtemplate.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/ci-workflowtemplate.yaml)

- [`ci.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/ci.yaml)

- [`cluster-wftmpl-dag.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/cluster-workflow-template/cluster-wftmpl-dag.yaml)
Expand Down Expand Up @@ -4754,6 +4784,8 @@ PersistentVolumeClaim is a user's request for and claim to a persistent volume

- [`ci-output-artifact.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/ci-output-artifact.yaml)

- [`ci-workflowtemplate.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/ci-workflowtemplate.yaml)

- [`ci.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/ci.yaml)

- [`fun-with-gifs.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/fun-with-gifs.yaml)
Expand All @@ -4780,6 +4812,8 @@ Volume represents a named volume in a pod that may be accessed by any container
<summary>Examples with this field (click to open)</summary>
<br>

- [`artifacts-workflowtemplate.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/artifacts-workflowtemplate.yaml)

- [`buildkit-template.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/buildkit-template.yaml)

- [`workspace-workflow.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/container-set-template/workspace-workflow.yaml)
Expand Down Expand Up @@ -4920,12 +4954,12 @@ A single application container that you want to run within a pod.

- [`artifactory-artifact.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/artifactory-artifact.yaml)

- [`artifacts-workflowtemplate.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/artifacts-workflowtemplate.yaml)

- [`buildkit-template.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/buildkit-template.yaml)

- [`ci-output-artifact.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/ci-output-artifact.yaml)

- [`ci-workflowtemplate.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/ci-workflowtemplate.yaml)

- [`ci.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/ci.yaml)

- [`clustertemplates.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/cluster-workflow-template/clustertemplates.yaml)
Expand Down Expand Up @@ -5229,10 +5263,14 @@ VolumeMount describes a mounting of a Volume within a container.
<summary>Examples with this field (click to open)</summary>
<br>

- [`artifacts-workflowtemplate.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/artifacts-workflowtemplate.yaml)

- [`buildkit-template.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/buildkit-template.yaml)

- [`ci-output-artifact.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/ci-output-artifact.yaml)

- [`ci-workflowtemplate.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/ci-workflowtemplate.yaml)

- [`ci.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/ci.yaml)

- [`workspace-workflow.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/container-set-template/workspace-workflow.yaml)
Expand Down Expand Up @@ -5354,6 +5392,8 @@ ResourceRequirements describes the compute resource requirements.

- [`ci-output-artifact.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/ci-output-artifact.yaml)

- [`ci-workflowtemplate.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/ci-workflowtemplate.yaml)

- [`ci.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/ci.yaml)

- [`dns-config.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/dns-config.yaml)
Expand Down Expand Up @@ -5587,6 +5627,8 @@ PersistentVolumeClaimSpec describes the common attributes of storage devices and

- [`ci-output-artifact.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/ci-output-artifact.yaml)

- [`ci-workflowtemplate.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/ci-workflowtemplate.yaml)

- [`ci.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/ci.yaml)

- [`cluster-wftmpl-dag.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/cluster-workflow-template/cluster-wftmpl-dag.yaml)
Expand Down Expand Up @@ -6043,6 +6085,8 @@ Represents an empty directory for a pod. Empty directory volumes support ownersh
<summary>Examples with this field (click to open)</summary>
<br>

- [`artifacts-workflowtemplate.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/artifacts-workflowtemplate.yaml)

- [`workspace-workflow.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/container-set-template/workspace-workflow.yaml)

- [`init-container.yaml`](https://github.com/argoproj/argo-workflows/blob/master/examples/init-container.yaml)
Expand Down
6 changes: 4 additions & 2 deletions docs/walk-through/continuous-integration-example.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
# Continuous Integration Example

Continuous integration is a popular application for workflows. Currently, Argo does not provide event triggers for automatically kicking off your CI jobs, but we plan to do so in the near future. Until then, you can easily write a cron job that checks for new commits and kicks off the needed workflow, or use your existing Jenkins server to kick off the workflow.
Continuous integration is a popular application for workflows.

A good example of a CI workflow spec is provided at <https://github.com/argoproj/argo-workflows/tree/master/examples/influxdb-ci.yaml>. Because it just uses the concepts that we've already covered and is somewhat long, we don't go into details here.
One example of a CI workflow is
<https://github.com/argoproj/argo-workflows/tree/master/examples/influxdb-ci.yaml>. Because it uses the concepts
that we've already covered, we don't go into details here.
116 changes: 74 additions & 42 deletions examples/artifacts-workflowtemplate.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -9,45 +9,83 @@ spec:
entrypoint: main
templates:
- name: main
volumes:
- name: in
emptyDir: { }
- name: out
emptyDir: { }
inputs:
artifacts:
- name: argo-logo
path: /out/assets/argo.png
- name: temps
path: /in/annual.csv
http:
url: https://raw.githubusercontent.com/argoproj/argo-workflows/master/docs/assets/argo.png
container:
image: argoproj/argosay:v2
command:
- sh
- -c
args:
- |
mkdir /out
cowsay "hello world" > /out/hello.txt
cat > /out/hello.json <<EOF
{"hello": {"world": true}}
EOF
echo '* {font-family: sans-serif}' > /out/assets/styles.css
cat > /out/index.html <<EOF
<html>
<head>
<link rel='stylesheet' href='assets/styles.css' type='text/css'/>
</head>
<body>
<h1>Hello World</h1>
<img src='assets/argo.png'/>
</body>
</html>
EOF
cat > /out/malicious.html <<EOF
<html>
<body>
<script>alert(1)</script>
<p>This page attempts to run a script that shows an alert, but the Argo Server UI Content-Security-Policy will prevent that.</p>
<p>To check, open your Web Console and see that "Blocked script execution ... because the document's frame is sandboxed." (or similar) is printed.</p>
</body>
</html>
EOF
url: https://datahub.io/core/global-temp/r/annual.csv
containerSet:
volumeMounts:
- mountPath: /in
name: in
- mountPath: /out
name: out
containers:
- name: setup
image: argoproj/argosay:v2
command:
- sh
- -c
args:
- |
mkdir -p /out/assets
- name: gnuplot
image: remuslazar/gnuplot
dependencies:
- setup
args:
- -e
- |
set xlabel 'Year'; set ylabel 'Mean';
set grid;
set datafile separator ',';
set term png size 600,400;
set output '/out/assets/global-temp.png';
plot '/in/annual.csv' every 2::0 skip 1 using 2:3 title 'Global Temperature' with lines linewidth 2;
- name: main
image: argoproj/argosay:v2
dependencies:
- setup
command:
- sh
- -c
args:
- |
cowsay "hello world" > /out/hello.txt
cat > /out/hello.json <<EOF
{"hello": {"world": true}}
EOF
echo '* {font-family: sans-serif}' > /out/assets/styles.css
cat > /out/index.html <<EOF
<html>
<head>
<link rel='stylesheet' href='assets/styles.css' type='text/css'/>
</head>
<body>
<h1>Global Temperature</h1>
<img src='assets/global-temp.png'/>
</body>
</html>
EOF
cat > /out/malicious.html <<EOF
<html>
<body>
<script>alert(1)</script>
<p>This page attempts to run a script that shows an alert, but the Argo Server UI Content-Security-Policy will prevent that.</p>
<p>To check, open your Web Console and see that "Blocked script execution ... because the document's frame is sandboxed." (or similar) is printed.</p>
</body>
</html>
EOF
outputs:
artifacts:
# saving single files
Expand All @@ -57,12 +95,6 @@ spec:
key: hello.txt
archive:
none: { }
- name: image-file
path: /out/assets/argo.png
s3:
key: argo.png
archive:
none: { }
# JSON files are shown with syntax highlighting.
- name: json-file
path: /out/hello.json
Expand Down
Loading

0 comments on commit 3fdf30d

Please sign in to comment.