Beta build #16411
Beta build #16411
Conversation
|
Mixing concerns or did you just forgot to base off master? |
aaronmarkham
force-pushed
the
beta_build
branch
from
359c7fc
to
863abc6
Compare
I messed up my master branch and tainted this one too. Should be good now. |
|
@reminisce @stu1130 |
|
The sanity check is flaky, the easiest way is to retrigger it |
So 2/2 failures on this PR, and 4/4 on #16392 is flaky? That's more like totally broken. |
|
Haha ok, it passed. Of course it did right after I leave that comment. But still that's a really low pass rate. Why should this hold everyone hostage? |
| utils.docker_run('ubuntu_cpu_jekyll', 'build_docs', false) | ||
|
|
||
| // archive so the publish pipeline can access the artifact | ||
| archiveArtifacts 'docs/_build/beta_website.tgz' |
There was a problem hiding this comment.
where is this being prepared ?
There was a problem hiding this comment.
Did I miss something?
Job calls ci/jenkins/Jenkinsfile_website_beta
--> unix_cpu_lite --> binary
--> docs_jekyll --> jekyll artifacts (stash)
--> docs_python --> python api artifacts (stash)
--> docs_prepare_beta (unstash both) --> archive beta_website.tgz
--> docs_publish_beta (using archive)
I did forget to update the filename in the publish job, so this was a worthwhile exercise 🏃 😄
There was a problem hiding this comment.
You are running utils.docker_run('ubuntu_cpu_jekyll', 'build_docs', false)
check runtime_function.sh:1819, the doc/_build/beta_website.tgz is not going to be created by magic 😄
You need to create a new runtime function, build_docs_beta for example that would do something like this:
build_docs_beta() {
pushd docs/_build
tar -xzf jekyll-artifacts.tgz
api_folder='html/api'
# Python has it's own landing page/site so we don't put it in /docs/api
mkdir -p $api_folder/python/docs && tar -xzf python-artifacts.tgz --directory $api_folder/python/docs
GZIP=-9 tar -zcvf beta_website.tgz -C html .
popd
}
There was a problem hiding this comment.
But please let's try to not diverge between bets and prod code as much as possible.
There was a problem hiding this comment.
But please let's try to not diverge between bets and prod code as much as possible.
This does have the drawback of not rendering most of the other microsites, but then we get a much faster way to stage updates. How about after most of the restructuring is done on the Python API we go back and add in the rest of the APIs so the staging run has everything in it?
I tried it again after I made the changes you suggested and it worked: This PR should be ready to merge now. |
* remove unused jenkinsfile * add a pipeline that publishes to staging beta site * add staging site tar file generation; fix outputs
Description
⭐️ This PR adds a pipeline that will deploy the Jekyll and Python docs to the staging beta website.
💥
Jenkinsfile_websitewas not being used since the new site deploy, so I removed it. PRs have been usingJenksinfile_website_full_pr.I setup a new Jenkins job for the build that can be manually triggered with restricted-website-build-master-beta.
The pipeline will then trigger a publish on restricted-website-publish-master-beta.
Usage
If you want to use the beta pipeline once this is merged, you can change the restricted-website-build-master-beta job to point to your fork and PR branch, build it, and once it is done preview your stuff at https://mxnet-beta.staged.apache.org/
Notes
I chose to only build Python and Jekyll to save some time in each test deploy. I think that in the future it would be nice to have parameters on the job, so you can select which API docs you want to include in the test deployment.