[Website 2.0] General Version Dropdown #17948

connorgoggins · 2020-04-01T00:16:11Z

Description

This PR resolves the issues outlined in the Github issue for the MXNet Website 2.0 General Version Dropdown. It serves to add a fully functional API version dropdown with consistent styling to the master website, and also modifies the Jenkins build instructions to download static artifacts from an external repository. Currently, this repository of static artifacts includes websites for the following past versions of the MXNet API:

1.6
1.5.0
1.4.1
1.3.1
1.2.1
1.1.0
1.0.0
0.11.0
0.12.1

Each version of the website also has a similarly formatted API version dropdown in the header of each page, making it easy for users to seamlessly navigate between different versions of the MXNet website.

Checklist

Essentials

Changes are complete (i.e. I finished coding on this PR)
Code is well-documented
To the best of my knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change

Changes

M docs/python_docs/themes/mx-theme/mxtheme/header_top.html
M docs/python_docs/themes/mx-theme/mxtheme/layout.html
M docs/static_site/Makefile
M docs/static_site/src/_includes/head.html
M docs/static_site/src/_includes/header.html

Comments

Preview is currently available here.

For instructions on adding future release versions (either as static artifacts or via dynamic build pipelines) please read the wiki page or the Confluence page.

@sandeep-krishnamurthy @aaronmarkham @szha @sojiadeshina @leezu

mxnet-bot · 2020-04-01T00:16:16Z

Hey @connorgoggins , Thanks for submitting the PR
All tests are already queued to run once. If tests fail, you can trigger one or more tests again with the following commands:

To trigger all jobs: @mxnet-bot run ci [all]
To trigger specific jobs: @mxnet-bot run ci [job1, job2]

CI supported jobs: [clang, edge, centos-gpu, centos-cpu, windows-cpu, miscellaneous, unix-cpu, windows-gpu, unix-gpu, website, sanity]

Note:
Only following 3 categories can trigger CI :PR Author, MXNet Committer, Jenkins Admin.
All CI tests must pass before the PR can be merged.

szha · 2020-04-01T05:37:51Z

Thanks for introducing the versioned website.

In terms of appearance, could we make the version selector more visually distinct from other links? From the current look it doesn't seem obvious that a dropdown menu is there.

Another ask is that since we are maintaining both the mxnet 1.x (v1.x branch, w/ 1.7 to be the next version) and 2.x (master branch, w/ 2.0 to be the next version), can we make nightly builds for these dev branches available? See #17701 for details.

docs/python_docs/themes/mx-theme/mxtheme/header_top.html

docs/static_site/Makefile

aaronmarkham

I have some organizational preferences for styles... so it is easier to maintain. Can you do that in a followup PR?

aaronmarkham · 2020-04-01T17:45:31Z

docs/python_docs/themes/mx-theme/mxtheme/layout.html

@@ -7,6 +7,27 @@
    <meta charset="utf-8">
    <meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
    <meta http-equiv="x-ua-compatible" content="ie=edge">
+    <style>
+    .dropdown {


Can you make sure all of the style stuff is documented - so we know what it does and how it interacts with all of the other pieces? Like how does this interact with this?
https://github.com/apache/incubator-mxnet/blob/master/docs/python_docs/_static/mxnet.css

Sure, I'll create a document/wiki page with styling info once the design is finalized.

aaronmarkham · 2020-04-01T17:46:36Z

docs/static_site/src/_includes/head.html

@@ -15,4 +15,25 @@

  <script src="{{'/assets/js/clipboard.js'|relative_url}}"></script>
  <script src="{{'/assets/js/copycode.js'|relative_url}}"></script>
+  <style>


Duplicated in layout.html. Should be abstracted to a single source so it's easier to manage.

aaronmarkham · 2020-04-01T17:46:57Z

docs/static_site/src/_includes/header.html

+        <div class="dropdown">
+          <span>master</span>
+          <div class="dropdown-content">
+            <a style="color:#FF4500;" href="/">master</a>


move to central css

aaronmarkham · 2020-04-01T17:47:33Z

docs/python_docs/themes/mx-theme/mxtheme/header_top.html

+        <div class="dropdown">
+          <span style="color:#FFFFFF">master</span>
+          <div class="dropdown-content">
+            <a style="color:#FF4500;font-weight: lighter;" href="/">master</a><br>


Can you move styles to central css file?

connorgoggins · 2020-04-01T17:58:10Z

@szha thanks for your feedback! I would be happy to modify the styling of the version dropdown. Can you give me an idea of what exactly you had in mind?

Regarding the nightly build configurations, since v1.x and v2.x use restricted nodes (see Jenkinsfile_website_full) these builds would need to be run on prod Jenkins, which I currently do not have admin access to. At the moment, I am trying to get access to prod Jenkins so I can set up these nightly build pipelines.

connorgoggins · 2020-04-01T23:44:10Z

@szha I created two PRs with changes that will add Jenkinsfiles to support nightly builds for v1.x and v2.x (master). When you have some free time, would you mind reviewing them?

marcoabreu · 2020-04-02T01:51:54Z

docs/static_site/Makefile

@@ -20,7 +20,7 @@ all: html
 html:
 	mkdir -p build
 	cd src && bundle install && JEKYLL_ENV=production bundle exec jekyll build --config _config_prod.yml -d ../build/html && cd ..
-
+	wget https://mxnet-static-artifacts.s3.amazonaws.com/versions.zip && unzip versions.zip -d build/html && rm -rf build/html/__MACOSX && rm versions.zip


Where does that come from? It seems like we're breaking out of a context here. S3 should not be the method how multiple Jenkins jobs exchange data

Thanks for your question, @marcoabreu. The content for the static artifacts is managed in this repository. Originally this line was a call to directly clone the repository, but based on feedback it was decided that it would be better to not rely on external repositories and pull the artifacts from an S3 bucket instead.

Here, S3 isn't being used as the way multiple Jenkins jobs exchange data per se. There are many older static artifacts in the ZIP that have gone through multiple rounds of changes which are tracked in the aforementioned repository, rather than simply being built by another Jenkins pipeline.

Here's an idea: add this to the Dockerfile so that it gets cached as part of the image and we don't have to grab that zip every time there's a test or build.

But how would you invalidate that cache?

I think it's important to have continuity to allow a contributor to understand and also reproduce how our website gets created. As it stands right now, people would not be able to facilitate this.

What happened to the idea with the Jenkins jobs creating artifacts and another job consuming these artifacts to then assemble the website?

Indeed, this is a static artifact that can't be directly updated by the community. For a complete solution, we definitely need them to be generated.

On the other hand, the lack of website versioning is such a pressing issue that it has come up multiple times, even in past release votes, and is still actively causing confusion for many current and prospective users. We must be losing user base because of this.

If the contributors could come up with a plan for a solution that's based on automatically generated content, and execute the plan before any of our next releases, I think we should accept this solution for now to address the immediate need for avoiding confusion.

I see that @connorgoggins is already contributing other PRs in this direction (#17956 #17957), I trust that the contributor has the intention to make the generation automated. @connorgoggins @sandeep-krishnamurthy contingent on if you could spell out a plan for automated generation for the website, I support accepting this patch.

Agreed @szha. The priority right now should be getting the working version dropdown in this PR into the production site as soon as possible.

@marcoabreu, I have created a confluence page so that all contributors can understand and reproduce the steps to add additional releases to the website. In the future, we will move towards creating dynamic pipelines to automatically generate and integrate new releases of MXNet into the website, but for now I believe the static artifact solution + master integration in this PR serves its purpose.

@marcoabreu Thanks for your feedback and I fully agree that if generation of web content happens through Jenkins in an automated way, all community members can participate in the updates. In that direction, @connorgoggins already has PRs to make 1.x and 2.x branches auto builds. The fix here is very critical for MXNet community. Without this change, users are unable to use MXNet website. One concern would be if there is a patch release on 1.6.x then someone needs to regenerate static artifacts. Given that @connorgoggins already documented steps for the same and his other contributions I see this as a non-blocking issue that will not leave MXNet community in limbo.
What do you think?

Sounds good to me. Thanks

connorgoggins · 2020-04-02T05:40:30Z

Update: the new version of the website preview reflects the changes I have made to this PR to incorporate feedback. @szha the styling of the dropdown in 1.6 and master has been updated to more clearly indicate that the menu option is a dropdown.

marcoabreu · 2020-04-02T10:50:37Z

To me it's unclear how that magic versions.zip gets created. Could you maybe make a diagram of how the individual versions get created, where they are stored and how they are merged? Some data flow would be good

connorgoggins · 2020-04-02T17:15:04Z

@marcoabreu see "Jenkins Artifact Generation" within the Static Artifact Publication section of the wiki page for detailed instructions on how to add a new artifact, create versions.zip, and upload to S3.

marcoabreu · 2020-04-02T17:27:58Z

I'm no longer an Amazon employee. This content should be hosted on confluence.

szha · 2020-04-02T18:30:24Z

The updated style looks good to me. Thanks @connorgoggins

connorgoggins · 2020-04-02T20:06:30Z

@marcoabreu the Confluence page with instructions for adding releases is now public.

sandeep-krishnamurthy

Thanks for your contributions!

…eployment

connorgoggins requested review from aaronmarkham and szha as code owners April 1, 2020 00:16

connorgoggins mentioned this pull request Apr 1, 2020

[Website 2.0] General Version Dropdown #17798

Closed

sandeep-krishnamurthy reviewed Apr 1, 2020

View reviewed changes

docs/python_docs/themes/mx-theme/mxtheme/header_top.html Outdated Show resolved Hide resolved

docs/static_site/Makefile Outdated Show resolved Hide resolved

aaronmarkham approved these changes Apr 1, 2020

View reviewed changes

connorgoggins requested a review from marcoabreu as a code owner April 2, 2020 00:16

marcoabreu reviewed Apr 2, 2020

View reviewed changes

sandeep-krishnamurthy approved these changes Apr 3, 2020

View reviewed changes

connorgoggins force-pushed the website_version_dropdown branch 3 times, most recently from e046e7c to 398beb5 Compare April 7, 2020 22:10

connorgoggins added 9 commits April 8, 2020 06:16

Added instructions for pulling & decompressing artifacts for static d…

cee4319

…eployment

Added master dropdown to header HTML

7ca2ea2

Added CSS content to style master dropdown

199b642

Using full path for wget for Jenkins

6d81012

Added install command for wget

da51752

Using sudo w/apt

f80a04a

Switching to git clone to satisfy Jenkins

2007cd2

Added step to archive artifacts

12768ce

Reverting Jekyll Jenkinsfile

afd89dd

connorgoggins added 24 commits April 8, 2020 06:16

Switching to non-restricted nodes for dev Jenkins build

e1706d5

Final restricted change

43aab2e

Fixed styling for dropdown

5f4bc4d

Adding dropdown to Python docs header

8fc1cb8

Adding CSS for Python docs header version dropdown

355e837

Fixed text alignment for hamburger menu

450b2a9

Trying to get styling working for Python docs api dropdown

93814b6

Trying to add css at _root

2d78232

Trying to add styling to fonts, added inline styling

40524fd

Trying style tags in layout

eac354f

Changed font weight

0890b52

Added missing semicolon in inline styling

6bb717e

Switched header to be consistent with older versions

e64e3b9

Changed header in master

9432f41

Switching hrefs to point to HTML files

fb59212

Added 1.6 link to master

9b69be0

Dropping unnecessary changes

3aabc7e

Revert _root.scss

fdf6e5e

Fixed routing bug

5431ce0

Reverting Jenkinsfile_website_full for PR

b190a2e

Switching from git clone to wget on S3

229c676

Adding relevant build dependencies, fixing ruby version for Jekyll

e954894

Added caret for dropdown

371094d

Consolidating CSS into one file for general, one for Python API

57a7a53

connorgoggins force-pushed the website_version_dropdown branch from 398beb5 to 57a7a53 Compare April 8, 2020 13:16

sandeep-krishnamurthy merged commit 5ff2994 into apache:master Apr 9, 2020

This was referenced Apr 10, 2020

[Website 2.0] Fix RVM version to satisfy Jekyll build in v1.x #18016

Merged

[Website 2.0] Fix RVM version to satisfy Jekyll build in v1.7.x #18071

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Website 2.0] General Version Dropdown #17948

[Website 2.0] General Version Dropdown #17948

connorgoggins commented Apr 1, 2020 •

edited

Loading

mxnet-bot commented Apr 1, 2020

szha commented Apr 1, 2020

aaronmarkham left a comment

aaronmarkham Apr 1, 2020

connorgoggins Apr 1, 2020

aaronmarkham Apr 1, 2020

aaronmarkham Apr 1, 2020

aaronmarkham Apr 1, 2020

connorgoggins commented Apr 1, 2020

connorgoggins commented Apr 1, 2020

marcoabreu Apr 2, 2020

connorgoggins Apr 2, 2020

aaronmarkham Apr 2, 2020

marcoabreu Apr 2, 2020

szha Apr 2, 2020

szha Apr 2, 2020

connorgoggins Apr 2, 2020

sandeep-krishnamurthy Apr 3, 2020

marcoabreu Apr 3, 2020

connorgoggins commented Apr 2, 2020

marcoabreu commented Apr 2, 2020

connorgoggins commented Apr 2, 2020

marcoabreu commented Apr 2, 2020

szha commented Apr 2, 2020

connorgoggins commented Apr 2, 2020

sandeep-krishnamurthy left a comment

[Website 2.0] General Version Dropdown #17948

[Website 2.0] General Version Dropdown #17948

Conversation

connorgoggins commented Apr 1, 2020 • edited Loading

Description

Checklist

Essentials

Changes

Comments

mxnet-bot commented Apr 1, 2020

szha commented Apr 1, 2020

aaronmarkham left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

connorgoggins commented Apr 1, 2020

connorgoggins commented Apr 1, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

connorgoggins commented Apr 2, 2020

marcoabreu commented Apr 2, 2020

connorgoggins commented Apr 2, 2020

marcoabreu commented Apr 2, 2020

szha commented Apr 2, 2020

connorgoggins commented Apr 2, 2020

sandeep-krishnamurthy left a comment

Choose a reason for hiding this comment

connorgoggins commented Apr 1, 2020 •

edited

Loading