[Website 2.0] General Version Dropdown

[connorgoggins]

Description

Currently, the MXNet Python API Docs on the website only show functions corresponding to the latest master API. The current version of the website also does not label the API version being used. This leads to frustration from the developer community, as the API functions and tutorials may work differently for older MXNet releases. This means that the current version of the Python docs can be considered broken, as developers using older versions of the API may experience errors simply by following the docs and tutorials as they are written and may decide that MXNet is broken.

It is important to note that this versioning issue is not only limited to the Python API docs, but also applies to the other language API docs and other site content (see features (https://mxnet.apache.org/features), setup instructions (https://mxnet.apache.org/get_started/ubuntu_setup), etc.).

At this point in time, there is no way for users to access older versions of the website content and API docs, or even to see which version of MXNet is described by the website content and documentation.

Proposed Resolution

To remedy this issue, a general website dropdown should be implemented which will allow users to easily navigate between different versions of the MXNet website using a dropdown menu.

These changes will be made in the interest of making the website more useful for MXNet developers by adding important missing functionality and content.

Feedback from the community regarding the proposed implementation is welcome.

Stakeholders

@sandeep-krishnamurthy
@aaronmarkham
@szha
@sojiadeshina

Proposed Implementation

The MXNet website content will default to the latest master, as it currently does. However, in the site header (alongside “Get Started”, “Blog”, “Features”, etc.) a dropdown menu will be added which will allow users to select the desired API version:

Note: in the final version, the dropdown menu will be properly styled with CSS in a manner that is consistent with the rest of the docs.

The root URL for the website will change from https://mxnet.apache.org/ to https://mxnet.apache.org/master/. All relative paths from the root will remain the same. Certain selected versions will be built dynamically to support additional changes, especially to Python docs (master, 1.6, 2.x, etc.), while older versions will be served as static artifacts. As before, the Python docs on the dynamically generated versions of the website will be built with Sphinx.

Deliverables

1. PR for General Website Dropdown

Acceptance Criteria

1. Dropdown should allow users to seamlessly switch between website versions
2. All other navigation functionality within each version of the website should be preserved
3. Styling should be consistent with the current version of the website after the dropdown addition

[leezu]

Thank you for the proposal @connorgoggins! Could you clarify why a Python API + Tutorials only solution is our best choice? My concern is that there may be other parts of the website that can diverge or already have diverged between MXNet 1.x and 2.x.

[connorgoggins]

@leezu thanks for your feedback! When you mention other parts of the website that might benefit from versioning, I believe you might be referring to the other language API docs for MXNet. Currently, each of the other language API docs (Java, Scala, etc.) have their own format, and based on the conversations that we have had with the developer community we believe that it would be best to leave the format of these other docs unchanged. Due to the unique format of the other language API docs, each language will require a unique way of supporting multiple API versions. Projects to support versioning for the other languages are scheduled as future work, but are not currently scoped within the parameters of this project.

[aaronmarkham]

I think a stopgap solution for the other languages is to generate them using the old branches - if they'll run, and post those as static artifacts of micro-sites that are browsable from a list. If they give any trouble - which they will as we try to go back to older and older versions, we leave those out for now.
To get the older stuff to run, we'll probably have to patch those old branches.
The can of worms here is that old branches will have buggy or inoperable tutorials, and if those get generated and posted, they'll get indexed and then get traffic. So we have to specifically prevent that from happening.

[leezu]

Thanks for clarifying @connorgoggins

I believe you might be referring to the other language API docs for MXNet

Besides the other language APIs, there are also installation instructions and other overview pages. For example https://mxnet.apache.org/features, https://mxnet.apache.org/get_started/ubuntu_setup, https://mxnet.apache.org/get_started/osx_setup

Due to the unique format of the other language API docs, each language will require a unique way of supporting multiple API versions.

Following the approach outlined in this proposal, that's correct. However, the work could be greatly reduced by just building each version of the website independently and adding a global version switch feature.

For example, consider https://cmake.org/cmake/help/v3.11/module/FetchContent.html
There is a note "This documents an old version of CMake. Click here to see the latest release. Or, select a version from the drop-down menu above." which allows users to switch to the latest version of the website.

[connorgoggins]

@leezu after discussing with @sojiadeshina, we decided that it's ok to have the setup instructions (build from source) only apply to master. Unique installation instructions (not building from source) for individual versions of MXNet are already supported and showcased on the website (for example, installation instructions for v0.11.0).

Regarding the content at https://mxnet.apache.org/features, creating a working version of each of the current code samples for the old releases of MXNet would not be possible. For example, the Horovod feature code example for distributed training is based on functionality only recently added to MXNet. If we were to add a general website dropdown, we would need to create entirely new feature code samples for each version, which would be outside the scope of this project.

We scoped this project by prioritizing the minimum but most critical functionality that is broken today for any MXNet user coming to the website, with a maximum project duration of 4 weeks. Since >80% of MXNet developers use the Python API, it was decided that the project should focus on versioning the Python API docs. As mentioned above, a general website dropdown would require additional supporting content for each version (such as the features page), necessitating a more extended timeline and continued support/maintenance once the project is completed. The general website dropdown is something that can be added to the list of Future Work projects for Website 2.0 (to be completed once more engineers are added to the project), but I believe that the most critical priority (and the focus of this project) should remain fixing the Python API docs as soon as possible.

[leezu]

Thanks @connorgoggins for clarifying.

@leezu after discussing with @sojiadeshina, we decided that it's ok to have the setup instructions (build from source) only apply to master

I disagree. The build from source instructions that are currently listed on the website do not work with the latest MXNet stable release. Users downloading one of our stable releases at https://mxnet.apache.org/get_started/download have no way to find instructions on how to compile and install it. (The source releases do not contain install instructions).

The general website dropdown is something that can be added to the list of Future Work projects for Website 2.0 (to be completed once more engineers are added to the project), but I believe that the most critical priority (and the focus of this project) should remain fixing the Python API docs as soon as possible.

Could you explain why building the website independently for every version and adding a static banner to the top on older versions of the website would be more effort than the solution proposed here? This may even take less resources?

My concern with the current proposal is that versioning for each part of the website needs to be handled independently, meaning that we will need resources to work on each part, which we are unlikely to find.

[leezu]

@aaronmarkham

I think a stopgap solution for the other languages is to generate them using the old branches - if they'll run, and post those as static artifacts of micro-sites that are browsable from a list. If they give any trouble - which they will as we try to go back to older and older versions, we leave those out for now.
To get the older stuff to run, we'll probably have to patch those old branches.

Getting websites for anything older than the latest stable release may not be necessary. I think it's reasonable to focus on 1.6, future 1.x releases, future 2.x release and the development version. Having a general versioning support for a limited number of versions seems preferable to having a Python-only versioning for all versions.

[connorgoggins]

@leezu building the entire website independently for every version would require a major amount of supplemental content. From the examples and descriptions on the features page, to the build from source instructions, to all the other language API docs, there is a great deal of additional content that needs to be added to each supported version of the new website. Much of this content does not currently exist, and would require time to develop and test before publishing to the website. Even considering just the other language API docs, each requires a specific build process, and there are errors with each of them that need to be resolved. Because this project has a maximum timeline of 4 weeks, it would not be possible for one engineer (myself) to add all of this additional content in addition to reworking the fundamental structure of the website to support multiple versions.

The stated objective of this project is to implement the minimum but most critical functionality that is broken today for any MXNet user coming to the website. This functionality has been decided to be the Python API docs, which >80% of our users rely on for MXNet development. The goal is to implement this functionality as soon as possible, since the current Python docs are considered broken. Pursuing the general dropdown instead would deviate from this objective, as the additional time it would require would delay the implementation of the Python docs fix for our users.

[aaronmarkham]

My suggestion is to just grab the old artifacts from the previous site builds here:
http://jenkins.mxnet-ci.amazon-ml.com/job/restricted-website-build/
This unzips to yield a versions folder that has artifacts for the following versions:

It's pretty much just C and Python and a little bit of Scala up to 1.2.1, then Scala's docs really come out, and then over time we get Clojure, Java and Julia. Perl is offsite. R wasn't generated in a pipeline until this new site. I could try to generate pdfs for old R versions if there was demand.
Anyways, for the C, Scala, Clojure, Java, and Julia microsites, we just grab those and deploy them as static assets.
If you want to fix C and split out C and C++ APIs, then we'll need to create some custom Doxyfiles for that and split the pipeline into two... maybe add a new tile for that. I'd leave that for later though.
For the Python versions, I'd focus on current stable, 1.7+ and master (2.x). If we want the old ones then just keep the old look and feel for when those get launched. Otherwise, there's a bunch of tech debt where you'd have to patch the old branches to make the docs build properly. Limited bang for the buck IMO.

[leezu]

Thanks @connorgoggins for clarifying.

My point is, the website on the 1.6 branch should currently not be broken (or easy to fix) and we can avoid it from breaking in the future. The website on the master branch is not broken either.
The Jenkins job publishing the website currently can be run for the 1.6 branch, 1.7 branch and master branch and we can enable it for more branches when new releases are made.
The Jenkins jobs can be modified to build https://mxnet.apache.org/v1.6/ for 1.6, https://mxnet.apache.org/v1.7, https://mxnet.apache.org/master/ respectively.

We can include older releases statically as per @aaronmarkham's suggestion.

In such case all that is needed would be a static banner or dropdown that contains links to the newer versions of the website. As an example, consider the banner at the top of https://cmake.org/cmake/help/v3.11/module/FetchContent.html .

This would solve the versioning issue for all parts of the website and should not exceed 4 weeks work. It may be even simpler than extending the Python docs as per your proposal. What do you think?

[connorgoggins]

@leezu ok, I think I understand. A static implementation for the older versions should be feasible within the scope of this project. I will move forward with implementing the general website version dropdown over the next four weeks, and will follow up if I run into any issues.

Thanks for your feedback!

[connorgoggins]

Update on progress so far:

• Fixed broken components of static artifacts for old versions (internal/external links, menus, etc.)
• Added missing supplemental content (missing tutorials, docs, etc.) to static artifacts for old versions
• Implemented working general version dropdown menu capable of switching between old artifacts
• Finished general version dropdown for master website (styling and functionality) - tested in browser w/inline changes to HTML/CSS, Jekyll build with changes passing on Jenkins

After I obtain the artifact of the full master website build with my changes from Jenkins, I will deploy the files on an EC2 instance with public access over a specific port. You will then be able to preview my changes and provide feedback.

@sandeep-krishnamurthy @aaronmarkham @szha @sojiadeshina @leezu

[szha]

Thanks for the update! I've been meaning to ask: what's the part in the website that requires a webserver that a static hosting solution cannot replace?

@aaronmarkham @ThomasDelteil feel free to chime in

[connorgoggins]

@szha thanks for your comment! The primary components that require a dynamic hosting solution are the API reference docs for each language in the latest versions (master, v1.6, and v1.x - future releases). For example, since we use Sphinx to generate the Python docs for the latest versions of the MXNet API, we need a build pipeline for the latest versions of the Python MXNet API that leverages Sphinx. We use other tools to dynamically build the docs for other languages (C++, Java, Julia, etc.), and they need to be called during the build process as well.

[aaronmarkham]

Thanks for the update! I've been meaning to ask: what's the part in the website that requires a webserver that a static hosting solution cannot replace?

@aaronmarkham @ThomasDelteil feel free to chime in

The Apache hosting is static. But, I think what you're looking for is the switch away from S3...
As far as s3 goes, it's not meant to be a web server and various things don't work, like assumed index pages in roots of folders which make for a lot of broken links and other problems that web devs would expect to work. Jekyll, for one, wouldn't work out of the box with s3... The friendly/vanity urls won't work.... Maybe with loads of tweaking you could get it to work. IDK.
I do miss the previews autogenerated with a PR's CI run. I think we could still do previews with s3 as long as cloudfront or something else sits in front of it.

[connorgoggins]

Preview of the new website with my changes is available here. Quip doc for feedback is available here. For non-Amazon contributors, here is a Google doc where you can share feedback.

[aaronmarkham]

Awesome progress.

• 1.6 is missing though...
• It's unfortunate that you get stuck in the older versions if you go back...Any way to inject a banner or some way of getting back to the main site?
• Can we get some no-index flag like a robots.txt in the old API sites, so old stuff isn't getting in google's search results? If you get stuck in an old version when coming from Google, with no way to get out... that's a bad experience. We used have that actually, so I really don't want to regress.

[connorgoggins]

@aaronmarkham thanks so much for your feedback!

• I should have mentioned, I'm putting together a build pipeline for 1.6 so we can support future changes. Consequently, 1.6 is currently not available in the preview, but will be available in the final version.
• There is a way to get back to the main site (and switch between old versions) - the dropdown that I added (see the vX.Y.Z menu in the nav bar for each older version). Maybe it would be easier to see if I changed the older dropdown to the same format as the new dropdown ("API Version" as the header)? The reason I had it in a different format is that many of the home pages for the old versions look similar, so I wanted users to immediately be able to tell which version they were on through the content of the homepage.
• We can do that, but just to be clear there is a way to go to master or other versions from each old version - through the version dropdown in the header

[aaronmarkham]

Ah I see what you did. I think it should be consistent... my preference would be to change "API Version" to be like what it looked like before. That way when you time travel, you have a familiar nav item and can get home.

[connorgoggins]

Another quick update: I have incorporated the feedback from @aaronmarkham into the latest version of the preview here (same URL). I decided to show the current version in the header for all versions (instead of "API Version" for master vs. the current version for the static artifacts). I believe this change makes the user interface more consistent across the artifacts and the master site, while also making it clear to users which version of the site they are currently using. Also happy to revert to a "Version (x.y.z)" or "Version (master)" format if that is preferable.

I also found that the tagged release for 1.6.0.rc0 and the branch for v1.6.x do not build - there are dependency issues associated with the build processes for Julia, Python, and R docs (see here for specific errors). Since each of these docs have different required dependencies and each is failing in a unique way, I decided to hold off on creating a dynamic pipeline for v1.6 until these issues are resolved. I have created a new Github issue to track progress on these build errors.

In the meantime, I have created a document to instruct users on how to add future releases to the new version of the website, explaining each step of the process from modifying the HTML/CSS of the source files to configuring Jenkins pipelines.

If we have consensus that the newly updated preview satisfies the goals of this project, I would like to move forward with the final steps of this project by submitting a PR with all of my changes and running the Jenkins production pipelines to publish the new website. If there is disagreement regarding my decision to not build the dynamic pipeline for v1.6, I'm more than happy to consider alternate paths forward.

[leezu]

Thank you @connorgoggins! Would it make sense to default to the last stable version (ie. v1.6)?
Are you going to backport the Jenkins fixes to v1.6 and deploy the dynamic pipeline later? Or will you only provide the static version of v1.6?

[connorgoggins]

@leezu thanks for your questions! Right now I'm inclined to only provide the static version of v1.6 rather than trying to develop fixes for all of the myriad issues w/Jenkins for 1.6, but the Github issue I've added should give future engineers a good starting point for identifying these issues and building out a solution.

[connorgoggins]

Update: a pull request with my changes to the master branch is now ready for review, and the website preview content has been refreshed. Once my PR is merged and the Jenkins prod pipeline for building/publishing the production website is triggered, my changes will be visible in the production website.

[connorgoggins]

Final update: the pull request has been merged, and the new MXNet production website has been deployed. It is currently available at https://mxnet.apache.org/. If you happen to find any bugs associated with the dropdown or the website content, please create a GitHub issue with the "Website" tag so the bug can be fixed.

[connorgoggins]

Thank you to all stakeholders, especially @sandeep-krishnamurthy, @aaronmarkham, @szha, @leezu, @sojiadeshina, and @marcoabreu, for your feedback and advice throughout the design and implementation process!

[sandeep-krishnamurthy]

Resolving as the feature is now available. Thanks @connorgoggins