Skip to content

Docs: Refer to the README.md in site/ for the docs #10402

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
nastra merged 1 commit into from
Jun 3, 2024
Merged

Docs: Refer to the README.md in site/ for the docs #10402

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 9 additions & 9 deletions site/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,6 @@ This subproject contains the [MkDocs projects](https://www.mkdocs.org/) that def
* Python >=3.9
* pip


## Usage

The directory structure in this repository aims to mimic the sitemap hierarchy of the website. This helps contributors find the source files needed when updating or adding new documentation. It's helpful to have some basic understanding of the MkDocs framework defaults.
Expand All @@ -35,7 +34,7 @@ In MkDocs, the [`docs_dir`](https://www.mkdocs.org/user-guide/configuration/#doc

### Iceberg docs layout

The static Iceberg website lives under the `/site` directory, while the versioned documentation lives under the `/docs` of the main Iceberg repository. The `/site/docs` directory is named that way to follow the [MkDocs convention](https://www.mkdocs.org/user-guide/configuration/#docs_dir). The `/docs` directory contains the current state of the versioned documentation with local revisions. Notice that the root `/site` and `/docs` just happened to share the same naming convention as MkDocs but does not correlate to the mkdocs
The static Iceberg website lives under the `/site` directory, while the versioned documentation lives under the `/docs` of the main Iceberg repository. The `/site/docs` directory is named that way to follow the [MkDocs convention](https://www.mkdocs.org/user-guide/configuration/#docs_dir). The `/docs` directory contains the current state of the versioned documentation with local revisions. Notice that the root `/site` and `/docs` just happened to share the same naming convention as MkDocs but does not correlate to the mkdocs.

The static Iceberg site pages are Markdown files that live at `/site/docs/*.md`. The versioned documentation are Markdown files that live at `/docs/docs/*.md` files. You may ask where the older versions of the docs and javadocs are, which is covered later in the build section.

Expand All @@ -58,11 +57,12 @@ The static Iceberg site pages are Markdown files that live at `/site/docs/*.md`.
├── mkdocs.yml
└── requirements.txt
```

### Building the versioned docs

The Iceberg versioned docs are committed in two [orphan](https://git-scm.com/docs/gitglossary#Documentation/gitglossary.txt-aiddeforphanaorphan) branches and mounted using [git worktree](https://git-scm.com/docs/git-worktree) at build time:

1. [`docs`](https://github.com/apache/iceberg/tree/docs) - contains the state of the documenation source files (`/docs`) during release. These versions are mounted at the `/site/docs/docs/<version>` directory at build time.
1. [`docs`](https://github.com/apache/iceberg/tree/docs) - contains the state of the documentation source files (`/docs`) during release. These versions are mounted at the `/site/docs/docs/<version>` directory at build time.
1. [`javadoc`](https://github.com/apache/iceberg/tree/javadoc) - contains prior statically generated versions of the javadocs mounted at `/site/docs/javadoc/<version>` directory at build time.

The `latest` version, is a soft link to the most recent [semver version](https://semver.org/) in the `docs` branch. The `nightly` version, is a soft link to the current local state of the `/docs` markdown files.
Expand Down Expand Up @@ -104,20 +104,20 @@ This step will generate the staged source code which blends into the original so

<!-- markdown-link-check-disable-next-line -->
To run this, run the `serve` recipe, which runs the `build` recipe and calls `mkdocs serve`. This will run locally at <http://localhost:8000>.
```
```sh
make serve
```

To clear all build files, run `clean`.
```
```sh
make clean
```

#### Offline mode

One of the great advantages to the MkDocs material plugin is the [offline feature](https://squidfunk.github.io/mkdocs-material/plugins/offline). You can view the Iceberg docs without the need of a server. To enable OFFLINE builds, add theOFFLINE environment variable to either `build` or `serve` recipes.

```
```sh
make build OFFLINE=true
```

Expand All @@ -132,11 +132,11 @@ Deploying the docs is a two step process:
> The `make release` directive is currently unavailable as we wanted to discuss the best way forward on how or if we should automate the release. It involves taking an existing snapshot of the versioned documentation, and potentially automerging the [`docs` branch](https://github.com/apache/iceberg/tree/docs) and the [`javadoc` branch](https://github.com/apache/iceberg/tree/javadoc) which are independent from the `main` branch. Once this is complete, we can create a pull request with an offline build of the documentation to verify everything renders correctly, and then have the release manager merge that PR to finalize the docs release. So the real process would be manually invoking a docs release action, then merging a pull request.

1. Release a new version by copying the current `/docs` directory to a new version directory in the `docs` branch and a new javadoc build in the `javadoc` branch.
```
```sh
make release ICEBERG_VERSION=${ICEBERG_VERSION}
```
1. Build and push the generated site to `asf-site`.
```
```sh
make deploy
```

Expand Down Expand Up @@ -166,7 +166,7 @@ As mentioned in the MkDocs section, when you build MkDocs `mkdocs build`, MkDocs
└── mkdocs.yml
```

Since there are multiple MkDocs projects that build independently, links between them will initially cause a warning when building. This occurs when `mkdocs-monorepo-plugin` compiles, it must first build the versioned documentation sites before aggregating the top-level site with the generated. Due to the delayed aggregation of subdocs of `mkdocs-monorepo-plugin` there may be warnings that display for the versioned docs that compile without being able to reference documentation it expects outside of the immediate poject due to being off by one or more directories. In other words, if the relative linking required doesn't mirror the directory layout on disk, these errors will occur. The only place this occurs now is with the nav link to javadoc. For more information, refer to: <https://github.com/backstage/mkdocs-monorepo-plugin#usage>
Since there are multiple MkDocs projects that build independently, links between them will initially cause a warning when building. This occurs when `mkdocs-monorepo-plugin` compiles, it must first build the versioned documentation sites before aggregating the top-level site with the generated. Due to the delayed aggregation of subdocs of `mkdocs-monorepo-plugin` there may be warnings that display for the versioned docs that compile without being able to reference documentation it expects outside the immediate project due to being off by one or more directories. In other words, if the relative linking required doesn't mirror the directory layout on disk, these errors will occur. The only place this occurs now is with the nav link to javadoc. For more information, refer to: <https://github.com/backstage/mkdocs-monorepo-plugin#usage>

To ensure the links work, you may use linkchecker to traverse the links on the livesite when you're running locally. This may eventually be used as part of the build unless a more suitable static solution is found.

Expand Down
70 changes: 1 addition & 69 deletions site/docs/how-to-release.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -319,75 +319,7 @@ Create a PR in the `iceberg` repo to make revapi run on the new release. For an

### Documentation Release

Documentation needs to be updated as a part of an Iceberg release after a release candidate is passed.
The commands described below assume you are in a directory containing a local clone of the `iceberg-docs`
repository and `iceberg` repository. Adjust the commands accordingly if it is not the case. Note that all
changes in `iceberg` need to happen against the `master` branch and changes in `iceberg-docs` need to happen
against the `main` branch.

#### Common documentation update

1. To start the release process, run the following steps in the `iceberg-docs` repository to copy docs over:
```shell
cp -r ../iceberg/format/* ../iceberg-docs/landing-page/content/common/
```
2. Change into the `iceberg-docs` repository and create a branch.
```shell
cd ../iceberg-docs
git checkout -b <BRANCH NAME>
```
3. Commit, push, and open a PR against the `iceberg-docs` repo (`<BRANCH NAME>` -> `main`)

#### Versioned documentation update

Once the common docs changes have been merged into `main`, the next step is to update the versioned docs.

1. In the `iceberg-docs` repository, cut a new branch using the version number as the branch name
```shell
cd ../iceberg-docs
git checkout -b <VERSION>
git push --set-upstream apache <VERSION>
```
2. Copy the versioned docs from the `iceberg` repo into the `iceberg-docs` repo
```shell
rm -rf ../iceberg-docs/docs/content
cp -r ../iceberg/docs ../iceberg-docs/docs/content
```
3. Commit the changes and open a PR against the `<VERSION>` branch in the `iceberg-docs` repo

#### Javadoc update

In the `iceberg` repository, generate the javadoc for your release and copy it to the `javadoc` folder in `iceberg-docs` repo:
```shell
cd ../iceberg
./gradlew refreshJavadoc
rm -rf ../iceberg-docs/javadoc
cp -r site/docs/javadoc/<VERSION NUMBER> ../iceberg-docs/javadoc
```

This resulted changes in `iceberg-docs` should be approved in a separate PR.

#### Update the latest branch

Since `main` is currently the same as the version branch, one needs to rebase `latest` branch against `main`:

```shell
git checkout latest
git rebase main
git push apache latest
```

#### Set latest version in iceberg-docs repo

The last step is to update the `main` branch in `iceberg-docs` to set the latest version.
A PR needs to be published in the `iceberg-docs` repository with the following changes:
1. Update variable `latestVersions.iceberg` to the new release version in `landing-page/config.toml`
2. Update variable `latestVersions.iceberg` to the new release version and
`versions.nessie` to the version of `org.projectnessie.nessie:*` from [mkdocs.yml](https://github.com/apache/iceberg/blob/main/site/mkdocs.yml) in `docs/config.toml`
3. Update list `versions` with the new release in `landing-page/config.toml`
4. Update list `versions` with the new release in `docs/config.toml`
5. Mark the current latest release notes to past releases under `landing-page/content/common/release-notes.md`
6. Add release notes for the new release version in `landing-page/content/common/release-notes.md`
Please follow the instructions on the GitHub repository in the [`README.md` in the `site/`](https://github.com/apache/iceberg/tree/main/site) directory.

# How to Verify a Release

Expand Down