Skip to content

Add local nightly build to test current docs changes#9943

Merged
nastra merged 3 commits intoapache:mainfrom
bitsondatadev:add-nightly-local
Mar 22, 2024
Merged

Add local nightly build to test current docs changes#9943
nastra merged 3 commits intoapache:mainfrom
bitsondatadev:add-nightly-local

Conversation

@bitsondatadev
Copy link
Collaborator

Add local nightly build to test current docs changes

@github-actions github-actions bot added the docs label Mar 13, 2024
@bitsondatadev bitsondatadev self-assigned this Mar 13, 2024
@bitsondatadev
Copy link
Collaborator Author

CC: @manuzhang I can't add you as a reviewer.

@manuzhang
Copy link
Member

Thanks @bitsondatadev, I will test locally later today.

mkdir docs/docs/nightly/

# Create symbolic links and copy configuration files for the 'nightly' documentation
ln -s "../../../../docs/docs/" docs/docs/nightly/docs
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I verified that this works locally, but will this work on the docs branch?


# Create symbolic links and copy configuration files for the 'nightly' documentation
ln -s "../../../../docs/docs/" docs/docs/nightly/docs
cp "../docs/mkdocs.yml" docs/docs/nightly/
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we make this a symlink as well? Otherwise you won't see changes made to the navigation

echo " --> create nightly"

# Remove any existing 'nightly' directory and recreate it
rm -rf docs/docs/nightly/
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is nightly not a symlink?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both nightly and latest both symlink the relevant docs directory, and copy the mkdocs.yml to update 1.5.0 to latest and similar with nightly.

/site/docs/docs/
            |
            |_nightly 
            |   |
            |   |_docs --symlink-->  /docs/docs/
            |   |_mkdocs.yaml (cp from /docs/docs/mkdocs.yaml & updated to "nightly")
            |
            |_latest
            |   |
            |   |_docs --symlink-->  ../1.5.0/docs/
            |   |_mkdocs.yaml (cp from ../1.5.0/mkdocs.yaml & updated to "latest")
            |
            |_1.5.0
            |
            |_<older_versions>

- Spark: spark-quickstart.md
- Hive: hive-quickstart.md
- Docs:
- nightly: '!include docs/docs/nightly/mkdocs.yml'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nightly and latest look similar and a bit confusing. I think we can merge latest and 1.5.0 to 1.5.0 (latest) or 1.5.0 (stable).

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm open to it, I think this was what we went with in the last discussion, but happy to bring this up. See here: #9943 (comment)

CC: @danielcweeks @RussellSpitzer any thoughts here?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The label isn't what matters the most. What we care about are docs under docs/latest/... URLs. We can't really combine those with, for example, the 1.5.0 docs because we want to have the ability to permalink to a specific version if that's what you're viewing.

The Iceberg versioned docs are committed in two [orphan](https://git-scm.com/docs/gitglossary#Documentation/gitglossary.txt-aiddeforphanaorphan) branches and mounted using [git worktree](https://git-scm.com/docs/git-worktree) at build time:

1. [`docs`](https://github.com/apache/iceberg/tree/docs) - contains the state of the documenation source files (`/docs`) during release. These versions are mounted at the `/site/docs/docs/<version>` directory at build time.
1. [`javadoc`](https://github.com/apache/iceberg/tree/javadoc) - contains prior statically generated versions of the javadocs mounted at `/site/docs/javadoc/<version>` directory at build time.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 -> 2

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: extra space before build time

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 -> 2

I prefer the markdown convention of using 1 all the time. This still renders ascending numbers in the output, and doesn't require updates to numbered lists to be of concern during reviews and when updating the docs.

Copy link
Contributor

@rdblue rdblue left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While I agree that it would be better to use symlinks, I think we want to get this out instead of refactoring. The non-symlink version is how it currently works so we don't need to change that now. I'm +1 for merging this now and improving it over time.

@nastra
Copy link
Contributor

nastra commented Mar 22, 2024

While I agree that it would be better to use symlinks, I think we want to get this out instead of refactoring.

I agree, let's get this in and we can improve and switch to using symlinks. @bitsondatadev could you open an issue for that please?

@nastra nastra merged commit c9795fd into apache:main Mar 22, 2024
zachdisc pushed a commit to zachdisc/iceberg that referenced this pull request Dec 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants