Skip to content

HDDS-4889. Add simple CI check for docs #2156

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
elek merged 5 commits into from
Apr 26, 2021
Merged

HDDS-4889. Add simple CI check for docs #2156

elek merged 5 commits into from
Apr 26, 2021

Conversation

@adoroszlai
Copy link
Contributor

@adoroszlai adoroszlai commented Apr 14, 2021

What changes were proposed in this pull request?

Currently docs are implicitly built in integration checks. (They used to be built in few other checks, too, but those no longer have Hugo available.) If there is a syntax error in the docs, all of these fail due to the same cause. Also, one needs to dig into build logs to find any problem, because Hugo errors are not collected.

This PR adds a separate check for building hadoop-hdds/docs with Hugo, collecting summary of Hugo-specific failures in the regular way.

https://issues.apache.org/jira/browse/HDDS-4889

How was this patch tested?

> Run hadoop-ozone/dev-support/checks/docs.sh
...
Installed hugo in /home/runner/work/hadoop-ozone/hadoop-ozone/.dev-tools/hugo
~/work/hadoop-ozone/hadoop-ozone
Start building sites … 

                   | EN | ZH  
-------------------+----+-----
  Pages            | 81 | 49  
  Paginator pages  |  0 |  0  
  Non-page files   | 22 | 22  
  Static files     | 26 | 26  
  Processed images |  0 |  0  
  Aliases          |  1 |  0  
  Sitemaps         |  2 |  1  
  Cleaned          |  0 |  0  

Total in 188 ms

https://github.com/adoroszlai/hadoop-ozone/runs/2344467280#step:4:1

Introduced reference to non-existing page locally, verified output:

$ hadoop-ozone/dev-support/checks/docs.sh
...
Error: Error building site: logged 1 error(s)

$ cat target/docs/summary.txt
ERROR 2021/04/14 17:30:42 [en] REF_NOT_FOUND: Ref "asdfJavaApi.md": "hadoop-hdds/docs/content/_index.md:33:138": page not found

$ cat target/docs/failures
1

@adoroszlai adoroszlai self-assigned this Apr 14, 2021
elek
elek approved these changes Apr 26, 2021
View reviewed changes
Copy link
Member

@elek elek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, thanks for the new check @adoroszlai

@elek elek merged commit 6af26ca into apache:master Apr 26, 2021
@adoroszlai adoroszlai deleted the HDDS-4889 branch April 26, 2021 08:30
@adoroszlai
Copy link
Contributor Author

adoroszlai commented Apr 26, 2021

Thanks @elek for reviewing and committing it.

errose28 added a commit to errose28/ozone that referenced this pull request May 4, 2021
@errose28
Merge remote-tracking branch 'upstream/master' into HDDS-3698-nonroll…
253cc55 
…ing-upgrade-master-merge2

* upstream/master: (56 commits)
  HDDS-2212. Genconf tool should generate config files for secure clust… (apache#1788)
  HDDS-5166. Remove duplicate assignment of OZONE_OPTS for freon and sh (apache#2195)
  Revert "HDDS-5144. Create github check to alert when dependency tree is changed (apache#2177)"
  HDDS-4983. Display key offset for each block in command key info (apache#2051)
  HDDS-5144. Create github check to alert when dependency tree is changed (apache#2177)
  HDDS-4585. Support bucket acl operation in S3g (apache#1701)
  HDDS-5153. Decommissioning a dead node should complete immediately (apache#2190)
  HDDS-5147. Intermittent test failure in TestContainerDeletionChoosingPolicy#testRandomChoosingPolicy (apache#2188)
  HDDS-5152. Fix Suggested leader in Client. (apache#2189)
  HDDS-5148. Bump ratis version to 2.1.0-ff8aa66-SNAPSHOT (apache#2184)
  HDDS-4515. Datanodes should be able to persist and load CRL (apache#2181)
  HDDS-5060. [SCM HA Security] Make InterSCM grpc channel secure. (apache#2187)
  HDDS-5051. Ensure failover to suggested leader if any for NotLeaderException. (apache#2141)
  HDDS-5127. Fix getServiceList when SCM HA is enabled (apache#2173)
  HDDS-4889. Add simple CI check for docs (apache#2156)
  HDDS-5131. Use timeout in github actions (apache#2176)
  HDDS-5103. Fix Install Snapshot Mechanism in SCMStateMachine. (apache#2155)
  HDDS-5124. Use OzoneConsts.OZONE_TIME_ZONE instead of "GMT" (apache#2166)
  HDDS-5047. Refactor Pipeline to use ReplicationConfig instead of factor/type (apache#2096)
  HDDS-5083. Bump version of common-compress (apache#2139)
  ...

Conflicts:
	hadoop-hdds/common/pom.xml
	hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/OzoneConfigKeys.java
	hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/OzoneConsts.java
	hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/HddsDatanodeService.java
	hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/SCMNodeManager.java
	hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/PipelineManager.java
	hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/server/SCMStorageConfig.java
	hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/server/StorageContainerManager.java
	hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/node/TestSCMNodeManager.java
	hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/MiniOzoneClusterImpl.java
	hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OMStorage.java
	hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OzoneManager.java
	hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/scm/ReconStorageContainerManagerFacade.java
errose28 added a commit to errose28/ozone that referenced this pull request May 13, 2021
@errose28
Merge branch 'HDDS-3698-nonrolling-upgrade' into HDDS-5138-admin-chec…
080019b 
…k-in-auth

* HDDS-3698-nonrolling-upgrade: (57 commits)
  Fix compilation errors afte merge Update javassist in recon pom Fix changes introduced in merge that failed TestSCMNodeManager upgrade tests Fix checkstyle Fix intermittent test failure TestSCMNodeManager#testSetNodeOpStateAndCommandFired after merge Skip scm init default layout version in TestOzoneConfigurationFields
  HDDS-2212. Genconf tool should generate config files for secure clust… (apache#1788)
  HDDS-5166. Remove duplicate assignment of OZONE_OPTS for freon and sh (apache#2195)
  Revert "HDDS-5144. Create github check to alert when dependency tree is changed (apache#2177)"
  HDDS-4983. Display key offset for each block in command key info (apache#2051)
  HDDS-5144. Create github check to alert when dependency tree is changed (apache#2177)
  HDDS-4585. Support bucket acl operation in S3g (apache#1701)
  HDDS-5153. Decommissioning a dead node should complete immediately (apache#2190)
  HDDS-5147. Intermittent test failure in TestContainerDeletionChoosingPolicy#testRandomChoosingPolicy (apache#2188)
  HDDS-5152. Fix Suggested leader in Client. (apache#2189)
  HDDS-5148. Bump ratis version to 2.1.0-ff8aa66-SNAPSHOT (apache#2184)
  HDDS-4515. Datanodes should be able to persist and load CRL (apache#2181)
  HDDS-5060. [SCM HA Security] Make InterSCM grpc channel secure. (apache#2187)
  HDDS-5051. Ensure failover to suggested leader if any for NotLeaderException. (apache#2141)
  HDDS-5127. Fix getServiceList when SCM HA is enabled (apache#2173)
  HDDS-4889. Add simple CI check for docs (apache#2156)
  HDDS-5131. Use timeout in github actions (apache#2176)
  HDDS-5103. Fix Install Snapshot Mechanism in SCMStateMachine. (apache#2155)
  HDDS-5124. Use OzoneConsts.OZONE_TIME_ZONE instead of "GMT" (apache#2166)
  HDDS-5047. Refactor Pipeline to use ReplicationConfig instead of factor/type (apache#2096)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@elek elek elek approved these changes

Assignees

@adoroszlai adoroszlai

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@adoroszlai @elek