From 8983495ebddf40fae1c02d110169d03f1c37220e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?M=C3=A1rton=20Elek?= Date: Fri, 7 Jun 2019 11:27:36 +0200 Subject: [PATCH 1/4] HDDS-1659. Define the process to add proposal/design docs to the Ozone subproject --- hadoop-hdds/docs/archetypes/design.md | 60 ++++++++++++ hadoop-hdds/docs/config.yaml | 7 +- .../design/ozone-enhancement-proposals.md | 97 +++++++++++++++++++ .../ozonedoc/layouts/_default/baseof.html | 41 ++++++++ .../ozonedoc/layouts/_default/section.html | 54 +++++++++++ .../ozonedoc/layouts/_default/single.html | 23 +++-- .../ozonedoc/layouts/design/section.html | 66 +++++++++++++ .../ozonedoc/layouts/design/single.html | 47 +++++++++ .../docs/themes/ozonedoc/layouts/index.html | 2 - .../ozonedoc/layouts/partials/header.html | 4 +- .../ozonedoc/layouts/partials/sidebar.html | 14 +-- 11 files changed, 394 insertions(+), 21 deletions(-) create mode 100644 hadoop-hdds/docs/archetypes/design.md create mode 100644 hadoop-hdds/docs/content/design/ozone-enhancement-proposals.md create mode 100644 hadoop-hdds/docs/themes/ozonedoc/layouts/_default/baseof.html create mode 100644 hadoop-hdds/docs/themes/ozonedoc/layouts/_default/section.html create mode 100644 hadoop-hdds/docs/themes/ozonedoc/layouts/design/section.html create mode 100644 hadoop-hdds/docs/themes/ozonedoc/layouts/design/single.html diff --git a/hadoop-hdds/docs/archetypes/design.md b/hadoop-hdds/docs/archetypes/design.md new file mode 100644 index 0000000000000..59dc5f2c3778e --- /dev/null +++ b/hadoop-hdds/docs/archetypes/design.md @@ -0,0 +1,60 @@ +--- +title: "{{ replace .Name "-" " " | title }}" +menu: main +jira: HDDS-XXX +summary: One sentence summary. Will be displayed at the main design doc table. +status: current +author: Your Name + +--- + + + +This the proposed template to document any proposal. It's recommended but not required the use exactly the some structure. Some proposal may require different structure, but we need the following information. + +## Summary + +> Give a one sentence summary, like the jira title. It will be displayed on the documentation page. Should be enough to understand + +## Problem statement (Motivation / Abstract) + +> What is the problem and how would you solve it? Think about an abstract of a paper: one paragraph overview. Why will the world better with this change? + +## Non-goals + + > Very important to define what is outside of the scope of this proposal + +## Technical Description (Architecture and implementation details) + + > Explain the problem in more details. How can it be reproduced? What is the current solution? What is the limitation of the current solution? + + > How the new proposed solution would solve the problem? Architectural design. + + > Implementation details. What should be changed in the code. Is it a huge change? Do we need to change wire protocol? Backward compatibility? + +## Alternatives + + > What are the other alternatives you considered and why do yoy prefer the proposed solution The goal of this section is to help people understand why this is the best solution now, and also to prevent churn in the future when old alternatives are reconsidered. + +Note: In some cases 4/5 can be combined. For example if you have multiple proposals, the first version may include multiple solutions. At the end ot the discussion we can move the alternatives to 5. and explain why the community is decided to use the selected option. + +## Implementation plan + + > Planning to implement the feature. Estimated size of the work? Do we need feature branch? Any migration plan, dependency? If it's not a big new feature it can be one sentence or optiona;. + +## References \ No newline at end of file diff --git a/hadoop-hdds/docs/config.yaml b/hadoop-hdds/docs/config.yaml index 2db1147cd5a54..6f64dbb89ba67 100644 --- a/hadoop-hdds/docs/config.yaml +++ b/hadoop-hdds/docs/config.yaml @@ -43,4 +43,9 @@ menu: name: Recipes title: Recipes url: prometheus.html - weight: 4 \ No newline at end of file + weight: 4 + - identifier: Design docs + name: Design docs + title: Design docs + url: design.html + weight: 30 diff --git a/hadoop-hdds/docs/content/design/ozone-enhancement-proposals.md b/hadoop-hdds/docs/content/design/ozone-enhancement-proposals.md new file mode 100644 index 0000000000000..78b92dbf9d09a --- /dev/null +++ b/hadoop-hdds/docs/content/design/ozone-enhancement-proposals.md @@ -0,0 +1,97 @@ +--- +title: Ozone Enhancement Proposals +summary: Definition of the process to share new technical proposals with the Ozone community. +date: 2019-06-07 +jira: HDDS-1659 +status: current +author: Anu Enginner, Marton Elek +--- + +## Problem statement + +Some of the biggers features requires well defined plans before the implementation. Until now it was managed by uploading PDF design docs to selected JIRA. There are multiple problems with the current practice. + + 1. There is no easy way to find existing up-to-date and outdated design docs. + 2. Design docs usually have better description of the problem that the user docs + 3. We need better tools to discuss the design docs in the development phase of the doc + +We propose to follow the same process what we have now, but instead of uploading a PDF to the JIRA, create a PR to merge the proposal document to the documentation project. + +## Non-goals + + * Modify the existing workflow or approval process + * Migrate existing documents + * Make it harder to create design docs (it should be easy to support the creation of proposals for any kind of tasks) + +## Proposed solution + + * Open a dedicated Jira (`HDDS-*` but with specific component) + * Use standard name prefix in the jira (easy to filter on the mailing list) `[OEP] + * Create a PR to merge the design doc (markdown) to `hadoop-hdds/docs/content/proposal` (will be part of the docs) + * Discuss it as before (lazy consesus, except if somebody calls for a real vote) + * Design docs can be updated according to the changes during the implementation + +## Document template + +This the proposed template to document any proposal. It's recommended but not required the use exactly the some structure. Some proposal may require different structure, but we need the following information. + +1. Summary + +> Give a one sentence summary, like the jira title. It will be displayed on the documentation page. Should be enough to understand + +2. Problem statement (Motivation / Abstract) + +> What is the problem and how would you solve it? Think about an abstract of a paper: one paragraph overview. Why will the world better with this change? + +3. Non-goals + + > Very important to define what is outside of the scope of this proposal + +4. Technical Description (Architecture and implementation details) + + > Explain the problem in more details. How can it be reproduced? What is the current solution? What is the limitation of the current solution? + + > How the new proposed solution would solve the problem? Architectural design. + + > Implementation details. What should be changed in the code. Is it a huge change? Do we need to change wire protocol? Backward compatibility? + +5. Alternatives + + > What are the other alternatives you considered and why do yoy prefer the proposed solution The goal of this section is to help people understand why this is the best solution now, and also to prevent churn in the future when old alternatives are reconsidered. + +Note: In some cases 4/5 can be combined. For example if you have multiple proposals, the first version may include multiple solutions. At the end ot the discussion we can move the alternatives to 5. and explain why the community is decided to use the selected option. + +6. Plan + + > Planning to implement the feature. Estimated size of the work? Do we need feature branch? Any migration plan, dependency? If it's not a big new feature it can be one sentence or optional. + +7. References + +## Workflows form other projects + +There are similar process in other open source projects. This document and the template is inspired by the following projects: + + * [Apache Kafka Improvement Proposals](https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals) + * [Apache Spark Project Improvement Proposals](https://spark.apache.org/improvement-proposals.html) + * [Kubernetes Enhancement Proposals](https://github.com/kubernetes/enhancements/tree/master/keps) + +Short summary if the porcesses: + +__Kafka__ process: + + * Create wiki page + * Start discussion on mail thread + * Vote on mail thread + +__Spark__ process: + + * Create JIRA (dedicated label) + * Discuss on the jira page + * Vote on dev list + +*Kubernetes*: + + * Deditaced git repository + * KEPs are committed to the repo + * Well defined approval process managed by SIGs (KEPs are assigned to SIGs) + diff --git a/hadoop-hdds/docs/themes/ozonedoc/layouts/_default/baseof.html b/hadoop-hdds/docs/themes/ozonedoc/layouts/_default/baseof.html new file mode 100644 index 0000000000000..c46f829cbf085 --- /dev/null +++ b/hadoop-hdds/docs/themes/ozonedoc/layouts/_default/baseof.html @@ -0,0 +1,41 @@ + +{{ partial "header.html" . }} + + + +{{ partial "navbar.html" . }} + +
+
+ {{ partial "sidebar.html" . }} +
+ +
+ {{ block "main" .}} + {{end}} +
+ +
+
+
+ + +{{ partial "footer.html" . }} + + + diff --git a/hadoop-hdds/docs/themes/ozonedoc/layouts/_default/section.html b/hadoop-hdds/docs/themes/ozonedoc/layouts/_default/section.html new file mode 100644 index 0000000000000..b138137bc7a92 --- /dev/null +++ b/hadoop-hdds/docs/themes/ozonedoc/layouts/_default/section.html @@ -0,0 +1,54 @@ + +{{ partial "header.html" . }} + + + +{{ partial "navbar.html" . }} + +
+
+ {{ partial "sidebar.html" . }} +
+ +
+ {{ .Content }} + + + + + + + + {{ range .Paginator.Pages }} + + + + {{ end }} + +
Title
{{.Title}}
+
+ +
+
+
+ + +{{ partial "footer.html" . }} + + + diff --git a/hadoop-hdds/docs/themes/ozonedoc/layouts/_default/single.html b/hadoop-hdds/docs/themes/ozonedoc/layouts/_default/single.html index ca2e1c4a65e00..3ba584bfd8936 100644 --- a/hadoop-hdds/docs/themes/ozonedoc/layouts/_default/single.html +++ b/hadoop-hdds/docs/themes/ozonedoc/layouts/_default/single.html @@ -16,23 +16,28 @@ --> {{ partial "header.html" . }} - + {{ partial "navbar.html" . }} -
-
+
+
{{ partial "sidebar.html" . }}
-

{{ .Title }}

-
- {{ .Content }} -
+ +
+

{{ .Title }}

+
+ {{ .Content }} +
+
+
-
+
+
{{ partial "footer.html" . }} - + diff --git a/hadoop-hdds/docs/themes/ozonedoc/layouts/design/section.html b/hadoop-hdds/docs/themes/ozonedoc/layouts/design/section.html new file mode 100644 index 0000000000000..83e0875089fb4 --- /dev/null +++ b/hadoop-hdds/docs/themes/ozonedoc/layouts/design/section.html @@ -0,0 +1,66 @@ + +{{ partial "header.html" . }} + + + +{{ partial "navbar.html" . }} + +
+
+ {{ partial "sidebar.html" . }} +
+ +
+
+

Available design docs

+ {{ .Content }} + + + + + + + + + + + {{ range .Paginator.Pages }} + + + + + + + {{ end }} + +
JIRATitleSummaryStatus
+ {{.Params.jira}} + {{.Title}}{{.Summary}}{{.Params.status}}
+
+
+ +
+
+
+ + +{{ partial "footer.html" . }} + + + + diff --git a/hadoop-hdds/docs/themes/ozonedoc/layouts/design/single.html b/hadoop-hdds/docs/themes/ozonedoc/layouts/design/single.html new file mode 100644 index 0000000000000..f92f226ec7950 --- /dev/null +++ b/hadoop-hdds/docs/themes/ozonedoc/layouts/design/single.html @@ -0,0 +1,47 @@ + +{{ partial "header.html" . }} + + +{{ partial "navbar.html" . }} +
+
+ {{ partial "sidebar.html" . }} +
+
+

[{{.Params.jira}}] {{ .Title }} ({{.Params.status}})

+
Authors: {{.Params.author}}
{{.Params.date | dateFormat "2006-01-02"}}
+

 

+ +
+
Summary
+
+ {{.Summary}} +
+
+ + {{ .Content }} +
+ +
+
+
+ +{{ partial "footer.html" . }} + + + diff --git a/hadoop-hdds/docs/themes/ozonedoc/layouts/index.html b/hadoop-hdds/docs/themes/ozonedoc/layouts/index.html index 17f0246abf403..e57edbe6e4b1b 100644 --- a/hadoop-hdds/docs/themes/ozonedoc/layouts/index.html +++ b/hadoop-hdds/docs/themes/ozonedoc/layouts/index.html @@ -25,8 +25,6 @@ {{ partial "sidebar.html" . }}
{{ .Content }} - -
diff --git a/hadoop-hdds/docs/themes/ozonedoc/layouts/partials/header.html b/hadoop-hdds/docs/themes/ozonedoc/layouts/partials/header.html index 35ba4c8042f94..6fc364420c1fb 100644 --- a/hadoop-hdds/docs/themes/ozonedoc/layouts/partials/header.html +++ b/hadoop-hdds/docs/themes/ozonedoc/layouts/partials/header.html @@ -26,9 +26,9 @@ Documentation for Apache Hadoop Ozone - + - + diff --git a/hadoop-hdds/docs/themes/ozonedoc/layouts/partials/sidebar.html b/hadoop-hdds/docs/themes/ozonedoc/layouts/partials/sidebar.html index 72b9cfc95c071..275d1f4f4069b 100644 --- a/hadoop-hdds/docs/themes/ozonedoc/layouts/partials/sidebar.html +++ b/hadoop-hdds/docs/themes/ozonedoc/layouts/partials/sidebar.html @@ -15,13 +15,13 @@ limitations under the License. --> - -{{ partial "footer.html" . }} - - - diff --git a/hadoop-hdds/docs/themes/ozonedoc/layouts/_default/section.html b/hadoop-hdds/docs/themes/ozonedoc/layouts/_default/section.html deleted file mode 100644 index b138137bc7a92..0000000000000 --- a/hadoop-hdds/docs/themes/ozonedoc/layouts/_default/section.html +++ /dev/null @@ -1,54 +0,0 @@ - -{{ partial "header.html" . }} - - - -{{ partial "navbar.html" . }} - -
-
- {{ partial "sidebar.html" . }} -
- -
- {{ .Content }} - - - - - - - - {{ range .Paginator.Pages }} - - - - {{ end }} - -
Title
{{.Title}}
-
- -
-
-
- - -{{ partial "footer.html" . }} - - - diff --git a/hadoop-hdds/docs/themes/ozonedoc/layouts/_default/single.html b/hadoop-hdds/docs/themes/ozonedoc/layouts/_default/single.html index 3ba584bfd8936..ca2e1c4a65e00 100644 --- a/hadoop-hdds/docs/themes/ozonedoc/layouts/_default/single.html +++ b/hadoop-hdds/docs/themes/ozonedoc/layouts/_default/single.html @@ -16,28 +16,23 @@ --> {{ partial "header.html" . }} - + {{ partial "navbar.html" . }} -
-
+
+
{{ partial "sidebar.html" . }}
- -
-

{{ .Title }}

-
- {{ .Content }} -
-
- +

{{ .Title }}

+
+ {{ .Content }} +
+
-
-
{{ partial "footer.html" . }} - + diff --git a/hadoop-hdds/docs/themes/ozonedoc/layouts/design/section.html b/hadoop-hdds/docs/themes/ozonedoc/layouts/design/section.html deleted file mode 100644 index 83e0875089fb4..0000000000000 --- a/hadoop-hdds/docs/themes/ozonedoc/layouts/design/section.html +++ /dev/null @@ -1,66 +0,0 @@ - -{{ partial "header.html" . }} - - - -{{ partial "navbar.html" . }} - -
-
- {{ partial "sidebar.html" . }} -
- -
-
-

Available design docs

- {{ .Content }} - - - - - - - - - - - {{ range .Paginator.Pages }} - - - - - - - {{ end }} - -
JIRATitleSummaryStatus
- {{.Params.jira}} - {{.Title}}{{.Summary}}{{.Params.status}}
-
-
- -
-
-
- - -{{ partial "footer.html" . }} - - - - diff --git a/hadoop-hdds/docs/themes/ozonedoc/layouts/design/single.html b/hadoop-hdds/docs/themes/ozonedoc/layouts/design/single.html deleted file mode 100644 index f92f226ec7950..0000000000000 --- a/hadoop-hdds/docs/themes/ozonedoc/layouts/design/single.html +++ /dev/null @@ -1,47 +0,0 @@ - -{{ partial "header.html" . }} - - -{{ partial "navbar.html" . }} -
-
- {{ partial "sidebar.html" . }} -
-
-

[{{.Params.jira}}] {{ .Title }} ({{.Params.status}})

-
Authors: {{.Params.author}}
{{.Params.date | dateFormat "2006-01-02"}}
-

 

- -
-
Summary
-
- {{.Summary}} -
-
- - {{ .Content }} -
- -
-
-
- -{{ partial "footer.html" . }} - - - diff --git a/hadoop-hdds/docs/themes/ozonedoc/layouts/index.html b/hadoop-hdds/docs/themes/ozonedoc/layouts/index.html index e57edbe6e4b1b..17f0246abf403 100644 --- a/hadoop-hdds/docs/themes/ozonedoc/layouts/index.html +++ b/hadoop-hdds/docs/themes/ozonedoc/layouts/index.html @@ -25,6 +25,8 @@ {{ partial "sidebar.html" . }}
{{ .Content }} + +
diff --git a/hadoop-hdds/docs/themes/ozonedoc/layouts/partials/header.html b/hadoop-hdds/docs/themes/ozonedoc/layouts/partials/header.html index 6fc364420c1fb..35ba4c8042f94 100644 --- a/hadoop-hdds/docs/themes/ozonedoc/layouts/partials/header.html +++ b/hadoop-hdds/docs/themes/ozonedoc/layouts/partials/header.html @@ -26,9 +26,9 @@ Documentation for Apache Hadoop Ozone - + - + diff --git a/hadoop-hdds/docs/themes/ozonedoc/layouts/partials/sidebar.html b/hadoop-hdds/docs/themes/ozonedoc/layouts/partials/sidebar.html index 275d1f4f4069b..72b9cfc95c071 100644 --- a/hadoop-hdds/docs/themes/ozonedoc/layouts/partials/sidebar.html +++ b/hadoop-hdds/docs/themes/ozonedoc/layouts/partials/sidebar.html @@ -15,13 +15,13 @@ limitations under the License. -->