Skip to content
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion docs/_config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -37,10 +37,11 @@ defaults:
values:
layout: single
author_profile: false
read_time: true
read_time: false
comments: false
share: true
related: false
sneak_preview: false
# _tutorials
- scope:
path: ""
Expand Down
10 changes: 6 additions & 4 deletions docs/_data/navigation.yml
Original file line number Diff line number Diff line change
@@ -1,21 +1,23 @@
main:
- title: "Getting Started"
url: /getting-started/
- title: "Blog"
url: /blog/
- title: "News"
url: /news/
- title: "Tutorials"
url: /tutorials/
- title: "Documentation"
url: https://ghpages-test.readthedocs.io/
url: https://deepspeed.readthedocs.io/
- title: "GitHub"
url: https://github.com/microsoft/DeepSpeed

lnav:
- title: "This is a floating nav bar."
- title: "DeepSpeed Documentation"
- title: "Getting Started"
url: /getting-started/
children:
- title: "Installation"
url: /getting-started/#installation
- title: "Configuration"
url: /getting-started/#deepspeed-configuration
- title: "DeepSpeed Features"
url: /features/
27 changes: 27 additions & 0 deletions docs/_layouts/news-home.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
---
layout: archive
---

{{ content }}


{% if paginator %}
{% assign posts = paginator.posts %}
{% else %}
{% assign posts = site.posts %}
{% endif %}


<h2>Features Coming Soon</h2>
{% assign soon = posts | where: "sneak_preview", "true" %}
{% for post in soon %}
{% include archive-single.html %}
{% endfor %}

<h2>{{ site.data.ui-text[site.locale].recent_posts | default: "Recent Posts" }}</h2>
{% assign news = posts | where: "sneak_preview", "false" %}
{% for post in news %}
{% include archive-single.html %}
{% endfor %}

{% include paginator.html %}
5 changes: 2 additions & 3 deletions docs/_posts/2020-03-17-reduce-scatter.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,11 @@
---
title: "ZeRO stage 1 with reduced communication"
date: 2020-03-13
sneak_preview: true
excerpt: "Partition-aware ZeRO with up to 2x reduction in communication time!"
---

# ZeRO stage 1 with reduced communication
* Partition-aware approach instead of initial implementation that used a global collective (all-reduce)
* Total communication volume reduction 1.5x -> 1x of data parallelism
* Up to 2x reduction in communication time compared to all-reduce

# Further updates coming soon!
## Further updates coming soon!
6 changes: 2 additions & 4 deletions docs/_posts/2020-03-17-zero-stage2.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,10 @@
---
title: "ZeRO stage 2"
date: 2020-03-13
sneak_preview: true
excerpt: "Reduce memory footprint to enable training 10B models without model parallelism!"
---

# Zero Stage 2
* Reduce memory footprint of gradients
* Train larger models: e.g., 10B parameters on 32GPUs without model parallelism
* Train larger batch sizes

# Further updates coming soon!
## Further updates coming soon!
2 changes: 1 addition & 1 deletion docs/_tutorials/getting-started.md
Original file line number Diff line number Diff line change
Expand Up @@ -213,7 +213,7 @@ deepspeed --include="worker-2:0,1" \
<client_entry.py> <client args> \
--deepspeed --deepspeed_config ds_config.json
```

This is a floating nav bar.
### MPI Compatibility
As described above, DeepSpeed provides its own parallel launcher to help launch
multi-node/multi-gpu training jobs. If you prefer to launch your training job
Expand Down
49 changes: 49 additions & 0 deletions docs/assets/css/main.scss
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
---
# Only the main Sass file needs front matter (the dashes are enough)
---

@charset "utf-8";

@import "minimal-mistakes/skins/{{ site.minimal_mistakes_skin | default: 'default' }}"; // skin
@import "minimal-mistakes"; // main partials

//
// DeepSpeed customizations
//


.site-title {
display: -webkit-box;
display: -ms-flexbox;
display: flex;
-ms-flex-item-align: center;
align-self: center;
font-weight: bold;
font-size: $type-size-2; // DeepSpeed: increase size
}


.toc {
font-family: $sans-serif-narrow;
color: $gray;
background-color: $background-color;
border: 1px solid $border-color;
border-radius: $border-radius;
-webkit-box-shadow: $box-shadow;
box-shadow: $box-shadow;
position: fixed;

.nav__title {
color: #fff;
font-size: $type-size-6;
background: $primary-color;
border-top-left-radius: $border-radius;
border-top-right-radius: $border-radius;
}

// Scrollspy marks toc items as .active when they are in focus
.active a {
@include yiq-contrasted($active-color);
}
}

3 changes: 0 additions & 3 deletions docs/blog/index.html

This file was deleted.

1 change: 1 addition & 0 deletions docs/features.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
---
title: "Feature Overview"
layout: single
permalink: /features/
toc: true
toc_label: "Contents"
---
Expand Down
Loading