Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion docs/_config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -37,10 +37,11 @@ defaults:
values:
layout: single
author_profile: false
read_time: true
read_time: false
comments: false
share: true
related: false
sneak_preview: false
# _tutorials
- scope:
path: ""
Expand Down
10 changes: 6 additions & 4 deletions docs/_data/navigation.yml
Original file line number Diff line number Diff line change
@@ -1,21 +1,23 @@
main:
- title: "Getting Started"
url: /getting-started/
- title: "Blog"
url: /blog/
- title: "News"
url: /news/
- title: "Tutorials"
url: /tutorials/
- title: "Documentation"
url: https://ghpages-test.readthedocs.io/
url: https://deepspeed.readthedocs.io/
- title: "GitHub"
url: https://github.com/microsoft/DeepSpeed

lnav:
- title: "This is a floating nav bar."
- title: "DeepSpeed Documentation"
- title: "Getting Started"
url: /getting-started/
children:
- title: "Installation"
url: /getting-started/#installation
- title: "Configuration"
url: /getting-started/#deepspeed-configuration
- title: "DeepSpeed Features"
url: /features/
27 changes: 27 additions & 0 deletions docs/_layouts/news-home.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
---
layout: archive
---

{{ content }}


{% if paginator %}
{% assign posts = paginator.posts %}
{% else %}
{% assign posts = site.posts %}
{% endif %}


<h2>Features Coming Soon</h2>
{% assign soon = posts | where: "sneak_preview", "true" %}
{% for post in soon %}
{% include archive-single.html %}
{% endfor %}

<h2>{{ site.data.ui-text[site.locale].recent_posts | default: "Recent Posts" }}</h2>
{% assign news = posts | where: "sneak_preview", "false" %}
{% for post in news %}
{% include archive-single.html %}
{% endfor %}

{% include paginator.html %}
5 changes: 2 additions & 3 deletions docs/_posts/2020-03-17-reduce-scatter.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,11 @@
---
title: "ZeRO stage 1 with reduced communication"
date: 2020-03-13
sneak_preview: true
excerpt: "Partition-aware ZeRO with up to 2x reduction in communication time!"
---

# ZeRO stage 1 with reduced communication
* Partition-aware approach instead of initial implementation that used a global collective (all-reduce)
* Total communication volume reduction 1.5x -> 1x of data parallelism
* Up to 2x reduction in communication time compared to all-reduce

# Further updates coming soon!
## Further updates coming soon!
6 changes: 2 additions & 4 deletions docs/_posts/2020-03-17-zero-stage2.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,10 @@
---
title: "ZeRO stage 2"
date: 2020-03-13
sneak_preview: true
excerpt: "Reduce memory footprint to enable training 10B models without model parallelism!"
---

# Zero Stage 2
* Reduce memory footprint of gradients
* Train larger models: e.g., 10B parameters on 32GPUs without model parallelism
* Train larger batch sizes

# Further updates coming soon!
## Further updates coming soon!
2 changes: 1 addition & 1 deletion docs/_tutorials/getting-started.md
Original file line number Diff line number Diff line change
Expand Up @@ -213,7 +213,7 @@ deepspeed --include="worker-2:0,1" \
<client_entry.py> <client args> \
--deepspeed --deepspeed_config ds_config.json
```

This is a floating nav bar.
### MPI Compatibility
As described above, DeepSpeed provides its own parallel launcher to help launch
multi-node/multi-gpu training jobs. If you prefer to launch your training job
Expand Down
48 changes: 48 additions & 0 deletions docs/assets/css/main.scss
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
---
# Only the main Sass file needs front matter (the dashes are enough)
---

@charset "utf-8";

@import "minimal-mistakes/skins/{{ site.minimal_mistakes_skin | default: 'default' }}"; // skin
@import "minimal-mistakes"; // main partials

//
// DeepSpeed customizations
//


.site-title {
display: -webkit-box;
display: -ms-flexbox;
display: flex;
-ms-flex-item-align: center;
align-self: center;
font-weight: bold;
font-size: $type-size-2; // DeepSpeed: increase size
}


.toc {
font-family: $sans-serif-narrow;
color: $gray;
background-color: $background-color;
border: 1px solid $border-color;
border-radius: $border-radius;
-webkit-box-shadow: $box-shadow;
box-shadow: $box-shadow;
position: fixed;

.nav__title {
color: #fff;
font-size: $type-size-6;
background: $primary-color;
border-top-left-radius: $border-radius;
border-top-right-radius: $border-radius;
}

// Scrollspy marks toc items as .active when they are in focus
.active a {
@include yiq-contrasted($active-color);
}
}
3 changes: 0 additions & 3 deletions docs/blog/index.html

This file was deleted.

1 change: 1 addition & 0 deletions docs/features.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
---
title: "Feature Overview"
layout: single
permalink: /features/
toc: true
toc_label: "Contents"
---
Expand Down
Loading