diff --git a/.github/workflows/check_docs.yml b/.github/workflows/check_docs.yml
new file mode 100644
index 000000000000..d104b9a81387
--- /dev/null
+++ b/.github/workflows/check_docs.yml
@@ -0,0 +1,17 @@
+name: Check docs
+on: [pull_request]
+jobs:
+  check-docs:
+    name: Check links in docs
+    runs-on: ubuntu-latest
+    steps:
+    - name: Checkout code
+      uses: actions/checkout@v2
+    - name: Check links in docs
+      uses: gaurav-nelson/github-action-markdown-link-check@v1
+      # checks all markdown files from /docs including all subfolders
+      with:
+        use-quiet-mode: 'yes'
+        use-verbose-mode: 'yes'
+        config-file: '.github/workflows/markdown.links.config.json'
+        folder-path: 'docs/,example/,production/'
diff --git a/.github/workflows/markdown.links.config.json b/.github/workflows/markdown.links.config.json
new file mode 100644
index 000000000000..4087ae8d9dc9
--- /dev/null
+++ b/.github/workflows/markdown.links.config.json
@@ -0,0 +1,7 @@
+{
+  "ignorePatterns": [
+    {
+      "pattern": "^http://prometheus.k3d.localhost:30080"
+    }
+  ]
+}
diff --git a/cmd/agent-operator/DEVELOPERS.md b/cmd/agent-operator/DEVELOPERS.md
index 4cabb53c2c41..a7da0f22187f 100644
--- a/cmd/agent-operator/DEVELOPERS.md
+++ b/cmd/agent-operator/DEVELOPERS.md
@@ -86,7 +86,7 @@ go run ./cmd/agent-operator
 ### Apply a GrafanaAgent custom resource
 
 Finally, you can apply an example GrafanaAgent custom resource. One is [provided
-for you](./agent-example-config.yaml). From the root of the repository, run:
+for you](../../cmd/agent-operator/agent-example-config.yaml). From the root of the repository, run:
 
 ```
 kubectl apply -f ./cmd/agent-operator/agent-example-config.yaml
diff --git a/docs/README.md b/docs/README.md
index 1117ad486d50..465a50d4bf05 100644
--- a/docs/README.md
+++ b/docs/README.md
@@ -23,18 +23,18 @@ with:
     2. [Logs](./overview.md#logs)
     3. [Comparison to alternatives](./overview.md#comparison-to-alternatives)
     4. [Next Steps](./overview.md#next-steps)
-2. [Getting Started](./getting-started.md)
-    1. [Docker-Compose Example](./getting-started.md#docker-compose-example)
-    2. [k3d Example](./getting-started.md#k3d-example)
-    3. [Installing](./getting-started.md#installing)
-    4. [Creating a Config File](./getting-started.md#creating-a-config-file)
-        1. [Integrations](./getting-started.md#integrations)
-        2. [Prometheus-like Config/Migrating from Prometheus](./getting-started.md#prometheus-like-configmigrating-from-prometheus)
-        3. [Loki Config/Migrating from Promtail](./getting-started.md#loki-configmigrating-from-promtail)
-    5. [Running](./getting-started.md#running)
+2. [Getting Started](./getting-started/_index.md)
+    1. [Docker-Compose Example](./getting-started/_index.md#docker-compose-example)
+    2. [k3d Example](./getting-started/_index.md#k3d-example)
+    3. [Installing](./getting-started/_index.md#installing)
+    4. [Creating a Config File](./getting-started/_index.md#creating-a-config-file)
+        1. [Integrations](./getting-started/_index.md#integrations)
+        2. [Prometheus-like Config/Migrating from Prometheus](./getting-started/_index.md#prometheus-like-configmigrating-from-prometheus)
+        3. [Loki Config/Migrating from Promtail](./getting-started/_index.md#loki-configmigrating-from-promtail)
+    5. [Running](./getting-started/_index.md#running)
 3. [Configure Grafana Agent](./configuration/_index.md)
-4. [Migration Guide](./migration-guide.md)
+4. [Upgrade Guide](./upgrade-guide.md)
 5. [API](./api.md)
 6. [Scraping Service Mode](./scraping-service.md)
 7. [Operation Guide](./operation-guide.md)
-8. [Windows Guide](./windows.md)
+8. [Windows Guide](./getting-started/install-agent-on-windows.md)
diff --git a/docs/api.md b/docs/api.md
index c88844a04e49..e73fc6ba28f6 100644
--- a/docs/api.md
+++ b/docs/api.md
@@ -103,7 +103,7 @@ URL-encoded names will be stored in decoded form. e.g., `hello%2Fworld` will
 represent the config named `hello/world`.
 
 The request body passed to this endpoint must match the format of
-[prometheus_instance_config](./configuration-reference.md#prometheus_instance_config)
+[prometheus_instance_config](./configuration/prometheus-config.md)
 defined in the Configuration Reference. The name field of the configuration is
 ignored and the name in the URL takes precedence. The request body must be
 formatted as YAML.
diff --git a/docs/configuration/_index.md b/docs/configuration/_index.md
index 02d055c160bb..387f02f32a2a 100644
--- a/docs/configuration/_index.md
+++ b/docs/configuration/_index.md
@@ -47,7 +47,7 @@ undefined. The full list of supported syntax can be found at Drone's
 ## Reloading (beta)
 
 The configuration file can be reloaded at runtime. Read the [API
-documentation](./api.md#reload-configuration-file-beta) for more information.
+documentation](../api.md#reload-configuration-file-beta) for more information.
 
 This functionality is in beta, and may have issues. Please open GitHub issues
 for any problems you encounter.
diff --git a/docs/configuration/prometheus-config.md b/docs/configuration/prometheus-config.md
index 8146811cde5e..269dc971ef7b 100644
--- a/docs/configuration/prometheus-config.md
+++ b/docs/configuration/prometheus-config.md
@@ -70,7 +70,7 @@ also be provided. Acceptable values for  `client_auth_type` are found in
 ## scraping_service_config
 
 The `scraping_service` block configures the
-[scraping service](./scraping-service.md), an operational
+[scraping service](../scraping-service.md), an operational
 mode where configurations are stored centrally in a KV store and a cluster of
 agents distribute discovery and scrape load between nodes.
 
diff --git a/docs/getting-started/_index.md b/docs/getting-started/_index.md
index b56cbc4193b9..477be32f33c7 100644
--- a/docs/getting-started/_index.md
+++ b/docs/getting-started/_index.md
@@ -7,7 +7,7 @@ weight = 100
 
 This guide helps users get started with the Grafana Agent. For getting started
 with the Grafana Agent Operator, please refer to the Operator-specific
-[documentation](./operator).
+[documentation](../operator/_index.md).
 
 ## Installation methods
 
@@ -73,7 +73,7 @@ NAMESPACE="default" /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com
 ### Kubernetes Manifest
 
 If you wish to manually modify the Kubernetes manifest before deploying it
-yourself, you can do so by downloading the [`agent.yaml` file](/production/kubernetes/agent.yaml).
+yourself, you can do so by downloading the [`agent.yaml` file](../../production/kubernetes/agent.yaml).
 
 ### Install locally
 
@@ -84,7 +84,7 @@ Red Hat, and Debian.
 
 ### Tanka
 
-We provide [Tanka](https://tanka.dev) configurations in our [`production/`](/production/tanka/grafana-agent) directory.
+We provide [Tanka](https://tanka.dev) configurations in our [`production/`](../../production/tanka/grafana-agent) directory.
 
 ## Create a Config File
 
@@ -141,7 +141,7 @@ source of metrics if you are running multiple Grafana Agents across multiple
 machines.
 
 Full configuration options can be found in the
-[configuration reference](../configuration).
+[configuration reference](../configuration/_index.md).
 
 ### Prometheus Config/Migrating from Prometheus
 
@@ -187,7 +187,7 @@ prometheus:
 ```
 
 Like with integrations, full configuration options can be found in the
-[configuration reference](./configuration-reference.md).
+[configuration](../configuration/_index.md).
 
 ### Loki Config/Migrating from Promtail
 
diff --git a/docs/operation-guide.md b/docs/operation-guide.md
index 385ca111e88a..86031a41a977 100644
--- a/docs/operation-guide.md
+++ b/docs/operation-guide.md
@@ -175,7 +175,7 @@ from that `remote_write` config separated by a `-`.
 The shared Instances mode is the new default, and the previous behavior is
 deprecated. If you wish to restore the old behavior, set `instance_mode:
 distinct` in the
-[`prometheus_config`](./configuration-reference.md#prometheus_config) block of
+[`prometheus_config`](./configuration/prometheus-config.md) block of
 your config file.
 
 Shared Instances are completely transparent to the user with the exception of
diff --git a/docs/operator/architecture.md b/docs/operator/architecture.md
index 44f4f891a10a..662d51859bf5 100644
--- a/docs/operator/architecture.md
+++ b/docs/operator/architecture.md
@@ -48,7 +48,7 @@ deployment will also be deleted.
 Reconciling creates a few cluster resources:
 
 1. A Secret is generated holding the
-   [configuration](../configuration-reference.md) of the Grafana Agent.
+   [configuration](../configuration/_index.md) of the Grafana Agent.
 2. Another Secret is created holding all referenced Secrets or ConfigMaps from
    the resource hierarchy. This ensures that Secrets referenced from a custom
    resource in another namespace can still be read.
diff --git a/docs/overview.md b/docs/overview.md
index a4850006d555..b0bf9ead9a41 100644
--- a/docs/overview.md
+++ b/docs/overview.md
@@ -88,7 +88,7 @@ developer team.
 Grafana Agent supports collecting traces and sending them to Tempo using its
 `tempo` subsystem. This is done by utilizing the upstream [OpenTelemetry Collector](https://github.com/open-telemetry/opentelemetry-collector).
 The agent is capable of ingesting OpenTelemetry, OpenCensus, Jaeger, Zipkin or Kafka spans.
-See documentation on how to configure [receivers](./configuration-reference.md#tempo_config).
+See documentation on how to configure [receivers](./configuration/tempo-config.md).
 The agent is capable of exporting to any OpenTelemetry GRPC compatible system.
 
 ## Comparison to Alternatives
@@ -119,6 +119,6 @@ and Tempo for traces, all-in-one.
 ## Next Steps
 
 For more information on installing and running the agent, see
-[Getting started](./getting-started.md) or
-[Configuration Reference](./configuration-reference.md) for a detailed reference
+[Getting started](./getting-started/_index.md) or
+[Configuration Reference](./configuration/_index.md) for a detailed reference
 on the configuration file.
diff --git a/docs/scraping-service.md b/docs/scraping-service.md
index 1723e870a3a2..add6bbbb86f5 100644
--- a/docs/scraping-service.md
+++ b/docs/scraping-service.md
@@ -35,7 +35,7 @@ remote_write:
 
 The full set of supported options for an instance configuration file is
 available in the
-[`prometheus_instance_config` section of Configuration Reference](./configuration-reference.md#prometheus_instance_config).
+[`prometheus-config.md` file](./configuration/prometheus-config.md).
 
 Having multiple instance configuration files is necessary for sharding; each
 config file is distributed to a particular agent on the cluster based on the
diff --git a/example/docker-compose/README.md b/example/docker-compose/README.md
index e9cd1aad5381..7a2cccebc076 100644
--- a/example/docker-compose/README.md
+++ b/example/docker-compose/README.md
@@ -57,7 +57,7 @@ Useful one-off queries to run once everything is up:
 
 ## Scraping Service
 
-A [Scraping Service](../docs/scraping-service.md) cluster example is also
+A [Scraping Service](../../docs/scraping-service.md) cluster example is also
 present in this directory. To start it, run:
 
 ```
diff --git a/production/README.md b/production/README.md
index ce3ca8266a67..c2a6551377bb 100644
--- a/production/README.md
+++ b/production/README.md
@@ -27,7 +27,7 @@ See the [Kubernetes README](./kubernetes/README.md) for more information.
 
 ## Windows Installation
 
-To run the Windows Installation, download the Windows Installer executable from the [release page](https://github.com/grafana/agent/releases). Then run the installer, this will setup the Agent and run the Agent as a Windows Service. More details can be found in the [Windows Guide](../docs/windows.md)
+To run the Windows Installation, download the Windows Installer executable from the [release page](https://github.com/grafana/agent/releases). Then run the installer, this will setup the Agent and run the Agent as a Windows Service. More details can be found in the [Windows Guide](../docs/getting-started/install-agent-on-windows.md)
 
 ## Running the Agent with Docker
 
diff --git a/production/kubernetes/README.md b/production/kubernetes/README.md
index be1b63fbdeb5..53aa4d372d55 100644
--- a/production/kubernetes/README.md
+++ b/production/kubernetes/README.md
@@ -58,7 +58,7 @@ of software installed:
 1. [Tanka](https://github.com/grafana/tanka) >= v0.8
 2. [`jsonnet-bundler`](https://github.com/jsonnet-bundler/jsonnet-bundler) >= v0.2.1
 
-See the [`template` Tanka environment](./build/template) for the current
+See the [`template` Tanka environment](./build/templates) for the current
 settings that initialize the Grafana Agent Tanka configs. To build the YAML
 file, execute the `./build/build.sh` script or run `make example-kubernetes`
 from the project's root directory.