Skip to content

Commit

Permalink
deploy: 3c9e819
Browse files Browse the repository at this point in the history
  • Loading branch information
bjw-s-bot[bot] committed Dec 21, 2023
1 parent 225779e commit bf56a42
Show file tree
Hide file tree
Showing 23 changed files with 501 additions and 57 deletions.
1 change: 1 addition & 0 deletions 404.html
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@
<link rel="stylesheet" href="././assets/css/mdbook-admonish.css">
<link rel="stylesheet" href="./theme/catppuccin.css">
<link rel="stylesheet" href="./theme/catppuccin-highlight.css">
<link rel="stylesheet" href="./mdbook-admonish.css">

</head>
<body>
Expand Down
1 change: 1 addition & 0 deletions automation/ansible/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@
<link rel="stylesheet" href="../../././assets/css/mdbook-admonish.css">
<link rel="stylesheet" href="../.././theme/catppuccin.css">
<link rel="stylesheet" href="../.././theme/catppuccin-highlight.css">
<link rel="stylesheet" href="../.././mdbook-admonish.css">

</head>
<body>
Expand Down
1 change: 1 addition & 0 deletions automation/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@
<link rel="stylesheet" href="../././assets/css/mdbook-admonish.css">
<link rel="stylesheet" href=".././theme/catppuccin.css">
<link rel="stylesheet" href=".././theme/catppuccin-highlight.css">
<link rel="stylesheet" href=".././mdbook-admonish.css">

</head>
<body>
Expand Down
1 change: 1 addition & 0 deletions automation/terraform/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@
<link rel="stylesheet" href="../../././assets/css/mdbook-admonish.css">
<link rel="stylesheet" href="../.././theme/catppuccin.css">
<link rel="stylesheet" href="../.././theme/catppuccin-highlight.css">
<link rel="stylesheet" href="../.././mdbook-admonish.css">

</head>
<body>
Expand Down
35 changes: 27 additions & 8 deletions css/chrome.css
Original file line number Diff line number Diff line change
Expand Up @@ -269,8 +269,14 @@ pre > .buttons button {
/* On mobile, make it easier to tap buttons. */
padding: 0.3rem 1rem;
}

.sidebar-resize-indicator {
/* Hide resize indicator on devices with limited accuracy */
display: none;
}
}
pre > code {
display: block;
padding: 1rem;
}

Expand Down Expand Up @@ -422,22 +428,35 @@ ul#searchresults span.teaser em {
position: absolute;
cursor: col-resize;
width: 0;
right: 0;
right: calc(var(--sidebar-resize-indicator-width) * -1);
top: 0;
bottom: 0;
display: flex;
align-items: center;
}

.sidebar-resize-handle .sidebar-resize-indicator {
width: 100%;
height: 12px;
background-color: var(--icons);
margin-inline-start: var(--sidebar-resize-indicator-space);
}

[dir=rtl] .sidebar .sidebar-resize-handle {
left: calc(var(--sidebar-resize-indicator-width) * -1);
right: unset;
}
[dir=rtl] .sidebar .sidebar-resize-handle { right: unset; left: 0; }
.js .sidebar .sidebar-resize-handle {
cursor: col-resize;
width: 5px;
width: calc(var(--sidebar-resize-indicator-width) - var(--sidebar-resize-indicator-space));
}
/* sidebar-hidden */
#sidebar-toggle-anchor:not(:checked) ~ .sidebar {
transform: translateX(calc(0px - var(--sidebar-width)));
transform: translateX(calc(0px - var(--sidebar-width) - var(--sidebar-resize-indicator-width)));
z-index: -1;
}
[dir=rtl] #sidebar-toggle-anchor:not(:checked) ~ .sidebar {
transform: translateX(var(--sidebar-width));
transform: translateX(calc(var(--sidebar-width) + var(--sidebar-resize-indicator-width)));
}
.sidebar::-webkit-scrollbar {
background: var(--sidebar-bg);
Expand All @@ -448,15 +467,15 @@ ul#searchresults span.teaser em {

/* sidebar-visible */
#sidebar-toggle-anchor:checked ~ .page-wrapper {
transform: translateX(var(--sidebar-width));
transform: translateX(calc(var(--sidebar-width) + var(--sidebar-resize-indicator-width)));
}
[dir=rtl] #sidebar-toggle-anchor:checked ~ .page-wrapper {
transform: translateX(calc(0px - var(--sidebar-width)));
transform: translateX(calc(0px - var(--sidebar-width) - var(--sidebar-resize-indicator-width)));
}
@media only screen and (min-width: 620px) {
#sidebar-toggle-anchor:checked ~ .page-wrapper {
transform: none;
margin-inline-start: var(--sidebar-width);
margin-inline-start: calc(var(--sidebar-width) + var(--sidebar-resize-indicator-width));
}
[dir=rtl] #sidebar-toggle-anchor:checked ~ .page-wrapper {
transform: none;
Expand Down
2 changes: 1 addition & 1 deletion css/print.css
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
}

#page-wrapper.page-wrapper {
transform: none;
transform: none !important;
margin-inline-start: 0px;
overflow-y: initial;
}
Expand Down
2 changes: 2 additions & 0 deletions css/variables.css
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@

:root {
--sidebar-width: 300px;
--sidebar-resize-indicator-width: 8px;
--sidebar-resize-indicator-space: 2px;
--page-padding: 15px;
--content-max-width: 750px;
--menu-bar-height: 50px;
Expand Down
49 changes: 48 additions & 1 deletion highlight.js

Large diffs are not rendered by default.

20 changes: 13 additions & 7 deletions index.html
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@
<link rel="stylesheet" href="././assets/css/mdbook-admonish.css">
<link rel="stylesheet" href="./theme/catppuccin.css">
<link rel="stylesheet" href="./theme/catppuccin-highlight.css">
<link rel="stylesheet" href="./mdbook-admonish.css">

</head>
<body>
Expand Down Expand Up @@ -148,7 +149,7 @@ <h1 class="menu-title">Home-Ops</h1>
<div id="content" class="content">
<main>
<h1 id="introduction"><a class="header" href="#introduction">Introduction</a></h1>
<div id="admonition-warning" class="admonition warning">
<div id="admonition-warning" class="admonition admonish-warning">
<div class="admonition-title">
<p>Warning</p>
<p><a class="admonition-anchor-link" href="#admonition-warning"></a></p>
Expand All @@ -161,14 +162,19 @@ <h1 id="introduction"><a class="header" href="#introduction">Introduction</a></h
<hr />
<br />
<div align="center">
<img src="https://github.com/bjw-s/bjw-s/blob/main/_assets/images/logo.jpg?raw=true" align="left" width="144px" height="144px"/>
<img src="https://github.com/bjw-s/bjw-s/blob/main/_assets/images/logo.jpg?raw=true" width="144px" height="144px"/>
<h2 id="my-home-operations-repository"><a class="header" href="#my-home-operations-repository">My Home Operations repository</a></h2>
<p><em>... managed by Flux, Renovate and GitHub Actions</em> 🤖</p>
</div>
<div align="center">
<p><a href="https://discord.gg/k8s-at-home"><img src="https://img.shields.io/discord/673534664354430999?style=for-the-badge&amp;label&amp;logo=discord&amp;logoColor=white&amp;color=blue" alt="Discord" /></a>
<a href="https://www.talos.dev/"><img src="https://img.shields.io/badge/v1.26-blue?style=for-the-badge&amp;logo=kubernetes&amp;logoColor=white" alt="Kubernetes" /></a>
<a href="https://github.com/bjw-s/home-ops/actions/workflows/schedule-renovate.yaml"><img src="https://img.shields.io/github/actions/workflow/status/bjw-s/home-ops/schedule-renovate.yaml?branch=main&amp;label=&amp;logo=renovatebot&amp;style=for-the-badge&amp;color=blue" alt="Renovate" /></a></p>
<p><a href="https://discord.gg/home-operations"><img src="https://img.shields.io/discord/673534664354430999?style=for-the-badge&amp;label&amp;logo=discord&amp;logoColor=white&amp;color=blue" alt="Discord" /></a> 
<a href="https://github.com/bjw-s/home-ops/actions/workflows/schedule-renovate.yaml"><img src="https://img.shields.io/github/actions/workflow/status/bjw-s/.github/schedule-renovate.yaml?branch=main&amp;label=&amp;logo=renovatebot&amp;style=for-the-badge&amp;color=blue" alt="Renovate" /></a></p>
<p>Main k8s cluster stats:</p>
<p><a href="https://www.talos.dev/"><img src="https://img.shields.io/badge/dynamic/yaml?url=https%3A%2F%2Fraw.githubusercontent.com%2Fbjw-s%2Fhome-ops%2Fmain%2Fkubernetes%2Fmain%2Ftalos%2Ftalconfig.yaml&amp;query=%24.kubernetesVersion&amp;flat-square&amp;logo=kubernetes&amp;logoColor=white&amp;label=version" alt="Kubernetes" />
</a> 
<a href="https://github.com/kashalls/kromgo/"><img src="https://img.shields.io/endpoint?url=https%3A%2F%2Fkromgo.bjw-s.dev%2Fquery%3Fformat%3Dendpoint%26metric%3Dmain_cluster_age_days&amp;style=flat-square&amp;label=Age" alt="Age-Days" /></a> 
<a href="https://github.com/kashalls/kromgo/"><img src="https://img.shields.io/endpoint?url=https%3A%2F%2Fkromgo.bjw-s.dev%2Fquery%3Fformat%3Dendpoint%26metric%3Dmain_cluster_node_count&amp;style=flat-square&amp;label=Nodes" alt="Node-Count" /></a> 
<a href="https://github.com/kashalls/kromgo/"><img src="https://img.shields.io/endpoint?url=https%3A%2F%2Fkromgo.bjw-s.dev%2Fquery%3Fformat%3Dendpoint%26metric%3Dmain_cluster_pod_count&amp;style=flat-square&amp;label=Pods" alt="Pod-Count" /></a> 
<a href="https://github.com/kashalls/kromgo/"><img src="https://img.shields.io/endpoint?url=https%3A%2F%2Fkromgo.bjw-s.dev%2Fquery%3Fformat%3Dendpoint%26metric%3Dmain_cluster_cpu_usage&amp;style=flat-square&amp;label=CPU" alt="CPU-Usage" /></a> 
<a href="https://github.com/kashalls/kromgo/"><img src="https://img.shields.io/endpoint?url=https%3A%2F%2Fkromgo.bjw-s.dev%2Fquery%3Fformat%3Dendpoint%26metric%3Dmain_cluster_memory_usage&amp;style=flat-square&amp;label=Memory" alt="Memory-Usage" /></a> </p>
</div>
<br><br>
<p>👋 Welcome to my Home Operations repository. This is a mono repository for my home infrastructure and Kubernetes cluster. I try to adhere to Infrastructure as Code (IaC) and GitOps practices using the tools like <a href="https://www.ansible.com/">Ansible</a>, <a href="https://www.terraform.io/">Terraform</a>, <a href="https://kubernetes.io/">Kubernetes</a>, <a href="https://github.com/fluxcd/flux2">Flux</a>, <a href="https://github.com/renovatebot/renovate">Renovate</a> and <a href="https://github.com/features/actions">GitHub Actions</a>.</p>
Expand Down
20 changes: 13 additions & 7 deletions introduction.html
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@
<link rel="stylesheet" href="././assets/css/mdbook-admonish.css">
<link rel="stylesheet" href="./theme/catppuccin.css">
<link rel="stylesheet" href="./theme/catppuccin-highlight.css">
<link rel="stylesheet" href="./mdbook-admonish.css">

</head>
<body>
Expand Down Expand Up @@ -148,7 +149,7 @@ <h1 class="menu-title">Home-Ops</h1>
<div id="content" class="content">
<main>
<h1 id="introduction"><a class="header" href="#introduction">Introduction</a></h1>
<div id="admonition-warning" class="admonition warning">
<div id="admonition-warning" class="admonition admonish-warning">
<div class="admonition-title">
<p>Warning</p>
<p><a class="admonition-anchor-link" href="#admonition-warning"></a></p>
Expand All @@ -161,14 +162,19 @@ <h1 id="introduction"><a class="header" href="#introduction">Introduction</a></h
<hr />
<br />
<div align="center">
<img src="https://github.com/bjw-s/bjw-s/blob/main/_assets/images/logo.jpg?raw=true" align="left" width="144px" height="144px"/>
<img src="https://github.com/bjw-s/bjw-s/blob/main/_assets/images/logo.jpg?raw=true" width="144px" height="144px"/>
<h2 id="my-home-operations-repository"><a class="header" href="#my-home-operations-repository">My Home Operations repository</a></h2>
<p><em>... managed by Flux, Renovate and GitHub Actions</em> 🤖</p>
</div>
<div align="center">
<p><a href="https://discord.gg/k8s-at-home"><img src="https://img.shields.io/discord/673534664354430999?style=for-the-badge&amp;label&amp;logo=discord&amp;logoColor=white&amp;color=blue" alt="Discord" /></a>
<a href="https://www.talos.dev/"><img src="https://img.shields.io/badge/v1.26-blue?style=for-the-badge&amp;logo=kubernetes&amp;logoColor=white" alt="Kubernetes" /></a>
<a href="https://github.com/bjw-s/home-ops/actions/workflows/schedule-renovate.yaml"><img src="https://img.shields.io/github/actions/workflow/status/bjw-s/home-ops/schedule-renovate.yaml?branch=main&amp;label=&amp;logo=renovatebot&amp;style=for-the-badge&amp;color=blue" alt="Renovate" /></a></p>
<p><a href="https://discord.gg/home-operations"><img src="https://img.shields.io/discord/673534664354430999?style=for-the-badge&amp;label&amp;logo=discord&amp;logoColor=white&amp;color=blue" alt="Discord" /></a> 
<a href="https://github.com/bjw-s/home-ops/actions/workflows/schedule-renovate.yaml"><img src="https://img.shields.io/github/actions/workflow/status/bjw-s/.github/schedule-renovate.yaml?branch=main&amp;label=&amp;logo=renovatebot&amp;style=for-the-badge&amp;color=blue" alt="Renovate" /></a></p>
<p>Main k8s cluster stats:</p>
<p><a href="https://www.talos.dev/"><img src="https://img.shields.io/badge/dynamic/yaml?url=https%3A%2F%2Fraw.githubusercontent.com%2Fbjw-s%2Fhome-ops%2Fmain%2Fkubernetes%2Fmain%2Ftalos%2Ftalconfig.yaml&amp;query=%24.kubernetesVersion&amp;flat-square&amp;logo=kubernetes&amp;logoColor=white&amp;label=version" alt="Kubernetes" />
</a> 
<a href="https://github.com/kashalls/kromgo/"><img src="https://img.shields.io/endpoint?url=https%3A%2F%2Fkromgo.bjw-s.dev%2Fquery%3Fformat%3Dendpoint%26metric%3Dmain_cluster_age_days&amp;style=flat-square&amp;label=Age" alt="Age-Days" /></a> 
<a href="https://github.com/kashalls/kromgo/"><img src="https://img.shields.io/endpoint?url=https%3A%2F%2Fkromgo.bjw-s.dev%2Fquery%3Fformat%3Dendpoint%26metric%3Dmain_cluster_node_count&amp;style=flat-square&amp;label=Nodes" alt="Node-Count" /></a> 
<a href="https://github.com/kashalls/kromgo/"><img src="https://img.shields.io/endpoint?url=https%3A%2F%2Fkromgo.bjw-s.dev%2Fquery%3Fformat%3Dendpoint%26metric%3Dmain_cluster_pod_count&amp;style=flat-square&amp;label=Pods" alt="Pod-Count" /></a> 
<a href="https://github.com/kashalls/kromgo/"><img src="https://img.shields.io/endpoint?url=https%3A%2F%2Fkromgo.bjw-s.dev%2Fquery%3Fformat%3Dendpoint%26metric%3Dmain_cluster_cpu_usage&amp;style=flat-square&amp;label=CPU" alt="CPU-Usage" /></a> 
<a href="https://github.com/kashalls/kromgo/"><img src="https://img.shields.io/endpoint?url=https%3A%2F%2Fkromgo.bjw-s.dev%2Fquery%3Fformat%3Dendpoint%26metric%3Dmain_cluster_memory_usage&amp;style=flat-square&amp;label=Memory" alt="Memory-Usage" /></a> </p>
</div>
<br><br>
<p>👋 Welcome to my Home Operations repository. This is a mono repository for my home infrastructure and Kubernetes cluster. I try to adhere to Infrastructure as Code (IaC) and GitOps practices using the tools like <a href="https://www.ansible.com/">Ansible</a>, <a href="https://www.terraform.io/">Terraform</a>, <a href="https://kubernetes.io/">Kubernetes</a>, <a href="https://github.com/fluxcd/flux2">Flux</a>, <a href="https://github.com/renovatebot/renovate">Renovate</a> and <a href="https://github.com/features/actions">GitHub Actions</a>.</p>
Expand Down
1 change: 1 addition & 0 deletions kubernetes/backups.html
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@
<link rel="stylesheet" href="../././assets/css/mdbook-admonish.css">
<link rel="stylesheet" href=".././theme/catppuccin.css">
<link rel="stylesheet" href=".././theme/catppuccin-highlight.css">
<link rel="stylesheet" href=".././mdbook-admonish.css">

</head>
<body>
Expand Down
7 changes: 4 additions & 3 deletions kubernetes/gitops.html
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@
<link rel="stylesheet" href="../././assets/css/mdbook-admonish.css">
<link rel="stylesheet" href=".././theme/catppuccin.css">
<link rel="stylesheet" href=".././theme/catppuccin-highlight.css">
<link rel="stylesheet" href=".././mdbook-admonish.css">

</head>
<body>
Expand Down Expand Up @@ -149,19 +150,19 @@ <h1 class="menu-title">Home-Ops</h1>
<main>
<h1 id="gitops"><a class="header" href="#gitops">GitOps</a></h1>
<p><a href="https://github.com/fluxcd/flux2">Flux</a> watches my <a href="https://github.com/bjw-s/home-ops/tree/main/kubernetes/">kubernetes</a> folder (see <a href="#directory-structure">Directory structure</a>) and makes the changes to my cluster based on the YAML manifests.</p>
<p>The way Flux works for me here is it will recursively search the <a href="https://github.com/bjw-s/home-ops/tree/main/kubernetes/cluster-0/apps">kubernetes/apps</a> folder until it finds the most top level <code>kustomization.yaml</code> per directory and then apply all the resources listed in it. That aforementioned <code>kustomization.yaml</code> will generally only have a namespace resource and one or many Flux kustomizations. Those Flux kustomizations will generally have a <code>HelmRelease</code> or other resources related to the application underneath it which will be applied.</p>
<p>The way Flux works for me here is it will recursively search the <a href="https://github.com/bjw-s/home-ops/tree/main/kubernetes/main/apps">kubernetes/apps</a> folder until it finds the most top level <code>kustomization.yaml</code> per directory and then apply all the resources listed in it. That aforementioned <code>kustomization.yaml</code> will generally only have a namespace resource and one or many Flux kustomizations. Those Flux kustomizations will generally have a <code>HelmRelease</code> or other resources related to the application underneath it which will be applied.</p>
<p><a href="https://github.com/renovatebot/renovate">Renovate</a> watches my <strong>entire</strong> repository looking for dependency updates, when they are found a PR is automatically created. When PRs are merged <a href="https://github.com/fluxcd/flux2">Flux</a> applies the changes to my cluster.</p>
<h2 id="directory-structure"><a class="header" href="#directory-structure">Directory structure</a></h2>
<p>My home-ops repository contains the following directories under <a href="https://github.com/bjw-s/home-ops/tree/main/kubernetes/">kubernetes</a>.</p>
<pre><code class="language-sh">📁 kubernetes # Kubernetes clusters defined as code
├─📁 cluster-0 # My main kubernetes cluster
├─📁 main # My main kubernetes cluster
│ ├─📁 bootstrap # Flux installation
│ ├─📁 flux # Main Flux configuration of repository
│ └─📁 apps # Apps deployed into my cluster grouped by namespace (see below)
└─📁 tools # Manifests that come in handy every now and then
</code></pre>
<h2 id="flux-resource-layout"><a class="header" href="#flux-resource-layout">Flux resource layout</a></h2>
<p>Below is a a high level look at the layout of how my directory structure with Flux works. In this brief example you are able to see that <code>authelia</code> will not be able to run until <code>glauth</code> and <code>cloudnative-pg</code> are running. It also shows that the <code>Cluster</code> custom resource depends on the <code>cloudnative-pg</code> Helm chart. This is needed because <code>cloudnative-pg</code> installs the <code>Cluster</code> custom resource definition in the Helm chart.</p>
<p>Below is a a high level look at the layout of how my directory structure with Flux works. In this brief example you are able to see that <code>authelia</code> will not be able to run until <code>glauth</code> and <code>cloudnative-pg</code> are running. It also shows that the <code>Cluster</code> custom resource depends on the <code>cloudnative-pg</code> Helm chart. This is needed because <code>cloudnative-pg</code> installs the <code>Cluster</code> custom resource definition in the Helm chart.</p>
<pre><code class="language-python"># Key: &lt;kind&gt; :: &lt;metadata.name&gt;
GitRepository :: home-ops-kubernetes
Kustomization :: cluster
Expand Down
3 changes: 2 additions & 1 deletion kubernetes/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@
<link rel="stylesheet" href="../././assets/css/mdbook-admonish.css">
<link rel="stylesheet" href=".././theme/catppuccin.css">
<link rel="stylesheet" href=".././theme/catppuccin-highlight.css">
<link rel="stylesheet" href=".././mdbook-admonish.css">

</head>
<body>
Expand Down Expand Up @@ -148,7 +149,7 @@ <h1 class="menu-title">Home-Ops</h1>
<div id="content" class="content">
<main>
<h1 id="kubernetes"><a class="header" href="#kubernetes">Kubernetes</a></h1>
<p>My main cluster is <a href="http://talos.dev/">Talos</a> provisioned on bare-metal using the official <code>talosctl</code> CLI tool. I render my <a href="https://github.com/bjw-s/home-ops/tree/main/infrastructure/talos/cluster-0">Talos configuration</a> using the <a href="https://github.com/budimanjojo/talhelper">talhelper</a> CLI tool. This allows me to keep the Talos configuration as DRY as possible.</p>
<p>My main cluster is <a href="http://talos.dev/">Talos</a> provisioned on bare-metal using the official <code>talosctl</code> CLI tool. I render my <a href="https://github.com/bjw-s/home-ops/tree/main/infrastructure/talos/main">Talos configuration</a> using the <a href="https://github.com/budimanjojo/talhelper">talhelper</a> CLI tool. This allows me to keep the Talos configuration as DRY as possible.</p>
<p>This is a semi hyper-converged cluster, workloads and block storage are sharing the same available resources on my nodes while I have a separate server for (NFS) file storage.</p>
<h2 id="core-components"><a class="header" href="#core-components">Core Components</a></h2>
<ul>
Expand Down
Loading

0 comments on commit bf56a42

Please sign in to comment.