feature request: allow unloading a suspended pool from memory without exporting first #5242
Comments
mailinglists35
changed the title
|
shutdown hangs for me even when a non-root pool gets suspended |
|
pinging @behlendorf |
|
note that in #3256 even if you bring back the device, the pool remains stuck in suspended state |
|
If someone is interested in working on this by all means let me know. |
|
I did a bit of research into un-suspending a pool. One thing that seems to cause problems is using volatile names for the vdevs such as /dev/sdX. If a pool is created as I realize the feature being requested here is to "abandon" a suspended pool but I figured I'd point out the issue with the vdev names. It seems the biggest issue in being able to abandon a suspended pool is dealing with unmounting its filesystems and dealing with any processes which may have its zvols open. |
|
I'm commeting here because this issue appears to be the best defined of the bunch of related issues, some of which have been referenced here earlier. After some private correspondence with others, I've seen two cases arise: The first, and best defined, is a pool which has been "cleanly" suspended via As an example of the second case, a pool with no redundancy had a spacemap in which one of its blkptr's 3 ditto blocks [EDIT]
That set of 4 events was repeated one more time prior to the panic. The feature desired, as indicated in this issue, is to unload (I've been using the term "evacuate") a pool in either of these states to avoid rebooting. Presumably, the faulted pool is some sort of back-up or is otherwise not critical to the system's operation but it does continue to consume resources and can also cause processes to block, etc. @behlendorf I plan on looking into this, beginning with what seems to be the more straightforward case of a pool that's simply been suspended. The second case is likelier to be trickier and, in thinking about it, if the system thinks a pool is toasted that bad (and in the case I described, the pool was toast, although likely mostly recoverable with some careful gymnastics), it would argue for a non-panicking mode for non-debug builds in which the pool were put in some other type of suspended state in which the only option would be to evacuate it. |
|
@dweeezil that's great. The suspended pool case is definitely something which can be handled better and should be relatively straightforward. It should largely be a matter of tearing down everything cleanly and returning errors to any active ZFS consumers. As for the second case if a VERIFY or ASSERT is hit that is by design intended to be fatal and unrecoverable. For cases where it's possible to hit a VERIFY due to an IO error we're going to need to replace that VERIFY with proper error handling code. So for example |
@dweeezil is there any hope for a functional PR in a not so distant future? :-) |
|
@mailinglists35 Sorry, but this issue seems to have resisted percolating sufficiently high on my to-do list to get the attention it deserves. Are you mainly interested in the case where a VERIFY is hit and causes a panic? Your original report didn't have any |
sorry, I assumed referencing an example issue in the first comment after filing the issue should be enough. Here is a relevant one from those enumerated initially, containing zpool status and dmesg also (no logs, just status) #3461 |
|
I think this comment summarizes in a clearer language the feature request: "@gordan-bobic commented on Dec 15, 2016 |
|
Hi @dweeezil |
|
related comment |
|
@mailinglists35 At the moment, I'm trying to get the device evacuation code to a point where it can be merged. It's been merged upstream but there are 2 lingering issues with ZoL. Other than the fact it's a killer feature, one of my main bits of interest is that it's a prerequisite to the overhauled spa import code which will ultimately allow a lot of interesting things to be done with different types of vdevs. After that's done, I really want to get back to getting the TRIM patch set merged (which has been languishing upstream as well for a very long time). As to this issue, I'll try to dig up my WIP branch in which I was working on it earler. The enhanced deadman code may likely help matters with this issue. Speaking of which, the new deadman code (just committed to master on Jan 25, 2018) has |
|
thank you! do you think @sanjeevbagewadi's diff can be used/integrated on your branch, or is it unrelated to this issue? |
|
@dweeezil also do you think this is challenging enough to only see the light at the time of 1.0.0 release? |
|
@mailinglists35 As you've likely noticed, I'm still grinding away on the device evacuation code. It does sound like @sanjeevbagewadi may be working on something similar based on the commentary in #6649 in which pool export is mentioned. I just did a bit more looking around at some of the underlying issues. Among plenty of other things, the whole code base right now pretty much assumes that |
|
This is unfortunate. I still run into this issue fairly regularly. A zpool clear has helped once or twice, but most of the time I still get a "cannot clear errors for [poolname]: I/O error". It's frustrating that a hard reboot is my only way to recover from this. |
|
Ditto here, this feature is much needed. Please see my comments on our specific use case here: #3461 (comment) |
|
Suffering this a Lot, my setup zfs Luks USB should be a way to re-add or redetect drives automatically |
I have an external USB drive that is powered from an unreliable source. this is how I do it: you would have to add linux device mapper between LUKS and physical device (I *think you can do this without recreating your pool) the magic is in being able to replace the physical device with the error dm target, which instantly kills any outstanding I/O (processeses then are exiting the D state) - https://wiki.gentoo.org/wiki/Device-mapper#Error |
with luks looks like is even easier. |
|
anyway i fixed the usb discconections with this: |
|
Wondering if this old issue might get some attention, dealing with it again today. |
|
I just want to give heads up on this. This is still current and still needed. |
|
There is a PR open and under review which implements this #11082. Additional reviewers and tester are welcome. |
|
i'm experimenting with iscsi + zfs and multihost=on and i don't get the point why i should reboot my system to fix an intermittent connection problem to the iscsi targets even worse, the link to https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-MM shows "404 Page not found" sorry, but this sucks big ass. whoever decided that this is only fixable by reboot did not take into account, that zfs may run on servers/hypervisors where reboot is not easily possible because users/clients need to get informed before. |
|
Note, you can increase the timeout before a multihost pool becomes suspended with |
|
from the manpage so, this is "by design" that suspended pool in multihost scenario needs reboot. i still don't get the real point. ok, it's security measure. but if i'm admin of all the servers and if i know what i am doing, why does zfs disempower me to do what i want ? i'm on a hypervisor system (proxmox) and there i running dozens of VMs. i shall reboot all of these because of this issue? please add that unload option or allow unsuspend with force option "i know what i'm doing" to re-enable or clearing suspended pool |
|
I just ran into similar issues while prototyping a ISCSI based zpool intended to provide HA extents and want to bump this topic up because it is a real showstopper. @devZer0: If you can create the pool that runs on ISCSI targets in "FreeBSD 12 ZFS" inside a VM you can take a suspended pool back online with "zfs clear pool" as expected. The VM itself should only require network and CPU resources if your ZVOL exports from there run in sync mode (I still benchmarking this but so far it looks promising and stable). |
|
@chris0001 please watch the discussion progress in the corresponding pull request 11082 there is nothing more to do than that as an enduser |
- [Fast castle boom](https://www.youtube.com/watch?v=JsTNM7j6fs4&t=119) - How to play Arena: [Hera's guide](https://piped.video/watch?v=8gXI4XGMPzQ&t=0), [Tatoh game in arena](https://www.youtube.com/watch?v=3qg4Xwm8CAo&t=1211) - [How to play Hideout](https://www.youtube.com/watch?v=DdK8QveBegw&t=652) - [How to play Blackforest](https://www.youtube.com/watch?v=1V_jsU9PF8Y) - Inside the mind of a pro player: [Episode 1](https://www.youtube.com/watch?v=54hRmrdzO-I), [Episode 2](https://www.youtube.com/watch?v=sZCs6dwH5qk&t=1727) feat(age_of_empires# Strategies against civilisations): Strategies against civilisations I'm using only the mongols, and so far I've seen/heard from the pros the next strategies: - Aztecs: - Steppe lancers good against eagle warriors - Heavy scorpions against eagle warriors and skirms - Cumans: - [Scout, if it drops two TCs in feudal, tower rush into archers](https://www.youtube.com/watch?v=H9QUNtFII1g&t=0) - [Put initial pressure](https://www.youtube.com/watch?v=R9qaFZzZgBY&t=1925): Nice initial pressure - Incas: - Steppe lancers good against eagle warriors - Heavy scorpions against eagle warriors and skirms - Khmer: boom, map control, monks and albadiers - Mayans: - Steppe lancers good against eagle warriors - Heavy scorpions against eagle warriors and skirms - Romans: - [Hera guide on how to beat them](https://www.youtube.com/watch?v=SA44-Y3XUy0&t=842) - Tartars: heavy scorpions - Turks: - [How to defend against them in Arena](https://www.youtube.com/watch?v=AI_JRA_nCpw&t=3710) feat(age_of_empires#Nice Games): Nice games Tournaments: - 2023 Masters of Arena 7 Final Tatoh vs Vinchester: - [Casted by T90](https://www.youtube.com/watch?v=3qg4Xwm8CAo&t=1211s) - [Pov by Tatoh](https://www.youtube.com/watch?v=AI_JRA_nCpw&t=8854) Showmatches: - [Hera vs TheViper | Battlegrounds 3 | BO5](https://www.youtube.com/watch?v=AlKMRQNMVzo&t=4306) - [The Viper VS Tatoh PA7](https://www.youtube.com/watch?v=5_p3TXasBHY&t=5319) 1vs1 games: - [Hindustanis vs Portuguese | Arabia | Hera vs Yo](https://www.youtube.com/watch?v=iZ7eWLLbh34) - [Dravidians vs Turks | African Clearing | Hera vs Yo](https://www.youtube.com/watch?v=tZyVLDwBfd4) feat(ansible_snippets#Run command on a working directory): Run command on a working directory ```yaml - name: Change the working directory to somedir/ and run the command as db_owner ansible.builtin.command: /usr/bin/make_database.sh db_user db_name become: yes become_user: db_owner args: chdir: somedir/ creates: /path/to/database ``` feat(ansible_snippets#Run handlers in the middle of the tasks file): Run handlers in the middle of the tasks file If you need handlers to run before the end of the play, add a task to flush them using the [meta module](https://docs.ansible.com/ansible/latest/collections/ansible/builtin/meta_module.html#meta-module), which executes Ansible actions: ```yaml tasks: - name: Some tasks go here ansible.builtin.shell: ... - name: Flush handlers meta: flush_handlers - name: Some other tasks ansible.builtin.shell: ... ``` The `meta: flush_handlers` task triggers any handlers that have been notified at that point in the play. Once handlers are executed, either automatically after each mentioned section or manually by the `flush_handlers meta` task, they can be notified and run again in later sections of the play. feat(ansible_snippets#Run command idempotently): Run command idempotently ```yaml - name: Register the runner in gitea become: true command: act_runner register --config config.yaml --no-interactive --instance {{ gitea_url }} --token {{ gitea_docker_runner_token }} args: creates: /var/lib/gitea_docker_runner/.runner ``` feat(ansible_snippets#Get the correct architecture string): Get the correct architecture string If you have an `amd64` host you'll get `x86_64`, but sometimes you need the `amd64` string. On those cases you can use the next snippet: ```yaml --- deb_architecture: aarch64: arm64 x86_64: amd64 --- - name: Download the act runner binary become: True ansible.builtin.get_url: url: https://dl.gitea.com/act_runner/act_runner-linux-{{ deb_architecture[ansible_architecture] }} dest: /usr/bin/act_runner mode: '0755' ``` feat(ansible_snippets#Check the instances that are going to be affected by playbook run): Check the instances that are going to be affected by playbook run Useful to list the instances of a dynamic inventory ```bash ansible-inventory -i aws_ec2.yaml --list ``` feat(ansible_snippets#Check if variable is defined or empty): Check if variable is defined or empty In Ansible playbooks, it is often a good practice to test if a variable exists and what is its value. Particularity this helps to avoid different “VARIABLE IS NOT DEFINED” errors in Ansible playbooks. In this context there are several useful tests that you can apply using [Jinja2 filters](https://docs.ansible.com/ansible/latest/user_guide/playbooks_filters.html) in Ansible. feat(ansible_snippets): Check if Ansible variable is defined (exists) ```yaml tasks: - shell: echo "The variable 'foo' is defined: '{{ foo }}'" when: foo is defined - fail: msg="The variable 'bar' is not defined" when: bar is undefined ``` feat(ansible_snippets#Check if Ansible variable is empty): Check if Ansible variable is empty ```yaml tasks: - fail: msg="The variable 'bar' is empty" when: bar|length == 0 - shell: echo "The variable 'foo' is not empty: '{{ foo }}'" when: foo|length > 0 ``` feat(ansible_snippets#Check if Ansible variable is defined and not empty): Check if Ansible variable is defined and not empty ```yaml tasks: - shell: echo "The variable 'foo' is defined and not empty" when: (foo is defined) and (foo|length > 0) - fail: msg="The variable 'bar' is not defined or empty" when: (bar is not defined) or (bar|length == 0) ``` feat(ansible_snippets#Download a file): Download a file ```yaml - name: Download foo.conf ansible.builtin.get_url: url: http://example.com/path/file.conf dest: /etc/foo.conf mode: '0440' ``` feat(authentik#Monitorization): Disregard monitorization I've skimmed through the prometheus metrics exposed at `:9300/metrics` in the core and they aren't that useful :( feat(bash_snippets#Get the root path of a git repository): Get the root path of a git repository ```bash git rev-parse --show-toplevel ``` feat(bash_snippets#Get epoch gmt time): Get epoch gmt time ```bash date -u '+%s' ``` feat(bash_snippets#Check the length of an array with jq): Check the length of an array with jq ``` echo '[{"username":"user1"},{"username":"user2"}]' | jq '. | length' ``` feat(bash_snippets#Exit the script if there is an error): Exit the script if there is an error ```bash set -eu ``` feat(bash_snippets#Prompt the user for data): Prompt the user for data ```bash read -p "Ask whatever" choice ``` feat(beets): Guide on how to start using it You’ll want to set a few basic options before you start using beets. The [configuration](https://beets.readthedocs.io/en/stable/reference/config.html) is stored in a text file. You can show its location by running `beet config -p`, though it may not exist yet. Run `beet config -e` to edit the configuration in your favorite text editor. The file will start out empty, but here’s good place to start: ```yaml directory: ~/music library: ~/data/musiclibrary.db ``` The default configuration assumes you want to start a new organized music folder (that directory above) and that you’ll copy cleaned-up music into that empty folder using beets’ `import` command. But you can configure beets to behave many other ways: - Start with a new empty directory, but move new music in instead of copying it (saving disk space). Put this in your config file: ```yaml import: move: yes ``` - Keep your current directory structure; importing should never move or copy files but instead just correct the tags on music. Put the line `copy: no` under the `import:` heading in your config file to disable any copying or renaming. Make sure to point `directory` at the place where your music is currently stored. - Keep your current directory structure and do not correct files’ tags: leave files completely unmodified on your disk. (Corrected tags will still be stored in beets’ database, and you can use them to do renaming or tag changes later.) Put this in your config file: ```yaml import: copy: no write: no ``` to disable renaming and tag-writing. feat(beets#Importing your library): Importing your library The next step is to import your music files into the beets library database. Because this can involve modifying files and moving them around, data loss is always a possibility, so now would be a good time to make sure you have a recent backup of all your music. We’ll wait. There are two good ways to bring your existing library into beets. You can either: (a) quickly bring all your files with all their current metadata into beets’ database, or (b) use beets’ highly-refined autotagger to find canonical metadata for every album you import. Option (a) is really fast, but option (b) makes sure all your songs’ tags are exactly right from the get-go. The point about speed bears repeating: using the autotagger on a large library can take a very long time, and it’s an interactive process. So set aside a good chunk of time if you’re going to go that route. If you’ve got time and want to tag all your music right once and for all, do this: ```bash beet import /path/to/my/music ``` (Note that by default, this command will copy music into the directory you specified above. If you want to use your current directory structure, set the import.copy config option.) To take the fast, un-autotagged path, just say: ```bash beet import -A /my/huge/mp3/library ``` Note that you just need to add `-A` for “don’t autotag”. feat(pydantic#To investigate): Investigate libraries [Integration of pydantic with pandas](https://pandera.readthedocs.io/en/stable/pydantic_integration.html) feat(copier): Introduce copier [Copier](https://github.com/copier-org/copier) is a library and CLI app for rendering project templates. - Works with local paths and Git URLs. - Your project can include any file and Copier can dynamically replace values in any kind of text file. - It generates a beautiful output and takes care of not overwriting existing files unless instructed to do so. This long article covers: - [Installation](copier.md#installation) - [Basic concepts](copier.md#basic-concepts) - [Usage](copier.md#usage) feat(kubectl_commands#Run a pod in a defined node): Run a pod in a defined node Get the node hostnames with `kubectl get nodes`, then override the node with: ```bash kubectl run mypod --image ubuntu:18.04 --overrides='{"apiVersion": "v1", "spec": {"nodeSelector": { "kubernetes.io/hostname": "my-node.internal" }}}' --command -- sleep 100000000000000 ``` feat(diffview): Introduce DiffView [Diffview](https://github.com/sindrets/diffview.nvim) is a single tabpage interface for easily cycling through diffs for all modified files for any git rev. Installation: If you're using it with NeoGit and Packer use: ```lua use { 'NeogitOrg/neogit', requires = { 'nvim-lua/plenary.nvim', 'sindrets/diffview.nvim', 'nvim-tree/nvim-web-devicons' } } ``` Usage: Calling `:DiffviewOpen` with no args opens a new `Diffview` that compares against the current index. You can also provide any valid git rev to view only changes for that rev. Examples: - `:DiffviewOpen` - `:DiffviewOpen HEAD~2` - `:DiffviewOpen HEAD~4..HEAD~2` - `:DiffviewOpen d4a7b0d` - `:DiffviewOpen d4a7b0d^!` - `:DiffviewOpen d4a7b0d..519b30e` - `:DiffviewOpen origin/main...HEAD` You can also provide additional paths to narrow down what files are shown `:DiffviewOpen HEAD~2 -- lua/diffview plugin`. Additional commands for convenience: - `:DiffviewClose`: Close the current diffview. You can also use `:tabclose`. - `:DiffviewToggleFiles`: Toggle the file panel. - `:DiffviewFocusFiles`: Bring focus to the file panel. - `:DiffviewRefresh`: Update stats and entries in the file list of the current Diffview. With a Diffview open and the default key bindings, you can: - Cycle through changed files with `<tab>` and `<s-tab>` - You can stage changes with `-` - Restore a file with `X` - Refresh the diffs with `R` - Go to the file panel with `<leader>e` feat(docker#Add healthcheck to your dockers): Add healthcheck to your dockers Health checks allow a container to expose its workload’s availability. This stands apart from whether the container is running. If your database goes down, your API server won’t be able to handle requests, even though its Docker container is still running. This makes for unhelpful experiences during troubleshooting. A simple `docker ps` would report the container as available. Adding a health check extends the `docker ps` output to include the container’s true state. You configure container health checks in your Dockerfile. This accepts a command which the Docker daemon will execute every 30 seconds. Docker uses the command’s exit code to determine your container’s healthiness: - `0`: The container is healthy and working normally. - `1`: The container is unhealthy; the workload may not be functioning. Healthiness isn’t checked straightaway when containers are created. The status will show as starting before the first check runs. This gives the container time to execute any startup tasks. A container with a passing health check will show as healthy; an unhealthy container displays unhealthy. In docker-compose you can write the healthchecks like the next snippet: ```yaml --- version: '3.4' services: jellyfin: image: linuxserver/jellyfin:latest container_name: jellyfin restart: unless-stopped healthcheck: test: curl http://localhost:8096/health || exit 1 interval: 10s retries: 5 start_period: 5s timeout: 10s ``` feat(docker#List the dockers of a registry): List the dockers of a registry List all repositories (effectively images): ```bash $: curl -X GET https://myregistry:5000/v2/_catalog > {"repositories":["redis","ubuntu"]} ``` List all tags for a repository: ```bash $: curl -X GET https://myregistry:5000/v2/ubuntu/tags/list > {"name":"ubuntu","tags":["14.04"]} ``` If the registry needs authentication you have to specify username and password in the curl command ```bash curl -X GET -u <user>:<pass> https://myregistry:5000/v2/_catalog curl -X GET -u <user>:<pass> https://myregistry:5000/v2/ubuntu/tags/list ``` feat(git#Remove tags): Remove tags To delete a tag you can run: ```bash git tag -d {{tag_name}} ``` To remove them remotely do ```bash git push --delete origin {{ tag_name }} ``` fix(gitea): Configure the gitea actions So far there is [only one possible runner](https://gitea.com/gitea/act_runner) which is based on docker and [`act`](https://github.com/nektos/act). Currently, the only way to install act runner is by compiling it yourself, or by using one of the [pre-built binaries](https://dl.gitea.com/act_runner). There is no Docker image or other type of package management yet. At the moment, act runner should be run from the command line. Of course, you can also wrap this binary in something like a system service, supervisord, or Docker container. You can create the default configuration of the runner with: ```bash ./act_runner generate-config > config.yaml ``` You can tweak there for example the `capacity` so you are able to run more than one workflow in parallel. Before running a runner, you should first register it to your Gitea instance using the following command: ```bash ./act_runner register --config config.yaml --no-interactive --instance <instance> --token <token> ``` Finally, it’s time to start the runner. ```bash ./act_runner --config config.yaml daemon ``` If you want to create your own act docker, you can start with this dockerfile: ```dockerfile FROM node:16-bullseye LABEL prune=false RUN mkdir /root/.aws COPY files/config /root/.aws/config COPY files/credentials /root/.aws/credentials RUN apt-get update && apt-get install -y \ python3 \ python3-pip \ python3-venv \ screen \ vim \ && python3 -m pip install --upgrade pip \ && rm -rf /var/lib/apt/lists/* RUN pip install \ molecule==5.0.1 \ ansible==8.0.0 \ ansible-lint \ yamllint \ molecule-plugins[ec2,docker,vagrant] \ boto3 \ botocore \ testinfra \ pytest RUN wget https://download.docker.com/linux/static/stable/x86_64/docker-24.0.2.tgz \ && tar xvzf docker-24.0.2.tgz \ && cp docker/* /usr/bin \ && rm -r docker docker-* ``` It's prepared for: - Working within an AWS environment - Run Ansible and molecule - Build dockers feat(gitea#Build a docker within a gitea action): Build a docker within a gitea action Assuming you're using the custom gitea_runner docker proposed above you can build and upload a docker to a registry with this action: ```yaml --- name: Publish Docker image "on": [push] jobs: build-and-push: runs-on: ubuntu-latest steps: - name: Checkout code uses: https://github.com/actions/checkout@v3 - name: Login to Docker Registry uses: https://github.com/docker/login-action@v2 with: registry: my_registry.org username: ${{ secrets.REGISTRY_USERNAME }} password: ${{ secrets.REGISTRY_PASSWORD }} - name: Set up QEMU uses: https://github.com/docker/setup-qemu-action@v2 - name: Set up Docker Buildx uses: https://github.com/docker/setup-buildx-action@v2 - name: Extract metadata (tags, labels) for Docker id: meta uses: https://github.com/docker/metadata-action@v4 with: images: my_registry.org/the_name_of_the_docker_to_build - name: Build and push uses: docker/build-push-action@v2 with: context: . platforms: linux/amd64,linux/arm64 push: true cache-from: type=registry,ref=my_registry.org/the_name_of_the_docker_to_build:buildcache cache-to: type=registry,ref=my_registry.org/the_name_of_the_docker_to_build:buildcache,mode=max tags: ${{ steps.meta.outputs.tags }} labels: ${{ steps.meta.outputs.labels }} ``` It uses a pair of nice features: - Multi-arch builds - [Cache](https://docs.docker.com/build/ci/github-actions/cache/) to speed up the builds As it reacts to all events it will build and push: - A tag with the branch name on each push to that branch - A tag with the tag on tag push feat(gitea#Bump the version of a repository on commits on master): Bump the version of a repository on commits on master - Create a SSH key for the CI to send commits to protected branches. - Upload the private key to a repo or organization secret called `DEPLOY_SSH_KEY`. - Upload the public key to the repo configuration deploy keys - Create the `bump.yaml` file with the next contents: ```yaml --- name: Bump version "on": push: branches: - main jobs: bump_version: if: "!startsWith(github.event.head_commit.message, 'bump:')" runs-on: ubuntu-latest name: "Bump version and create changelog" steps: - name: Check out uses: actions/checkout@v3 with: fetch-depth: 0 # Fetch all history - name: Configure SSH run: | echo "${{ secrets.DEPLOY_SSH_KEY }}" > ~/.ssh/deploy_key chmod 600 ~/.ssh/deploy_key dos2unix ~/.ssh/deploy_key ssh-agent -a $SSH_AUTH_SOCK > /dev/null ssh-add ~/.ssh/deploy_key - name: Bump the version run: cz bump --changelog --no-verify - name: Push changes run: | git remote add ssh [email protected]:templates/ansible-role.git git pull ssh main git push ssh main git push ssh --tags ``` It assumes that you have `cz` (commitizen) and `dos2unix` installed in your runner. feat(gitea#Skip gitea actions job on changes of some files): Skip gitea actions job on changes of some files There are some expensive CI pipelines that don't need to be run for example if you changed a line in the `README.md`, to skip a pipeline on changes of certain files you can use the `paths-ignore` directive: ```yaml --- name: Ansible Testing "on": push: paths-ignore: - 'meta/**' - Makefile - README.md - renovate.json - CHANGELOG.md - .cz.toml - '.gitea/workflows/**' jobs: test: name: Test runs-on: ubuntu-latest steps: ... ``` The only downside is that if you set this pipeline as required in the branch protection, the merge button will look yellow instead of green when the pipeline is skipped. feat(gotify): Introduce gotify [Gotify](https://github.com/gotify/server) is a simple server for sending and receiving messages in real-time per WebSocket. Not there yet: - [Reactions on the notifications](gotify/server#494) feat(grafana): Introduce grafana [Grafana](https://grafana.com/grafana) is a web application to create dashboards. [Installation](https://grafana.com/docs/grafana/latest/setup-grafana/installation/docker/#run-grafana-via-docker-compose): We're going to install it with docker-compose and connect it to [Authentik](authentik.md). [Create the Authentik connection](https://goauthentik.io/integrations/services/grafana/): Assuming that you have [the terraform authentik provider configured](authentik.md), use the next terraform code: ```hcl variable "grafana_name" { type = string description = "The name shown in the Grafana application." default = "Grafana" } variable "grafana_redirect_uri" { type = string description = "The redirect url configured on Grafana." } variable "grafana_icon" { type = string description = "The icon shown in the Grafana application" default = "/application-icons/grafana.svg" } resource "authentik_application" "grafana" { name = var.grafana_name slug = "grafana" protocol_provider = authentik_provider_oauth2.grafana.id meta_icon = var.grafana_icon lifecycle { ignore_changes = [ # The terraform provider is continuously changing the attribute even though it's set meta_icon, ] } } resource "authentik_provider_oauth2" "grafana" { name = var.grafana_name client_id = "grafana" authorization_flow = data.authentik_flow.default-authorization-flow.id property_mappings = [ data.authentik_scope_mapping.email.id, data.authentik_scope_mapping.openid.id, data.authentik_scope_mapping.profile.id, ] redirect_uris = [ var.grafana_redirect_uri, ] signing_key = data.authentik_certificate_key_pair.default.id access_token_validity = "minutes=120" } data "authentik_certificate_key_pair" "default" { name = "authentik Self-signed Certificate" } data "authentik_flow" "default-authorization-flow" { slug = "default-provider-authorization-implicit-consent" } output "grafana_oauth_id" { value = authentik_provider_oauth2.grafana.client_id } output "grafana_oauth_secret" { value = authentik_provider_oauth2.grafana.client_secret } ``` feat(jellyfin#Jellyfin Desktop): Introduce Jellyfin Desktop - Download the latest deb package from the [releases page](https://github.com/jellyfin/jellyfin-media-player/releases) - Install the dependencies - Run `dpkg -i` If you're on a TV you may want to [enable the TV mode](jellyfin/jellyfin-media-player#11) so that the remote keys work as expected. The play/pause/next/prev won't work until [this issue is solved](jellyfin/jellyfin-media-player#3), but it's not that bad to use the "Ok" and then navigate with the arrow keys. feat(jellyfin#Jellycon): Introduce Jellycon JellyCon is a lightweight Kodi add-on that lets you browse and play media files directly from your Jellyfin server within the Kodi interface. It can be thought of as a thin frontend for a Jellyfin server. It's not very pleasant to use though. feat(kodi): Introduce Kodi [Kodi](https://kodi.tv/) is a entertainment center software. It basically converts your device into a smart tv feat(koel): Introduce Koel [koel](https://koel.dev/) is a personal music streaming server. Note: Use [`mopidy`](mopidy.md) instead There are [docker-compose files](https://github.com/koel/docker) to host the service. Although they behave a little bit weird For example, you need to [specify the DB_PORT](koel/docker#168). It has had several PR to fix it but weren't merged [1](https://github.com/koel/docker/pull/165/files), [2](https://github.com/koel/docker/pull/162/files). The API is [not very well documented](koel/koel#535): - [Here you can see how to authenticate](https://github.com/X-Ryl669/kutr/wiki/Communication-API#authentication) - [Here are the api docs](https://github.com/koel/koel/blob/master/api-docs/api.yaml#L763) feat(zfs#Rename or move a dataset): Rename or move a dataset NOTE: if you want to rename the topmost dataset look at [rename the topmost dataset](#rename-the-topmost-dataset) instead. File systems can be renamed by using the `zfs rename` command. You can perform the following operations: - Change the name of a file system. - Relocate the file system within the ZFS hierarchy. - Change the name of a file system and relocate it within the ZFS hierarchy. The following example uses the `rename` subcommand to rename of a file system from `kustarz` to `kustarz_old`: ```bash zfs rename tank/home/kustarz tank/home/kustarz_old ``` The following example shows how to use zfs `rename` to relocate a file system: ```bash zfs rename tank/home/maybee tank/ws/maybee ``` In this example, the `maybee` file system is relocated from `tank/home` to `tank/ws`. When you relocate a file system through rename, the new location must be within the same pool and it must have enough disk space to hold this new file system. If the new location does not have enough disk space, possibly because it has reached its quota, rename operation fails. The rename operation attempts an unmount/remount sequence for the file system and any descendent file systems. The rename command fails if the operation is unable to unmount an active file system. If this problem occurs, you must forcibly unmount the file system. You'll loose the snapshots though, as explained below. feat(zfs#Rename the topmost dataset): Rename the topmost dataset If you want to rename the topmost dataset you [need to rename the pool too](openzfs/zfs#4681) as these two are tied. ```bash $: zpool status -v pool: tets state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM tets ONLINE 0 0 0 c0d1 ONLINE 0 0 0 c1d0 ONLINE 0 0 0 c1d1 ONLINE 0 0 0 errors: No known data errors ``` To fix this, first export the pool: ```bash $ zpool export tets ``` And then imported it with the correct name: ```bash $ zpool import tets test ``` After the import completed, the pool contains the correct name: ```bash $ zpool status -v pool: test state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM test ONLINE 0 0 0 c0d1 ONLINE 0 0 0 c1d0 ONLINE 0 0 0 c1d1 ONLINE 0 0 0 errors: No known data errors ``` Now you may need to fix the ZFS mountpoints for each dataset ```bash zfs set mountpoint="/opt/zones/[Newmountpoint]" [ZFSPOOL/[ROOTor other filesystem] ``` feat(zfs#Rename or move snapshots): Rename or move snapshots If the dataset has snapshots you need to rename them too. They must be renamed within the same pool and dataset from which they were created though. For example: ```bash zfs rename tank/home/cindys@083006 tank/home/cindys@today ``` In addition, the following shortcut syntax is equivalent to the preceding syntax: ```bash zfs rename tank/home/cindys@083006 today ``` The following snapshot rename operation is not supported because the target pool and file system name are different from the pool and file system where the snapshot was created: ```bash $: zfs rename tank/home/cindys@today pool/home/cindys@saturday cannot rename to 'pool/home/cindys@today': snapshots must be part of same dataset ``` You can recursively rename snapshots by using the `zfs rename -r` command. For example: ```bash $: zfs list NAME USED AVAIL REFER MOUNTPOINT users 270K 16.5G 22K /users users/home 76K 16.5G 22K /users/home users/home@yesterday 0 - 22K - users/home/markm 18K 16.5G 18K /users/home/markm users/home/markm@yesterday 0 - 18K - users/home/marks 18K 16.5G 18K /users/home/marks users/home/marks@yesterday 0 - 18K - users/home/neil 18K 16.5G 18K /users/home/neil users/home/neil@yesterday 0 - 18K - $: zfs rename -r users/home@yesterday @2daysago $: zfs list -r users/home NAME USED AVAIL REFER MOUNTPOINT users/home 76K 16.5G 22K /users/home users/home@2daysago 0 - 22K - users/home/markm 18K 16.5G 18K /users/home/markm users/home/markm@2daysago 0 - 18K - users/home/marks 18K 16.5G 18K /users/home/marks users/home/marks@2daysago 0 - 18K - users/home/neil 18K 16.5G 18K /users/home/neil users/home/neil@2daysago 0 - 18K - ``` feat(zfs#See the differences between two backups): See the differences between two backups To identify the differences between two snapshots, use syntax similar to the following: ```bash $ zfs diff tank/home/tim@snap1 tank/home/tim@snap2 M /tank/home/tim/ + /tank/home/tim/fileB ``` The following table summarizes the file or directory changes that are identified by the `zfs diff` command. | File or Directory Change | Identifier | | --- | --- | | File or directory has been modified or file or directory link has changed | M | | File or directory is present in the older snapshot but not in the more recent snapshot | — | | File or directory is present in the more recent snapshot but not in the older snapshot | + | | File or directory has been renamed | R | feat(zfs#Create a cold backup of a series of datasets): Create a cold backup of a series of datasets If you've used the `-o keyformat=raw -o keylocation=file:///etc/zfs/keys/home.key` arguments to encrypt your datasets you can't use a `keyformat=passphase` encryption on the cold storage device. You need to copy those keys on the disk. One way of doing it is to: - Create a 100M LUKS partition protected with a passphrase where you store the keys. - The rest of the space is left for a partition for the zpool. feat(zfs#Clear a permanent ZFS error in a healthy pool): Clear a permanent ZFS error in a healthy pool Sometimes when you do a `zpool status` you may see that the pool is healthy but that there are "Permanent errors" that may point to files themselves or directly to memory locations. You can read [this long discussion](openzfs/zfs#9705) on what does these permanent errors mean, but what solved the issue for me was to run a new scrub `zpool scrub my_pool` It takes a long time to run, so be patient. feat(zfs#ZFS pool is in suspended mode): ZFS pool is in suspended mode Probably because you've unplugged a device without unmounting it. If you want to remount the device [you can follow these steps](openzfsonosx/zfs#104 (comment)) to symlink the new devfs entries to where zfs thinks the vdev is. That way you can regain access to the pool without a reboot. So if zpool status says the vdev is /dev/disk2s1, but the reattached drive is at disk4, then do the following: ```bash cd /dev sudo rm -f disk2s1 sudo ln -s disk4s1 disk2s1 sudo zpool clear -F WD_1TB sudo zpool export WD_1TB sudo rm disk2s1 sudo zpool import WD_1TB ``` If you don't care about the zpool anymore, sadly your only solution is to [reboot the server](openzfs/zfs#5242). Real ugly, so be careful when you umount zpools. feat(linux_snippets#Get the current git branch): Get the current git branch ```bash git branch --show-current ``` feat(linux_snippets#Install latest version of package from backports): Install latest version of package from backports Add the backports repository: ```bash vi /etc/apt/sources.list.d/bullseye-backports.list ``` ``` deb http://deb.debian.org/debian bullseye-backports main contrib deb-src http://deb.debian.org/debian bullseye-backports main contrib ``` Configure the package to be pulled from backports ```bash vi /etc/apt/preferences.d/90_zfs ``` ``` Package: src:zfs-linux Pin: release n=bullseye-backports Pin-Priority: 990 ``` feat(linux_snippets#Rename multiple files matching a pattern): Rename multiple files matching a pattern There is `rename` that looks nice, but you need to install it. Using only `find` you can do: ```bash find . -name '*yml' -exec bash -c 'echo mv $0 ${0/yml/yaml}' {} \; ``` If it shows what you expect, remove the `echo`. feat(linux_snippets#Force ssh to use password authentication): Force ssh to use password authentication ```bash ssh -o PreferredAuthentications=password -o PubkeyAuthentication=no [email protected] ``` feat(linux_snippets#Do a tail -f with grep): Do a tail -f with grep ```bash tail -f file | grep --line-buffered my_pattern ``` feat(linux_snippets#Check if a program exists in the user's PATH): Check if a program exists in the user's PATH ```bash command -v <the_command> ``` Example use: ```bash if ! command -v <the_command> &> /dev/null then echo "<the_command> could not be found" exit fi ``` feat(mediatracker): Introduce MediaTracker [MediaTracker](https://github.com/bonukai/MediaTracker) is a self hosted media tracker for movies, tv shows, video games, books and audiobooks [Installation](https://github.com/bonukai/MediaTracker#installation): With docker compose: ```yaml version: "3" services: mediatracker: container_name: mediatracker ports: - 7481:7481 volumes: - /home/YOUR_HOME_DIRECTORY/.config/mediatracker/data:/storage - assetsVolume:/assets environment: SERVER_LANG: en TMDB_LANG: en AUDIBLE_LANG: us TZ: Europe/London image: bonukai/mediatracker:latest volumes: assetsVolume: null ``` If you attach more than one docker network the container becomes unreachable :S. Install the jellyfin plugin: They created a [Jellyfin plugin](https://github.com/bonukai/jellyfin-plugin-mediatracker) so that all scrobs are sent automatically to the mediatracker - Add new Repository in Jellyfin (Dashboard -> Plugins -> Repositories -> +) from url `https://raw.githubusercontent.com/bonukai/jellyfin-plugin-mediatracker/main/manifest.json` - Install MediaTracker plugin from Catalogue (Dashboard -> Plugins -> Catalogue) Some tips on usage: - Add the shows you want to watch to the watchlist so that it's easier to find them - When you're ending an episode, click on the episode number on the watchlist element and then rate the episode itself. - You can create public lists to share with the rest of the users, the way to share it though [is a bit archaic so far](bonukai/MediaTracker#527), it's only through the list link, in the interface they won't be able to see it. feat(molecule#Molecule doesn't find the `molecule.yaml` file): Molecule doesn't find the `molecule.yaml` file This is expected default behavior since Molecule searches for scenarios using the `molecule/*/molecule.yml` glob. But if you would like to change the suffix to yaml, you can do that if you set the `MOLECULE_GLOB` environment variable like this: ```bash export MOLECULE_GLOB='molecule/*/molecule.yaml' ``` feat(python_jinja2#Escape jinja expansion on a jinja template): Escape jinja expansion on a jinja template ```jinja {% raw %} Anything in this block is treated as raw text, including {{ curly braces }} and {% other block-like syntax %} {% endraw %} ``` feat(qbittorrent#Tools): Add interesting tools to explore - [qbittools](https://github.com/buroa/qbittools): a feature rich CLI for the management of torrents in qBittorrent. - [qbit_manage](https://github.com/StuffAnThings/qbit_manage): tool will help manage tedious tasks in qBittorrent and automate them. feat(sanoid#Prune snapshots): Prune snapshots If you want to manually prune the snapshots after you tweaked `sanoid.conf` you can run: ```bash sanoid --prune-snapshots ``` feat(sanoid#Send encrypted backups to a encrypted dataset): Send encrypted backups to a encrypted dataset `syncoid`'s default behaviour is to create the destination dataset without encryption so the snapshots are transferred and can be read without encryption. You can check this with the `zfs get encryption,keylocation,keyformat` command both on source and destination. To prevent this from happening you have to [pass the `--sendoptions='w'](jimsalterjrs/sanoid#548) to `syncoid` so that it tells zfs to send a raw stream. If you do so, you also need to [transfer the key file](jimsalterjrs/sanoid#648) to the destination server so that it can do a `zfs loadkey` and then mount the dataset. For example: ```bash server-host:$ sudo zfs list -t filesystem NAME USED AVAIL REFER MOUNTPOINT server_data 232M 38.1G 230M /var/server_data server_data/log 111K 38.1G 111K /var/server_data/log server_data/mail 111K 38.1G 111K /var/server_data/mail server_data/nextcloud 111K 38.1G 111K /var/server_data/nextcloud server_data/postgres 111K 38.1G 111K /var/server_data/postgres server-host:$ sudo zfs get keylocation server_data/nextcloud NAME PROPERTY VALUE SOURCE server_data/nextcloud keylocation file:///root/zfs_dataset_nextcloud_pass local server-host:$ sudo syncoid --recursive --skip-parent --sendoptions=w server_data [email protected]:backup_pool INFO: Sending oldest full snapshot server_data/log@autosnap_2021-06-18_18:33:42_yearly (~ 49 KB) to new target filesystem: 17.0KiB 0:00:00 [1.79MiB/s] [=================================================> ] 34% INFO: Updating new target filesystem with incremental server_data/log@autosnap_2021-06-18_18:33:42_yearly ... syncoid_caedrium.com_2021-06-22:10:12:55 (~ 15 KB): 41.2KiB 0:00:00 [78.4KiB/s] [===================================================================================================================================================] 270% INFO: Sending oldest full snapshot server_data/mail@autosnap_2021-06-18_18:33:42_yearly (~ 49 KB) to new target filesystem: 17.0KiB 0:00:00 [ 921KiB/s] [=================================================> ] 34% INFO: Updating new target filesystem with incremental server_data/mail@autosnap_2021-06-18_18:33:42_yearly ... syncoid_caedrium.com_2021-06-22:10:13:14 (~ 15 KB): 41.2KiB 0:00:00 [49.4KiB/s] [===================================================================================================================================================] 270% INFO: Sending oldest full snapshot server_data/nextcloud@autosnap_2021-06-18_18:33:42_yearly (~ 49 KB) to new target filesystem: 17.0KiB 0:00:00 [ 870KiB/s] [=================================================> ] 34% INFO: Updating new target filesystem with incremental server_data/nextcloud@autosnap_2021-06-18_18:33:42_yearly ... syncoid_caedrium.com_2021-06-22:10:13:42 (~ 15 KB): 41.2KiB 0:00:00 [50.4KiB/s] [===================================================================================================================================================] 270% INFO: Sending oldest full snapshot server_data/postgres@autosnap_2021-06-18_18:33:42_yearly (~ 50 KB) to new target filesystem: 17.0KiB 0:00:00 [1.36MiB/s] [===============================================> ] 33% INFO: Updating new target filesystem with incremental server_data/postgres@autosnap_2021-06-18_18:33:42_yearly ... syncoid_caedrium.com_2021-06-22:10:14:11 (~ 15 KB): 41.2KiB 0:00:00 [48.9KiB/s] [===================================================================================================================================================] 270% server-host:$ sudo scp /root/zfs_dataset_nextcloud_pass 192.168.122.94: ``` ```bash backup-host:$ sudo zfs set keylocation=file:///root/zfs_dataset_nextcloud_pass backup_pool/nextcloud backup-host:$ sudo zfs load-key backup_pool/nextcloud backup-host:$ sudo zfs mount backup_pool/nextcloud ``` If you also want to keep the `encryptionroot` you need to [let zfs take care of the recursion instead of syncoid](jimsalterjrs/sanoid#614). In this case you can't use syncoid's stuff like `--exclude` from the manpage of zfs: ``` -R, --replicate Generate a replication stream package, which will replicate the specified file system, and all descendent file systems, up to the named snapshot. When received, all properties, snap‐ shots, descendent file systems, and clones are preserved. If the -i or -I flags are used in conjunction with the -R flag, an incremental replication stream is generated. The current values of properties, and current snapshot and file system names are set when the stream is received. If the -F flag is specified when this stream is received, snapshots and file systems that do not exist on the sending side are destroyed. If the -R flag is used to send encrypted datasets, then -w must also be specified. ``` In this case this should work: ```bash /sbin/syncoid --recursive --force-delete --sendoptions="Rw" zpool/backups [email protected]:zpool/backups ``` feat(terraform#Create a list of resources based on a list of strings): Create a list of resources based on a list of strings ```hcl variable "subnet_ids" { type = list(string) } resource "aws_instance" "server" { # Create one instance for each subnet count = length(var.subnet_ids) ami = "ami-a1b2c3d4" instance_type = "t2.micro" subnet_id = var.subnet_ids[count.index] tags = { Name = "Server ${count.index}" } } ``` If you want to use this generated list on another resource extracting for example the id you can use ```hcl aws_instance.server.*.id ``` feat(vim#Run a command when opening vim): Run a command when opening vim ```bash nvim -c ':DiffViewOpen' ``` fix(zfs_exporter): Update the alerts to the more curated version feat(zfs_exporter#Useful inhibits): Useful inhibits Some you may want to inhibit some of these rules for some of your datasets. These subsections should be added to the `alertmanager.yml` file under the `inhibit_rules` field. Ignore snapshots on some datasets: Sometimes you don't want to do snapshots on a dataset ```yaml - target_matchers: - alertname = ZfsDatasetWithNoSnapshotsError - hostname = my_server_1 - filesystem = tmp ``` Ignore snapshots growth: Sometimes you don't mind if the size of the data saved in the filesystems doesn't change too much between snapshots doesn't change much specially in the most frequent backups because you prefer to keep the backup cadence. It's interesting to have the alert though so that you can get notified of the datasets that don't change that much so you can tweak your backup policy (even if zfs snapshots are almost free). ```yaml - target_matchers: - alertname =~ "ZfsSnapshotType(Frequently|Hourly)SizeError" - filesystem =~ "(media/(docs|music))" ```
and others
when a pool is encountering i/o errors (because of a software bug or hardware issue/vdev gone missing), zol suspends the pool until it is brought to functional state.
most of the time this is impossible and user must hard reboot the server (hardware button or echo b > sysrq-trigger)
the forced reboot does not export the pool but simply decouples the zfs code from the storage devices.
please add this feature natively so I can "unload" a pool without commiting the export operation to disk, to further be able to unload the zfs modules and try the latest git master version (for example) without having to reboot the os.
The text was updated successfully, but these errors were encountered: