diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml new file mode 100644 index 0000000000..094c57d562 --- /dev/null +++ b/.gitlab-ci.yml @@ -0,0 +1,6 @@ +include: + - project: 'QubesOS/qubes-continuous-integration' + file: '/gitlab-website.yml' + +build:doc: + extends: .website diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index f73117d06d..1e2bd856e4 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,10 +1,12 @@ -Contributing to `qubes-doc` -=========================== +# Contributing to `qubes-doc` Thank you for your interest in contributing to `qubes-doc`, the Qubes OS -Project's dedicated documentation repository! Please take a moment to read our -[Documentation Guidelines] before you begin writing. These guidelines are -important to maintaining high quality documentation, and following them will -increase the likelihood that your contribution will be accepted. +Project's dedicated documentation repository! Please see [how to edit the +documentation](https://www.qubes-os.org/doc/how-to-edit-the-documentation/) for +detailed contribution instructions. -[Documentation Guidelines]: https://www.qubes-os.org/doc/doc-guidelines/ +In addition, please take a moment to read our [documentation style +guide](https://www.qubes-os.org/doc/documentation-style-guide/) before +contributing. These guidelines are important to maintaining high standards of +quality, and following them will increase the likelihood that your contribution +will be accepted. diff --git a/README.md b/README.md index fffeec8d17..0ab6702532 100644 --- a/README.md +++ b/README.md @@ -1,17 +1,11 @@ -Qubes OS Documentation -====================== +# Qubes OS Documentation Canonical URL: https://www.qubes-os.org/doc/ -All [Qubes OS Project] documentation pages are stored as plain text -files in this dedicated repository. By cloning and regularly pulling from -this repo, users can maintain their own up-to-date offline copy of all -Qubes documentation rather than relying solely on the Web. - -For more information about the documentation, including how to contribute, -please see the [Documentation Guidelines]. - - -[Qubes OS Project]: https://github.com/QubesOS -[documentation guidelines]: https://www.qubes-os.org/doc/doc-guidelines/ +All [Qubes OS Project](https://github.com/QubesOS) documentation pages are +stored as plain text files in this dedicated repository. By cloning and +regularly pulling from this repo, users can maintain their own up-to-date +offline copy of all Qubes documentation rather than relying solely on the Web. +To contribute, please see [how to edit the +documentation](https://www.qubes-os.org/doc/how-to-edit-the-documentation/). diff --git a/_dev/Makefile b/_dev/Makefile index 8177cd009c..642e3223b4 100644 --- a/_dev/Makefile +++ b/_dev/Makefile @@ -9,7 +9,7 @@ BUILDDIR = _build # User-friendly check for sphinx-build ifeq ($(shell which $(SPHINXBUILD) >/dev/null 2>&1; echo $$?), 1) -$(error The '$(SPHINXBUILD)' command was not found. Make sure you have Sphinx installed, then set the SPHINXBUILD environment variable to point to the full path of the '$(SPHINXBUILD)' executable. Alternatively you can add the directory with the executable to your PATH. If you don't have Sphinx installed, grab it from http://sphinx-doc.org/) +$(error The '$(SPHINXBUILD)' command was not found. Make sure you have Sphinx installed, then set the SPHINXBUILD environment variable to point to the full path of the '$(SPHINXBUILD)' executable. Alternatively you can add the directory with the executable to your PATH. If you don't have Sphinx installed, grab it from https://www.sphinx-doc.org/) endif # Internal variables. diff --git a/_dev/_build/.gitignore b/_dev/_build/.gitignore deleted file mode 100644 index e69de29bb2..0000000000 diff --git a/developer/building/development-workflow.md b/developer/building/development-workflow.md index 3f9b2fcd4a..0146fe3ae5 100644 --- a/developer/building/development-workflow.md +++ b/developer/building/development-workflow.md @@ -1,25 +1,23 @@ --- +lang: en layout: doc -title: Development Workflow permalink: /doc/development-workflow/ redirect_from: - /en/doc/development-workflow/ - /doc/DevelopmentWorkflow/ - /wiki/DevelopmentWorkflow/ +ref: 66 +title: Development workflow --- -Development Workflow -==================== - A workflow for developing Qubes OS+ First things first, setup [QubesBuilder](/doc/qubes-builder/). This guide assumes you're using qubes-builder to build Qubes. -Repositories and committing Code --------------------------------- +# Repositories and committing Code -Qubes is split into a bunch of git repos. This are all contained in the +Qubes is split into a bunch of git repos. These are all contained in the `qubes-src` directory under qubes-builder. Subdirectories there are separate components, stored in separate git repositories. @@ -117,7 +115,7 @@ cd ../.. vi series.conf ~~~ -#### Building RPMS +#### Building RPMs TODO: Is this step generic for all subsystems? @@ -130,13 +128,13 @@ distinguish between different versions of the same package. You might want to take a moment here to review (git diff, git status), commit your changes locally. -To actually build RPMS, in qubes-builder: +To actually build RPMs, in qubes-builder: ~~~ make linux-kernel ~~~ -RPMS will appear in qubes-src/linux-kernel/pkgs/fc20/x86\_64: +RPMs will appear in qubes-src/linux-kernel/pkgs/fc20/x86\_64: ~~~ -rw-rw-r-- 1 user user 42996126 Nov 17 04:08 kernel-3.4.18-1debug20121116c.pvops.qubes.x86_64.rpm @@ -149,25 +147,24 @@ RPMS will appear in qubes-src/linux-kernel/pkgs/fc20/x86\_64: ### Useful [QubesBuilder](/doc/qubes-builder/) commands -1. `make check` - will check if all the code was committed into repository and +1. `make check` - will check if all the code was committed into repository and if all repository are tagged with signed tag. -2. `make show-vtags` - show version of each component (based on git tags) - +2. `make show-vtags` - show version of each component (based on git tags) - mostly useful just before building ISO. **Note:** this will not show version -for components containing changes since last version tag -3. `make push` - push change from **all** repositories to git server. You must +for components containing changes since last version tag. +3. `make push` - push change from **all** repositories to git server. You must set proper remotes (see above) for all repositories first. -4. `make prepare-merge` - fetch changes from remote repositories (can be +4. `make prepare-merge` - fetch changes from remote repositories (can be specified on commandline via GIT\_SUBDIR or GIT\_REMOTE vars), (optionally) verify tags and show the changes. This do not merge the changes - there are left for review as FETCH\_HEAD ref. You can merge them using `git merge FETCH_HEAD` (in each repo directory). Or `make do-merge` to merge all of them. -Copying Code to dom0 --------------------- +## Copying Code to dom0 When developing it is convenient to be able to rapidly test changes. Assuming you're developing Qubes on Qubes, you should be working in a special VM for -Qubes and occasionally you will want to transfer code or rpms back to dom0 for +Qubes and occasionally you will want to transfer code or RPMs back to dom0 for testing. Here are some handy scripts Marek has shared to facilitate this. @@ -309,7 +306,6 @@ to `testbuilder` VM. Otherwise it creates remote pointing at github account of the same name. In any case it points at repository matching current directory name. - ## Sending packages to different VM Other useful script(s) can be used to setup local package repository hosted in @@ -325,7 +321,7 @@ current and current-testing). ### RPM packages - yum repo -In source VM, grab [linux-yum] repository (below is assumed you've made it in +In source VM, grab [linux-yum](https://github.com/QubesOS/qubes-linux-yum) repository (below is assumed you've made it in `~/repo-yum-upload` directory) and replace `update_repo.sh` script with: ~~~ @@ -341,7 +337,7 @@ find -type f -name '*.rpm' -delete qrexec-client-vm $VMNAME local.UpdateYum ~~~ -In target VM, setup actual yum repository (also based on [linux-yum], this time +In target VM, setup actual yum repository (also based on [linux-yum](https://github.com/QubesOS/qubes-linux-yum), this time without modifications). You will also need to setup some gpg key for signing packages (it is possible to force yum to install unsigned packages, but it isn't possible for `qubes-dom0-update` tool). Fill `~/.rpmmacros` with @@ -389,7 +385,7 @@ Of course you will also need to setup qrexec policy in dom0 If you want to access the repository from network, you need to setup HTTP server serving it, and configure the system to let other machines actually reach this HTTP server. You can use for example using [port -forwarding][port-forwarding] or setting up Tor hidden service. Configuration +forwarding](/doc/firewall/#port-forwarding-to-a-qube-from-the-outside-world) or setting up Tor hidden service. Configuration details of those services are outside of the scope of this page. Usage: setup `builder.conf` in source VM to use your dummy-uploader repository: @@ -421,10 +417,10 @@ Remember to also import gpg public key using `rpm --import`. Steps are mostly the same as in the case of yum repo. The only details that differ: - - use [linux-deb] instead of [linux-yum] as a base - both in source and target VM - - use different `update_repo.sh` script in source VM (below) - - use `local.UpdateApt` qrexec service in target VM (code below) - - in target VM additionally place `update-local-repo.sh` script in repository dir (code below) +- use [linux-deb](https://github.com/QubesOS/qubes-linux-deb) instead of [linux-yum](https://github.com/QubesOS/qubes-linux-yum) as a base - both in source and target VM +- use different `update_repo.sh` script in source VM (below) +- use `local.UpdateApt` qrexec service in target VM (code below) +- in target VM additionally place `update-local-repo.sh` script in repository dir (code below) `update_repo.sh` script: @@ -540,7 +536,3 @@ Usage: add this line to `/etc/apt/sources.list` on test machine (adjust host and ~~~ deb http://local-test.lan/linux-deb/r3.1 jessie-unstable main ~~~ - -[port-forwarding]: /doc/firewall/#port-forwarding-to-a-qube-from-the-outside-world -[linux-yum]: https://github.com/QubesOS/qubes-linux-yum -[linux-deb]: https://github.com/QubesOS/qubes-linux-deb diff --git a/developer/building/qubes-builder-details.md b/developer/building/qubes-builder-details.md index f1f85b3e7e..42b3ddd0af 100644 --- a/developer/building/qubes-builder-details.md +++ b/developer/building/qubes-builder-details.md @@ -1,16 +1,15 @@ --- +lang: en layout: doc -title: Qubes Builder Details permalink: /doc/qubes-builder-details/ redirect_from: - /en/doc/qubes-builder-details/ - /doc/QubesBuilderDetails/ - /wiki/QubesBuilderDetails/ +ref: 65 +title: Qubes builder details --- -[QubesBuilder](/doc/qubes-builder/) "API" -======================================== - Components Makefile.builder file -------------------------------- @@ -18,28 +17,28 @@ Components Makefile.builder file Variables for Linux build: -- `RPM_SPEC_FILES` List (space separated) of spec files for RPM package build. Path should be relative to component root directory. [QubesBuilder](/doc/qubes-builder/) will install all BuildRequires (in chroot environment) before the build. In most Qubes components all spec files are kept in *rpm\_spec* directory. This is mainly used for Fedora packages build. -- `ARCH_BUILD_DIRS` List (space separated) of directories with PKGBUILD files for Archlinux package build. Similar to RPM build, [QubesBuilder](/doc/qubes-builder/) will install all makedepends, then build the package. +- `RPM_SPEC_FILES` List (space separated) of spec files for RPM package build. Path should be relative to component root directory. [QubesBuilder](/doc/qubes-builder/) will install all BuildRequires (in chroot environment) before the build. In most Qubes components all spec files are kept in *rpm\_spec* directory. This is mainly used for Fedora packages build. +- `ARCH_BUILD_DIRS` List (space separated) of directories with PKGBUILD files for Archlinux package build. Similar to RPM build, [QubesBuilder](/doc/qubes-builder/) will install all makedepends, then build the package. Most components uses *archlinux* directory for this purpose, so its good to keep this style. Variables for Windows build: -- `WIN_COMPILER` Choose which compiler should be used for this component, thus which build scripts. Currently two options available: - - `WDK` - Windows Driver Kit (default). Command used to build: *build -cZg*. - - `mingw` - MinGW (Windows gcc port). Command used to build: *make all* -- `WIN_SOURCE_SUBDIRS` List of directories in which above command should be run. In most cases it will be only one entry: current directory (*.*). -- `WIN_PREBUILD_CMD` Command to run before build, mostly useful for WDK build (in mingw case, you can use makefile for this purpose). Can be used to set some variables, preprocess some files etc. -- `WIN_SIGN_CMD` Command used to sign resulting binaries. Note that default value is *sign.bat*. If you don't want to sign binaries, specify some placeholder here (eg. *true*). Check existing components (eg. vmm-xen-windows-pvdrivers) for example scripts. This command will be run with certain environment variables: - - `CERT_FILENAME` Path to key file (pfx format) - - `CERT_PASSWORD` Key password - - `CERT_PUBLIC_FILENAME` Certificate path in the case of self-signed cert - - `CERT_CROSS_CERT_FILENAME` Certificate path in the case of correct autheticode cert - - `SIGNTOOL` Path to signtool -- `WIN_PACKAGE_CMD` Command used to produce installation package (msi or msm). Default value is *wix.bat*, similar to above - use *true* if you don't want this command. -- `WIN_OUTPUT_HEADERS` Directory (relative to `WIN_SOURCE_SUBDIRS` element) with public headers of the package - for use in other components. -- `WIN_OUTPUT_LIBS` Directory (relative to `WIN_SOURCE_SUBDIRS` element) with libraries (both DLL and implib) of the package - for use in other components. Note that [QubesBuilder](/doc/qubes-builder/) will copy files specified as *\$(WIN\_OUTPUT\_LIBS)/\*/\** to match WDK directory layout (*\/\/\*), so you in mingw build you need to place libraries in some additional subdirectory. -- `WIN_BUILD_DEPS` List of components required to build this one. [QubesBuilder](/doc/qubes-builder/) will copy files specified with `WIN_OUTPUT_HEADERS` and `WIN_OUTPUT_LIBS` of those components to some directory and provide its path with `QUBES_INCLUDES` and `QUBES_LIBS` variables. Use those variables in your build scripts (*sources* or *Makefile* - depending on selected compiler). You can assume that the variables are always set and directories always exists, even if empty. +- `WIN_COMPILER` Choose which compiler should be used for this component, thus which build scripts. Currently two options available: + - `WDK` - Windows Driver Kit (default). Command used to build: *build -cZg*. + - `mingw` - MinGW (Windows gcc port). Command used to build: *make all* +- `WIN_SOURCE_SUBDIRS` List of directories in which above command should be run. In most cases it will be only one entry: current directory (*.*). +- `WIN_PREBUILD_CMD` Command to run before build, mostly useful for WDK build (in mingw case, you can use makefile for this purpose). Can be used to set some variables, preprocess some files etc. +- `WIN_SIGN_CMD` Command used to sign resulting binaries. Note that default value is *sign.bat*. If you don't want to sign binaries, specify some placeholder here (eg. *true*). Check existing components (eg. vmm-xen-windows-pvdrivers) for example scripts. This command will be run with certain environment variables: + - `CERT_FILENAME` Path to key file (pfx format) + - `CERT_PASSWORD` Key password + - `CERT_PUBLIC_FILENAME` Certificate path in the case of self-signed cert + - `CERT_CROSS_CERT_FILENAME` Certificate path in the case of correct autheticode cert + - `SIGNTOOL` Path to signtool +- `WIN_PACKAGE_CMD` Command used to produce installation package (msi or msm). Default value is *wix.bat*, similar to above - use *true* if you don't want this command. +- `WIN_OUTPUT_HEADERS` Directory (relative to `WIN_SOURCE_SUBDIRS` element) with public headers of the package - for use in other components. +- `WIN_OUTPUT_LIBS` Directory (relative to `WIN_SOURCE_SUBDIRS` element) with libraries (both DLL and implib) of the package - for use in other components. Note that [QubesBuilder](/doc/qubes-builder/) will copy files specified as *\$(WIN\_OUTPUT\_LIBS)/\*/\** to match WDK directory layout (*\/\/\*), so you in mingw build you need to place libraries in some additional subdirectory. +- `WIN_BUILD_DEPS` List of components required to build this one. [QubesBuilder](/doc/qubes-builder/) will copy files specified with `WIN_OUTPUT_HEADERS` and `WIN_OUTPUT_LIBS` of those components to some directory and provide its path with `QUBES_INCLUDES` and `QUBES_LIBS` variables. Use those variables in your build scripts (*sources* or *Makefile* - depending on selected compiler). You can assume that the variables are always set and directories always exists, even if empty. builder.conf settings --------------------- @@ -51,4 +50,4 @@ Most settings are documented in *builder.conf.default* file, which can be used a Notes ----- -* For a list of custom TemplateVMs available in QubesBuilder look at [Supported Versions page](https://www.qubes-os.org/doc/supported-versions/). +- For a list of custom TemplateVMs available in QubesBuilder look at [Supported Versions page](/doc/supported-releases/). diff --git a/developer/building/qubes-builder.md b/developer/building/qubes-builder.md index 008c00d957..b50ce540a5 100644 --- a/developer/building/qubes-builder.md +++ b/developer/building/qubes-builder.md @@ -1,17 +1,17 @@ --- +lang: en layout: doc -title: Qubes Builder permalink: /doc/qubes-builder/ redirect_from: - /en/doc/qubes-builder/ - /doc/QubesBuilder/ - /wiki/QubesBuilder/ +ref: 64 +title: Qubes builder --- **Note: See [ISO building instructions](/doc/qubes-iso-building/) for a streamlined overview on how to use the build system.** -Building Qubes from scratch -=========================== We have a fully automated build system for Qubes, that downloads, builds and packages all the Qubes components, and finally should spit out a ready-to-use @@ -19,27 +19,29 @@ installation ISO, all in a [secure](/news/2016/05/30/build-security/) way. In order to use it, you should use an rpm-based distro, like Fedora :), and should ensure the following packages are installed: -- sudo -- gnupg -- git -- createrepo -- rpm-build -- make -- wget -- rpmdevtools -- python3-sh -- dialog -- rpm-sign -- dpkg-dev -- debootstrap -- python3-pyyaml -- devscripts -- perl-Digest-MD5 -- perl-Digest-SHA +- sudo +- gnupg +- git +- createrepo +- rpm-build +- make +- wget +- rpmdevtools +- python3-sh +- dialog +- rpm-sign +- dpkg-dev +- debootstrap +- python3-pyyaml +- devscripts +- perl-Digest-MD5 +- perl-Digest-SHA Usually you can install those packages by just issuing: - sudo dnf install gnupg git createrepo rpm-build make wget rpmdevtools python3-sh dialog rpm-sign dpkg-dev debootstrap python3-pyyaml devscripts perl-Digest-MD5 perl-Digest-SHA +```shell +sudo dnf install gnupg git createrepo rpm-build make wget rpmdevtools python3-sh dialog rpm-sign dpkg-dev debootstrap python3-pyyaml devscripts perl-Digest-MD5 perl-Digest-SHA +``` The build system creates build environments in chroots and so no other packages are needed on the host. All files created by the build system are contained within the qubes-builder directory. @@ -49,95 +51,107 @@ The build system is configured via builder.conf file. You can use the setup.sh script to create and modify this file. Alternatively, you can copy the provided default builder.conf, and modify it as needed, e.g.: - cp example-configs/qubes-os-master.conf builder.conf - # edit the builder.conf file and set the following variables: - NO_SIGN=1 +```shell +cp example-configs/qubes-os-master.conf builder.conf +# edit the builder.conf file and set the following variables: +NO_SIGN=1 +``` -One additional useful requirement is that 'sudo root' must work without any prompt, which is default on most distros (e.g. 'sudo bash' brings you the root shell without asking for any password). +One additional useful requirement is that 'sudo root' must work without any prompt, which is default on most distros (e.g. 'sudo bash' brings you the root shell without asking for any password). This is important as the builder needs to switch to root and then back to user several times during the build process. -Additionally, if building with signing enabled (NO\_SIGN is not set), you must adjust \~/.rpmmacro file so that it points to the GPG key used for package signing, e.g.: +Additionally, if building with signing enabled (NO\_SIGN is not set), you must adjust `\~/.rpmmacros` file so that it points to the GPG key used for package signing, e.g.: - %_signature gpg - %_gpg_path /home/user/.gnupg - %_gpg_name AC1BF9B3 # <-- Key ID used for signing +```bash +%_signature gpg +%_gpg_path /home/user/.gnupg +%_gpg_name AC1BF9B3 # <-- Key ID used for signing +``` It is also recommended that you use an empty passphrase for the private key used for signing. Contrary to a popular belief, this doesn't affect your key or sources security -- if somebody compromised your system, then the game is over anyway, whether you have used an additional passphrase for the key or not. So, to build Qubes you would do: - # Import the Qubes master key - gpg --recv-keys 0xDDFA1A3E36879494 - - # Verify its fingerprint, set as 'trusted'. - # This is described here: - # https://www.qubes-os.org/doc/VerifyingSignatures - - wget https://keys.qubes-os.org/keys/qubes-developers-keys.asc - gpg --import qubes-developers-keys.asc - - git clone git://github.com/QubesOS/qubes-builder.git qubes-builder - cd qubes-builder - - # Verify its integrity: - git tag -v `git describe` - - cp example-configs/qubes-os-master.conf builder.conf - # edit the builder.conf file and set the following variables: - # NO_SIGN="1" - - # Download all components: - - make get-sources - - # And now to build all Qubes rpms (this will take a few hours): - - make qubes - - # ... and then to build the ISO - - make iso +```shell +# Import the Qubes master key +gpg --recv-keys 0xDDFA1A3E36879494 + +# Verify its fingerprint, set as 'trusted'. +# This is described here: +# https://www.qubes-os.org/doc/VerifyingSignatures + +wget https://keys.qubes-os.org/keys/qubes-developers-keys.asc +gpg --import qubes-developers-keys.asc + +git clone git://github.com/QubesOS/qubes-builder.git qubes-builder +cd qubes-builder + +# Verify its integrity: +git tag -v `git describe` + +cp example-configs/qubes-os-master.conf builder.conf +# edit the builder.conf file and set the following variables: +# NO_SIGN="1" + +# Download all components: + +make get-sources + +# And now to build all Qubes RPMs (this will take a few hours): + +make qubes + +# ... and then to build the ISO + +make iso +``` And this should produce a shiny new ISO. You can also build selected component separately. Eg. to compile only gui virtualization agent/daemon: - make gui-daemon +```shell +make gui-daemon +``` -You can get a full list from make help. +You can get a full list from make help. -Making customized build ------------------------ +## Making customized build ### Manual source modification You can also modify sources somehow if you wish. Here are some basic steps: -1. Download qubes-builder as described above (if you want to use marmarek's branches, you should also download qubes-builder from his repo - replace 'QubesOS' with 'marmarek' in above git clone command) -2. Edit builder.conf (still the same as above), some useful additions: - - You can also set GIT\_PREFIX="marmarek/qubes-" to use marmarek's repo instead of "mainstream" - it contains newer (but less tested) versions - -3. Download unmodified sources +1. Download qubes-builder as described above (if you want to use marmarek's branches, you should also download qubes-builder from his repo - replace 'QubesOS' with 'marmarek' in above git clone command) +2. Edit builder.conf (still the same as above), some useful additions: + - You can also set GIT\_PREFIX="marmarek/qubes-" to use marmarek's repo instead of "mainstream" - it contains newer (but less tested) versions +3. Download unmodified sources - make get-sources + ```shell + make get-sources + ``` -4. **Make your modifications here** +4. **Make your modifications here** -5. Build the Qubes +5. Build the Qubes `make qubes` actually is just meta target which builds all required components in correct order. The list of components is configured in builder.conf. You can also check the current value at the end of `make - help`, or using `make build-info`. + help`, or using `make build-info`. 6. `get-sources` is already done, so continue with the next one. You can skip `sign-all` if you've disabled signing - make vmm-xen core-admin linux-kernel gui-daemon template desktop-linux-kde installer-qubes-os manager linux-dom0-updates + ```shell + make vmm-xen core-admin linux-kernel gui-daemon template desktop-linux-kde installer-qubes-os manager linux-dom0-updates + ``` -1. build iso installation image +7. build iso installation image - make iso + ```shell + make iso + ``` ### Use pre-built Qubes packages @@ -146,19 +160,25 @@ This is especially true for `gcc`, which takes several hours to build. Before creating the `chroot`, add this to your `builder.conf`: - USE_QUBES_REPO_VERSION = $(RELEASE) +``` +USE_QUBES_REPO_VERSION = $(RELEASE) +``` It will add the 'current' Qubes repository to your `chroot` environment. Next, specify which components (`gcc`, for example) you want to download instead of compiling: - COMPONENTS := $(filter-out gcc,$(COMPONENTS)) +``` +COMPONENTS := $(filter-out gcc,$(COMPONENTS)) +``` Alternatively, edit the actual COMPONENTS list which is defined in the included version-dependent config from example-configs (see series of include directives near the beginning of `builder.conf`). This way, you can build only the packages in which you are interested. If you also want to use the 'current-testing' repository, add this to your configuration: - USE_QUBES_REPO_TESTING = 1 +``` +USE_QUBES_REPO_TESTING = 1 +``` In the case of an existing `chroot`, for mock-enabled builds, this works immediately because `chroot` is constructed each time separately. For legacy builds, it will not add the necessary configuration into the build environment unless a specific builder change or configuration would force rebuilding chroot. @@ -166,26 +186,26 @@ For legacy builds, it will not add the necessary configuration into the build en Also, once enabled, disabling this setting will not disable repositories in relevant chroots. And even if it did, there could be some leftover packages installed from those repos (which may or may not be desirable). -**Note** +**Note** If you are building Ubuntu templates, you cannot use this option. This is because Qubes does not provide official packages for Ubuntu templates. -Code verification keys management ---------------------------------- +## Code verification keys management [QubesBuilder](/doc/qubes-builder/) by default verifies signed tags on every downloaded code. Public keys used for that are stored in `keyrings/git`. By default Qubes developers' keys are imported automatically, but if you need some additional keys (for example your own), you can add them using: - GNUPGHOME=$PWD/keyrings/git gpg --import /path/to/key.asc - GNUPGHOME=$PWD/keyrings/git gpg --edit-key ID_OF_JUST_IMPORTED_KEY - # here use "trust" command to set key fully or ultimately trusted - only those keys are accepted by QubesBuilder +```shell +GNUPGHOME=$PWD/keyrings/git gpg --import /path/to/key.asc +GNUPGHOME=$PWD/keyrings/git gpg --edit-key ID_OF_JUST_IMPORTED_KEY +# here use "trust" command to set key fully or ultimately trusted - only those keys are accepted by QubesBuilder +``` All Qubes developers' keys are signed by the Qubes Master Signing Key (which is set as ultimately trusted key), so are trusted automatically. If you are the owner of Master key and want to revoke such signature, use the `revsig` gpg key edit command and update the key in qubes-developers-keys.asc - now the key will be no longer trusted (unless manually set as such). -Further information -------------------- +## Further information For advanced [QubesBuilder](/doc/qubes-builder/) use, and preparing sources, take a look at [QubesBuilderDetails](/doc/qubes-builder-details/) page, or [QubesBuilder's doc directory](https://github.com/marmarek/qubes-builder/tree/master/doc). diff --git a/developer/building/qubes-iso-building.md b/developer/building/qubes-iso-building.md index f09d7af015..f6e6a305c5 100644 --- a/developer/building/qubes-iso-building.md +++ b/developer/building/qubes-iso-building.md @@ -1,6 +1,6 @@ --- +lang: en layout: doc -title: Qubes ISO Building permalink: /doc/qubes-iso-building/ redirect_from: - /doc/qubes-r3-building/ @@ -8,11 +8,10 @@ redirect_from: - /en/doc/qubes-iso-building/ - /doc/QubesR3Building/ - /wiki/QubesR3Building/ +ref: 63 +title: Qubes ISO building --- -Building Qubes OS ISO -========================= - Build Environment ----------------- @@ -42,7 +41,7 @@ Get the necessary keys to verify the sources (run these and other commands below ~~~ wget https://keys.qubes-os.org/keys/qubes-master-signing-key.asc -gpg --import qubes-master-signing-key.asc +gpg --import qubes-master-signing-key.asc gpg --edit-key 36879494 fpr # Verify fingerprint! See Note below! @@ -53,7 +52,7 @@ gpg --import qubes-developers-keys.asc ~~~ **Note** In the above process, we do *not* rely on the security of our server (keys.qubes-os.org) nor the connection (ssl, cert) -- we only rely on you getting the Qubes Master Signing Key fingerprint *somehow* and ensuring they match! -See [Verifying Signatures](/security/verifying-signatures/#1-get-the-qubes-master-signing-key-and-verify-its-authenticity) for verification sources. +See [verifying signatures](/security/verifying-signatures/#how-to-import-and-authenticate-the-qubes-master-signing-key) for verification sources. Now let's bootstrap the builder. Unfortunately, the builder cannot verify itself (the classic Chicken and Egg problem), so we need to verify the signature manually: @@ -68,7 +67,6 @@ git tag -v `git describe` Assuming the verification went fine, we're good to go with all the rest without ever thinking more about verifying digital signatures on all the rest of the components, apart from an additional step if doing a non-scripted build. The builder will do that for us for each component, every time we build, even for all auxiliary files (e.g. Xen or Linux kernel sources). - Build using setup script ----------------- @@ -128,7 +126,6 @@ make iso Enjoy your new ISO! - Build using manual steps ----------------- diff --git a/developer/building/qubes-template-configs.md b/developer/building/qubes-template-configs.md new file mode 100644 index 0000000000..71139116d7 --- /dev/null +++ b/developer/building/qubes-template-configs.md @@ -0,0 +1,8 @@ +--- +lang: en +layout: doc +permalink: /doc/qubes-template-configs/ +redirect_to: https://github.com/QubesOS/qubes-template-configs +ref: 248 +title: Qubes template configs +--- diff --git a/developer/code/code-signing.md b/developer/code/code-signing.md index e826561e7f..268894f589 100644 --- a/developer/code/code-signing.md +++ b/developer/code/code-signing.md @@ -1,21 +1,18 @@ --- +lang: en layout: doc -title: Code Signing permalink: /doc/code-signing/ +ref: 51 +title: Code signing --- -Code Signing -============ +All contributions to the Qubes OS [source code](/doc/source-code/) must be cryptographically signed by the author's PGP key. -All contributions to the Qubes OS [source code] must be cryptographically signed by the author's PGP key. - - -Generating a Key ----------------- +## Generating a Key (Note: If you already have a PGP key, you may skip this step.) -Alex Cabal has written an excellent [guide] on creating a PGP keypair. +Alex Cabal has written an excellent [guide](https://alexcabal.com/creating-the-perfect-gpg-keypair/) on creating a PGP keypair. Below, we reproduce just the minimum steps in generating a keypair using GnuPG. Please read Cabal's full guide for further important details. @@ -56,7 +53,7 @@ Real name: Bilbo Baggins E-mail address: bilbo@shire.org -Comment: +Comment: You selected this USER-ID: "Bilbo Baggins " @@ -78,8 +75,7 @@ uid Bilbo Baggins sub 4096R/69B0EA85 2013-03-13 ~~~ -Upload the Key --------------- +## Upload the Key For others to find the public key, please upload it to a server. @@ -88,8 +84,7 @@ $ gpg --send-keys --keyserver pool.sks-keyservers.net 69B0EA85 gpg: sending key 488BA441 to hkp server pool.sks-keyservers.net ``` -Using PGP with Git ------------------- +## Using PGP with Git If you're submitting a patch via GitHub (or a similar Git server), please sign your Git commits. @@ -109,10 +104,10 @@ your Git commits. Alternatively, manually specify when a commit is to be signed: ~~~ - commit -S + git commit -S ~~~ -3. (Optional) Create signed tags. +3. (Optional) Create signed tags. Signed commits are totally sufficient to contribute to Qubes OS. However, if you have commits which are not signed and you do not want to change them, you can create a signed tag for the commit and push it before the check. @@ -141,25 +136,22 @@ your Git commits. vtag = !git tag -v `git describe` ~~~ -GitHub Signature Verification (optional) ----------------------------------------- +## GitHub Signature Verification (optional) GitHub shows a green `Verified` label indicating that the GPG signature could be verified using any of the contributor’s GPG keys uploaded to GitHub. You can upload your public key on GitHub by adding your public GPG key on the [New GPG -key][GitHub New GPG key] under the [SSH GPG keys page][GitHub SSH GPG keys -page]. +key](https://github.com/settings/gpg/new) under the [SSH GPG keys page](https://github.com/settings/keys). -Code Signature Checks ---------------------- +## Code Signature Checks -The [signature-checker] checks if code contributions are signed. -Although GitHub adds a little green `Verified` button next to the commit, the [signature-checker] uses this algorithm to check if a commit is correctly signed: +The [signature-checker](https://github.com/marmarek/signature-checker) checks if code contributions are signed. +Although GitHub adds a little green `Verified` button next to the commit, the [signature-checker](https://github.com/marmarek/signature-checker) uses this algorithm to check if a commit is correctly signed: -1. Is the commit signed? +1. Is the commit signed? If the commit is not signed, you can see the message > policy/qubesos/code-signing — No signature found -2. If the commit is signed, the key is downloaded from a GPG key server. +2. If the commit is signed, the key is downloaded from a GPG key server. If you can see the following error message, please check if you have uploaded the key to a key server. > policy/qubesos/code-signing — Unable to verify (no valid key found) @@ -169,23 +161,29 @@ Although GitHub adds a little green `Verified` button next to the commit, the [s In this case, you have several options to sign the commit: -1. Amend the commit and replace it with a signed commit. +1. Amend the commit and replace it with a signed commit. You can use this command to create a new signed commit: + ``` git commit --amend -S ``` + This also rewrites the commit so you need to push it forcefully: + ``` git push -f ``` -2. Create a signed tag for the unsigned commit. + +2. Create a signed tag for the unsigned commit. If the commit is back in history and you do not want to change it, you can create a signed tag for this commit and push the signature. You can use the alias from above: + ``` git checkout git spush ``` + Now, the signature checker needs to re-check the signature. Please comment on the pull request that you would like to have the signatures checked again. @@ -193,25 +191,13 @@ In this case, you have several options to sign the commit: > policy/qubesos/code-signing — Unable to verify (no valid key found) -This means that the [signature-checker] has found a signature for the commit +This means that the [signature-checker](https://github.com/marmarek/signature-checker) has found a signature for the commit but is not able to verify it using the any key available. This might be that you forgot to upload the key to a key server. Please upload it. +## Using PGP with Email -Using PGP with Email --------------------- - -If you're submitting a patch by emailing the [developer mailing list], simply sign your email with your PGP key. -One good way to do this is with a program like [Enigmail]. +If you're submitting a patch by emailing the [developer mailing list](/support/#qubes-devel), simply sign your email with your PGP key. +One good way to do this is with a program like [Enigmail](https://www.enigmail.net/). Enigmail is a security addon for the Mozilla Thunderbird email client that allows you to easily digitally encrypt and sign your emails. - - -[guide]: https://alexcabal.com/creating-the-perfect-gpg-keypair/ -[source code]: /doc/source-code/ -[developer mailing list]: /support/#qubes-devel -[Enigmail]: https://www.enigmail.net/ -[signature-checker]: https://github.com/marmarek/signature-checker -[GitHub New GPG key]: https://github.com/settings/gpg/new -[GitHub SSH GPG keys page]: https://github.com/settings/keys - diff --git a/developer/code/coding-style.md b/developer/code/coding-style.md index f37e2b9071..4468df0bb9 100644 --- a/developer/code/coding-style.md +++ b/developer/code/coding-style.md @@ -1,26 +1,25 @@ --- +lang: en layout: doc -title: Coding Style permalink: /doc/coding-style/ redirect_from: - /en/doc/coding-style/ - /doc/CodingStyle/ - /wiki/CodingStyle/ - /trac/wiki/CodingStyle/ +ref: 53 +title: Coding style --- -Coding Guidelines for Qubes Developers -====================================== - Rationale --------- Maintaining proper coding style is very important for any large software project, such as Qubes. Here's why: -- It eases maintenance tasks, such as adding new functionality or generalizing code later, -- It allows others (as well as the future you!) to easily understand fragments of code and what they were supposed to do, and thus makes it easier to later extend them with newer functionality or bug fixes, -- It allows others to easily review the code and catch various bugs, -- It provides for an aesthetically pleasing experience when one reads the code... +- It eases maintenance tasks, such as adding new functionality or generalizing code later, +- It allows others (as well as the future you!) to easily understand fragments of code and what they were supposed to do, and thus makes it easier to later extend them with newer functionality or bug fixes, +- It allows others to easily review the code and catch various bugs, +- It provides for an aesthetically pleasing experience when one reads the code... Often, developers, usually smart ones, undersell the value of proper coding style, thinking that it's much more important how their code works. These developers expect that if their code solves some problem using a nice and neat trick, then that's all that is really required. Such thinking shows, however, immaturity and is a signal that the developer, no matter how bright and smart, might not be a good fit for larger projects. Writing a clever exploit for a Black Hat show is one thing - writing useful software supposed to be used and maintained for years is quite a different story. If you want to show off what a smart programmer you are, then you should become a researcher and write exploits. If, on the other hand, you want to be part of a team that makes real, useful software, you should ensure your coding style is impeccable. At Qubes project, we often took shortcuts and wrote nasty code, and this has always back fired at us, sometime months, sometime years later, the net result being we had to spend time fixing code, rather than implementing new functionality. @@ -29,25 +28,25 @@ And here's a [link to the real case](https://groups.google.com/forum/#!msg/qubes General typographic conventions ------------------------------- -- **Use space-expanded tabs that equal 4 spaces.** Yes, we know, there are many arguments for using "real" tabs instead of space-expanded tabs, but we need to pick one convention to make the project consistent. One argument for using space-expanded tabs is that this way the programmer is in control of how the code will look like, despite how other users have configured their editors to visualize the tabs (of course, we assume any sane person uses a fixed-width font for viewing the source code). If it makes you feel any better, assume this is just an arbitrary choice made to enforce a unified style. +- **Use space-expanded tabs that equal 4 spaces.** Yes, we know, there are many arguments for using "real" tabs instead of space-expanded tabs, but we need to pick one convention to make the project consistent. One argument for using space-expanded tabs is that this way the programmer is in control of how the code will look like, despite how other users have configured their editors to visualize the tabs (of course, we assume any sane person uses a fixed-width font for viewing the source code). If it makes you feel any better, assume this is just an arbitrary choice made to enforce a unified style. -- **Maintain max. line length of 80 characters**. Even though today's monitors often are very wide and it's often not a problem to have 120 characters displayed in an editor, maintaining shorter line lengths improves readability. It also allows others to have two parallel windows open, side by side, each with different parts of the source code. +- **Maintain max. line length of 80 characters**. Even though today's monitors often are very wide and it's often not a problem to have 120 characters displayed in an editor, maintaining shorter line lengths improves readability. It also allows others to have two parallel windows open, side by side, each with different parts of the source code. -- **Naming conventions for any OS *other than Windows***: - - `ClassName` - - `some_variable`, `some_function`, `some_argument` +- **Naming conventions for any OS *other than Windows***: + - `ClassName` + - `some_variable`, `some_function`, `some_argument` -- **Naming convention *for Windows OS*** -- exceptionally to preserve Windows conventions please use the following: - - `ClassName`, `FunctionName` - - `pszArgumentOne`, `hPipe` -- use Hungarian notation for argument and variables +- **Naming convention *for Windows OS*** -- exceptionally to preserve Windows conventions please use the following: + - `ClassName`, `FunctionName` + - `pszArgumentOne`, `hPipe` -- use Hungarian notation for argument and variables -- **Maintain a decent amount of horizontal spacing**, e.g. add a space after `if` or before `{` in C, and similar in other languages. Whether and where to also use spaces within expressions, such as (x\*2+5) vs. (x \* 2 + 5) is left to the developer's judgment. Do not put spaces immediately after or before the brackets in expressions, so avoid constructs like this: `if ( condition )` and use ones like this: `if (condition)` instead. +- **Maintain a decent amount of horizontal spacing**, e.g. add a space after `if` or before `{` in C, and similar in other languages. Whether and where to also use spaces within expressions, such as (x\*2+5) vs. (x \* 2 + 5) is left to the developer's judgment. Do not put spaces immediately after or before the brackets in expressions, so avoid constructs like this: `if ( condition )` and use ones like this: `if (condition)` instead. -- **Use single new lines** ('\\n' aka LF) in any non-Windows source code. On Windows, exceptionally, use the CRLF line endings (--). This will allow the source code to be easily viewable in various Windows-based programs. +- **Use single new lines** ('\\n' aka LF) in any non-Windows source code. On Windows, exceptionally, use the CRLF line endings (--). This will allow the source code to be easily viewable in various Windows-based programs. -- **Use descriptive names for variables and functions**! Really, at a time when most editors have auto-completion features, there is no excuse for using short variable names. +- **Use descriptive names for variables and functions**! Really, at a time when most editors have auto-completion features, there is no excuse for using short variable names. -- **Comments should be indented together with the code**, e.g. like this: +- **Comments should be indented together with the code**, e.g. like this: ~~~ for (...) { @@ -61,33 +60,33 @@ General typographic conventions File naming conventions ----------------------- -- All file names written with small letters, use dash to separate words, rather than underscores, e.g. `qubes-dom0-update`. Never use spaces! +- All file names written with small letters, use dash to separate words, rather than underscores, e.g. `qubes-dom0-update`. Never use spaces! **File naming in Linux/Unix-like systems:** -- User commands that operate on particular VMs (also those accessible in VMs): `/usr/bin/qvm-*` -- User commands that apply to the whole system (Dom0 only): `/usr/bin/qubes-*` -- Auxiliary executables and scripts in `/usr/libexec/qubes/` (Note: previously we used `/usr/lib/qubes` but decided to change that) -- Helper, non-executable files in `/usr/share/qubes/` -- Various config files in `/etc/qubes` -- Qubes RPC services in `/etc/qubes-rpc`. Qubes RPC Policy definitions (only in Dom0) in `/etc/qubes-rpc/policy/` -- All VM-related configs, images, and other files in `/var/lib/qubes/` -- System-wide temporary files which reflect the current state of system in `/var/run/qubes` -- Logs: either log to the system-wide messages, or to `/var/log/qubes/` +- User commands that operate on particular VMs (also those accessible in VMs): `/usr/bin/qvm-*` +- User commands that apply to the whole system (Dom0 only): `/usr/bin/qubes-*` +- Auxiliary executables and scripts in `/usr/libexec/qubes/` (Note: previously we used `/usr/lib/qubes` but decided to change that) +- Helper, non-executable files in `/usr/share/qubes/` +- Various config files in `/etc/qubes` +- Qubes RPC services in `/etc/qubes-rpc`. Qubes RPC Policy definitions (only in Dom0) in `/etc/qubes-rpc/policy/` +- All VM-related configs, images, and other files in `/var/lib/qubes/` +- System-wide temporary files which reflect the current state of system in `/var/run/qubes` +- Logs: either log to the system-wide messages, or to `/var/log/qubes/` **File naming in Windows systems:** -- All base qubes-related files in `C:\Program Files\Invisible Things Lab\Qubes\` (Exceptionally spaces are allowed here to adhere to Windows naming conventions) -- Other, third-party files, not Qubes-specific, such as e.g. Xen PV drivers might be in different vendor subdirs, e.g. `C:\Program Files\Xen PV Drivers` +- All base qubes-related files in `C:\Program Files\Invisible Things Lab\Qubes\` (Exceptionally spaces are allowed here to adhere to Windows naming conventions) +- Other, third-party files, not Qubes-specific, such as e.g. Xen PV drivers might be in different vendor subdirs, e.g. `C:\Program Files\Xen PV Drivers` General programming style guidelines ------------------------------------ -- Do not try to impress with your coding kung-fu, do not use tricks to save 2 lines of code, always prefer readability over trickiness! -- Make sure your code compiles and builds without warnings. -- Always think first about interfaces (e.g. function arguments, or class methods) and data structures before you start writing the actual code. -- Use comments to explain non-trivial code fragments, or expected behavior of more complex functions, if it is not clear from their name. -- Do **not** use comments for code fragments where it is immediately clear what the code does. E.g. avoid constructs like this: +- Do not try to impress with your coding kung-fu, do not use tricks to save 2 lines of code, always prefer readability over trickiness! +- Make sure your code compiles and builds without warnings. +- Always think first about interfaces (e.g. function arguments, or class methods) and data structures before you start writing the actual code. +- Use comments to explain non-trivial code fragments, or expected behavior of more complex functions, if it is not clear from their name. +- Do **not** use comments for code fragments where it is immediately clear what the code does. E.g. avoid constructs like this: ~~~ // Return window id @@ -97,7 +96,7 @@ General programming style guidelines } ~~~ -- Do **not** use comments to disable code fragments. In production code there should really be no commented or disabled code fragments. If you really, really have a good reason to retain some fragment of unused code, use \#if or \#ifdef to disable it, e.g.: +- Do **not** use comments to disable code fragments. In production code there should really be no commented or disabled code fragments. If you really, really have a good reason to retain some fragment of unused code, use \#if or \#ifdef to disable it, e.g.: ~~~ #if 0 @@ -117,42 +116,42 @@ General programming style guidelines > But generally, there is little excuse to keep old, unused code fragments in the code. One should really use the functionality provided by the source code management system, such as git, instead. E.g. create a special branch for storing the old, unused code -- this way you will always be able to merge this code into upstream in the future. -- Do not hardcode values in the code! The only three numbers that are an exception here and for which it is acceptable to hardcode them are: `0`, `1` and `-1`, and frankly the last two are still controversial... +- Do not hardcode values in the code! The only three numbers that are an exception here and for which it is acceptable to hardcode them are: `0`, `1` and `-1`, and frankly the last two are still controversial... Source Code management (Git) guidelines --------------------------------------- -- Use git to maintain all code for Qubes project. +- Use git to maintain all code for Qubes project. -- Before you start using git, make sure you understand that git is a decentralized Source Code Management system, and that it doesn't behave like traditional, centralized source code management systems, such as SVN. Here's a good [introductory book on git](http://git-scm.com/book). Read it. +- Before you start using git, make sure you understand that git is a decentralized Source Code Management system, and that it doesn't behave like traditional, centralized source code management systems, such as SVN. Here's a good [introductory book on git](http://git-scm.com/book). Read it. -- Qubes code is divided into many git repositories. There are several reasons for that: - - This creates natural boundaries between different code blocks, enforcing proper interfaces, and easing independent development to be conducted on various code parts at the same time, without the fear of running into conflicts. - - By maintaining relatively small git repositories, it is easy for new developers to understand the code and contribute new patches, without the need to understand all the other code. - - Code repositories represent also licensing boundaries. So, e.g. because `core-agent-linux` and `core-agent-windows` are maintained in two different repositories, it is possible to have the latter under a proprietary, non-GPL license, while keeping the former fully open source. - - We have drastically changed the layout and naming of the code repositories shortly after Qubes OS R2 Beta 2 release. For details on the current code layout, please read [this article](https://blog.invisiblethings.org/2013/03/21/introducing-qubes-odyssey-framework.html). +- Qubes code is divided into many git repositories. There are several reasons for that: + - This creates natural boundaries between different code blocks, enforcing proper interfaces, and easing independent development to be conducted on various code parts at the same time, without the fear of running into conflicts. + - By maintaining relatively small git repositories, it is easy for new developers to understand the code and contribute new patches, without the need to understand all the other code. + - Code repositories represent also licensing boundaries. So, e.g. because `core-agent-linux` and `core-agent-windows` are maintained in two different repositories, it is possible to have the latter under a proprietary, non-GPL license, while keeping the former fully open source. + - We have drastically changed the layout and naming of the code repositories shortly after Qubes OS R2 Beta 2 release. For details on the current code layout, please read [this article](https://blog.invisiblethings.org/2013/03/21/introducing-qubes-odyssey-framework.html). Commit message guidelines ------------------------- Please attempt to follow these conventions when writing your Git commit messages: - * Separate the subject line from the body with a blank line. - * Limit the subject line to approximately 50 characters. - * Capitalize the subject line. - * Do not end the subject line with a period. - * Use the imperative mood in the subject line. - * Wrap the body at 72 characters. - * Use the body to explain *what* and *why* rather than *how*. +- Separate the subject line from the body with a blank line. +- Limit the subject line to approximately 50 characters. +- Capitalize the subject line. +- Do not end the subject line with a period. +- Use the imperative mood in the subject line. +- Wrap the body at 72 characters. +- Use the body to explain *what* and *why* rather than *how*. For details, examples, and the rationale behind each of these conventions, please see [this blog post](https://chris.beams.io/posts/git-commit/), which is the source of this list. Security coding guidelines -------------------------- -- As a general rule: **untrusted input** is our \#1 enemy! -- Any input that comes from untrusted, or less trusted, or just differently-trusted, entity should always be considered as malicious and should always be sanitized and verified. So, if your software runs in Dom0 and processes some input from any of the VMs, this input should be considered to be malicious. Even if your software runs in a VM, and processes input from some other VM, you should also assume that the input is malicious and verify it. -- Use `untrusted_` prefix for all variables that hold values read from untrusted party and which have not yet been verified to be decent, e.g.: +- As a general rule: **untrusted input** is our \#1 enemy! +- Any input that comes from untrusted, or less trusted, or just differently-trusted, entity should always be considered as malicious and should always be sanitized and verified. So, if your software runs in Dom0 and processes some input from any of the VMs, this input should be considered to be malicious. Even if your software runs in a VM, and processes input from some other VM, you should also assume that the input is malicious and verify it. +- Use `untrusted_` prefix for all variables that hold values read from untrusted party and which have not yet been verified to be decent, e.g.: ~~~ read_struct(untrusted_conf); @@ -165,22 +164,22 @@ Security coding guidelines height = untrusted_conf.height; ~~~ -- Use others variables, without the `untrusted_` prefix to hold the sanitized values, as shown above. +- Use others variables, without the `untrusted_` prefix to hold the sanitized values, as shown above. Python-specific guidelines -------------------------- -- Please follow the guidelines [here](http://www.python.org/dev/peps/pep-0008/), unless they were in conflict with what is written on this page. +- Please follow the guidelines [here](http://www.python.org/dev/peps/pep-0008/), unless they were in conflict with what is written on this page. C and C++ specific guidelines ----------------------------- -- Do not place code in `*.h` files. -- Use `const` whenever possible, e.g. in function arguments passed via pointers. -- Do not mix procedural and objective code together -- if you write in C++, use classes and objects. -- Think about classes hierarchy, before starting to implement specific methods. +- Do not place code in `*.h` files. +- Use `const` whenever possible, e.g. in function arguments passed via pointers. +- Do not mix procedural and objective code together -- if you write in C++, use classes and objects. +- Think about classes hierarchy, before starting to implement specific methods. Bash-specific guidelines ------------------------ -- Avoid writing scripts in bash whenever possible. Use python instead. Bash-scripts are Unix-specific and will not work under Windows VMs, or in Windows admin domain, or Windows gui domain. +- Avoid writing scripts in bash whenever possible. Use python instead. Bash-scripts are Unix-specific and will not work under Windows VMs, or in Windows admin domain, or Windows gui domain. diff --git a/developer/code/license.md b/developer/code/license.md index 1a7879a2f9..f7fe3682e8 100644 --- a/developer/code/license.md +++ b/developer/code/license.md @@ -1,21 +1,23 @@ --- +lang: en layout: doc -title: License permalink: /doc/license/ redirect_from: - /en/doc/license/ - /doc/QubesLicensing/ - /wiki/QubesLicensing/ +ref: 52 +title: Software license --- -Qubes OS License -================ +Qubes OS is a compilation of software packages, each under its own license. The +compilation is made available under the GNU General Public License version 2 +(GPLv2). -Qubes is a compilation of software packages, each under its own license. The compilation is made available under the GNU General Public License version 2. +The source code of Qubes OS is contained in repositories under the +[@QubesOS](https://github.com/QubesOS) account on GitHub. This source code is +made available under GPLv2, unless there is a `LICENSE` file in the root of the +containing repository that specifies a different license. -The full text of the GPL v2 license can be found [here](http://www.gnu.org/licenses/gpl-2.0.html). - -Note on rights to double-licensing of the Qubes code ----------------------------------------------------- - -Invisible Things Lab (ITL), who has funded and run the Qubes project since the beginning, and who has contributed the majority of Qubes-specific code (specifically: `core-*`, `gui-*`, and `qubes-*` repositories) would like to have a right to redistribute parts of this code under proprietary licenses. This is especially important for Qubes R3 and later, where the new architecture allows the creation of many editions of Qubes, using different hypervisors, some of which might not be open source. That's why we ask every developer who contributes code to Qubes project to grant ITL permission to reuse the code under a different license, and to express this consent by including the [standard signed-off line](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/process/submitting-patches.rst?id=HEAD#n416) in the commit. +The full text of the GPLv2 license can be found +[here](https://www.gnu.org/licenses/gpl-2.0.html). diff --git a/developer/code/source-code.md b/developer/code/source-code.md index 6f1f531b13..85e5cf13a9 100644 --- a/developer/code/source-code.md +++ b/developer/code/source-code.md @@ -1,27 +1,26 @@ --- +lang: en layout: doc -title: Source Code permalink: /doc/source-code/ redirect_from: - /en/doc/source-code/ - /doc/SourceCode/ - /wiki/SourceCode/ +ref: 54 +title: Source code --- -Qubes Source Code Repositories -============================== - All the Qubes code is kept in Git repositories. We have divided the project into several components, each of which has its own separate repository, for example: - * `core-admin.git` -- The core Qubes infrastructure, responsible for VM +* `core-admin.git` -- The core Qubes infrastructure, responsible for VM management, VM templates, fs sharing, etc. - * `gui-daemon.git` -- GUI virtualization, Dom0 side. - * `gui-agent-linux.git` -- GUI virtualization, Linux VM side. - * `linux-template-builder.git` -- Scripts and other files used to create Qubes +* `gui-daemon.git` -- GUI virtualization, Dom0 side. +* `gui-agent-linux.git` -- GUI virtualization, Linux VM side. +* `linux-template-builder.git` -- Scripts and other files used to create Qubes template images. -All of our repositories are available under the [QubesOS GitHub account]. +All of our repositories are available under the [QubesOS GitHub account](https://github.com/QubesOS/). To clone a repository: @@ -35,9 +34,9 @@ e.g.: git clone https://github.com/QubesOS/qubes-core-admin.git core-admin ~~~ -To build Qubes you do not need to download all these repositories. -If you use [qubes builder] you can specify *what* you want to build, and download only the repositories needed to build that target. - +To build Qubes you do not need to download all these repositories. +If you use [qubes builder](/doc/QubesBuilder/) you can specify *what* you want to build, and download only the repositories needed to build that target. + If you really do want to clone **all** of the repositories, you can use these commands: ~~~ @@ -53,21 +52,20 @@ find . -mindepth 1 -maxdepth 1 -type d -exec git -C {} fetch --tags --recurse-su (Alternatively, you can pull instead of just fetching.) - How to Send Patches ------------------- -If you want to [contribute code] to the project, there are two ways. Whichever -method you choose, you must [sign your code] before it can be accepted. +If you want to [contribute code](/doc/contributing/#contributing-code) to the project, there are two ways. Whichever +method you choose, you must [sign your code](/doc/code-signing/) before it can be accepted. -* **Preferred**: Use GitHub's [fork & pull requests]. +* **Preferred**: Use GitHub's [fork & pull requests](https://guides.github.com/activities/forking/). Opening a pull request on GitHub greatly eases the code review and tracking process. In addition, especially for bigger changes, it's a good idea to send - a message to the [qubes-devel mailing list] in order to notify people who + a message to the [qubes-devel mailing list](/support/#qubes-devel) in order to notify people who do not receive GitHub notifications. -* Send a patch to the [qubes-devel mailing list] (`git format-patch`). +* Send a patch to the [qubes-devel mailing list](/support/#qubes-devel) (`git format-patch`). 1. Make all the changes in your working directory, i.e. edit files, move them around (you can use 'git mv' for this), etc. @@ -82,11 +80,3 @@ method you choose, you must [sign your code] before it can be accepted. name and email, so that *your* name will be used as a commit's author. 5. Send your patch to `qubes-devel`. Start the message subject with `[PATCH]`. - - -[QubesOS GitHub account]: https://github.com/QubesOS/ -[contribute code]: /doc/contributing/#contributing-code -[sign your code]: /doc/code-signing/ -[fork & pull requests]: https://guides.github.com/activities/forking/ -[qubes-devel mailing list]: /support/#qubes-devel -[qubes builder]: /doc/QubesBuilder/ diff --git a/developer/debugging/automated-tests.md b/developer/debugging/automated-tests.md index c222a34c29..3fd4ed24b3 100644 --- a/developer/debugging/automated-tests.md +++ b/developer/debugging/automated-tests.md @@ -1,20 +1,18 @@ --- +lang: en layout: doc -title: Automated Tests permalink: /doc/automated-tests/ redirect_from: - /en/doc/automated-tests/ - /doc/AutomatedTests/ +ref: 45 +title: Automated tests --- -Automated Tests -=============== +## Unit and Integration Tests -Unit and Integration Tests --------------------------- - -Starting with Qubes R3 we use [python unittest][unittest] to perform automatic tests of Qubes OS. -Despite the name, we use it for both [unit tests](https://en.wikipedia.org/wiki/Unit_tests) and [integration tests](https://en.wikipedia.org/wiki/Integration_tests). +Starting with Qubes R3 we use [python unittest](https://docs.python.org/3/library/unittest.html) to perform automatic tests of Qubes OS. +Despite the name, we use it for both [unit tests](https://en.wikipedia.org/wiki/Unit_tests) and [integration tests](https://en.wikipedia.org/wiki/Integration_tests). The main purpose is, of course, to deliver much more stable releases. The integration tests must be run in dom0, but some unit tests can run inside a VM as well. @@ -46,79 +44,84 @@ Our test runner runs mostly the same as the standard one, but it has some nice a You can use `python3 -m qubes.tests.run -h` to get usage information: - [user@dom0 ~]$ python3 -m qubes.tests.run -h - usage: run.py [-h] [--verbose] [--quiet] [--list] [--failfast] [--no-failfast] - [--do-not-clean] [--do-clean] [--loglevel LEVEL] - [--logfile FILE] [--syslog] [--no-syslog] [--kmsg] [--no-kmsg] - [TESTNAME [TESTNAME ...]] - - positional arguments: - TESTNAME list of tests to run named like in description - (default: run all tests) - - optional arguments: - -h, --help show this help message and exit - --verbose, -v increase console verbosity level - --quiet, -q decrease console verbosity level - --list, -l list all available tests and exit - --failfast, -f stop on the first fail, error or unexpected success - --no-failfast disable --failfast - --loglevel LEVEL, -L LEVEL - logging level for file and syslog forwarding (one of: - NOTSET, DEBUG, INFO, WARN, WARNING, ERROR, CRITICAL; - default: DEBUG) - --logfile FILE, -o FILE - if set, test run will be also logged to file - --syslog reenable logging to syslog - --no-syslog disable logging to syslog - --kmsg, --very-brave-or-very-stupid - log most important things to kernel ring-buffer - --no-kmsg, --i-am-smarter-than-kay-sievers - do not abuse kernel ring-buffer - --allow-running-along-qubesd - allow running in parallel with qubesd; this is - DANGEROUS and WILL RESULT IN INCONSISTENT SYSTEM STATE - --break-to-repl break to REPL after tests - - When running only specific tests, write their names like in log, in format: - MODULE+"/"+CLASS+"/"+FUNCTION. MODULE should omit initial "qubes.tests.". - Example: basic/TC_00_Basic/test_000_create +``` +[user@dom0 ~]$ python3 -m qubes.tests.run -h +usage: run.py [-h] [--verbose] [--quiet] [--list] [--failfast] [--no-failfast] + [--do-not-clean] [--do-clean] [--loglevel LEVEL] + [--logfile FILE] [--syslog] [--no-syslog] [--kmsg] [--no-kmsg] + [TESTNAME [TESTNAME ...]] + +positional arguments: + TESTNAME list of tests to run named like in description + (default: run all tests) + +optional arguments: + -h, --help show this help message and exit + --verbose, -v increase console verbosity level + --quiet, -q decrease console verbosity level + --list, -l list all available tests and exit + --failfast, -f stop on the first fail, error or unexpected success + --no-failfast disable --failfast + --loglevel LEVEL, -L LEVEL + logging level for file and syslog forwarding (one of: + NOTSET, DEBUG, INFO, WARN, WARNING, ERROR, CRITICAL; + default: DEBUG) + --logfile FILE, -o FILE + if set, test run will be also logged to file + --syslog reenable logging to syslog + --no-syslog disable logging to syslog + --kmsg, --very-brave-or-very-stupid + log most important things to kernel ring-buffer + --no-kmsg, --i-am-smarter-than-kay-sievers + do not abuse kernel ring-buffer + --allow-running-along-qubesd + allow running in parallel with qubesd; this is + DANGEROUS and WILL RESULT IN INCONSISTENT SYSTEM STATE + --break-to-repl break to REPL after tests + +When running only specific tests, write their names like in log, in format: +MODULE+"/"+CLASS+"/"+FUNCTION. MODULE should omit initial "qubes.tests.". +Example: basic/TC_00_Basic/test_000_create +``` For instance, to run only the tests for the fedora-21 template, you can use the `-l` option, then filter the list: - [user@dom0 ~]$ python3 -m qubes.tests.run -l | grep fedora-21 - network/VmNetworking_fedora-21/test_000_simple_networking - network/VmNetworking_fedora-21/test_010_simple_proxyvm - network/VmNetworking_fedora-21/test_020_simple_proxyvm_nm - network/VmNetworking_fedora-21/test_030_firewallvm_firewall - network/VmNetworking_fedora-21/test_040_inter_vm - vm_qrexec_gui/TC_00_AppVM_fedora-21/test_000_start_shutdown - vm_qrexec_gui/TC_00_AppVM_fedora-21/test_010_run_gui_app - vm_qrexec_gui/TC_00_AppVM_fedora-21/test_050_qrexec_simple_eof - vm_qrexec_gui/TC_00_AppVM_fedora-21/test_051_qrexec_simple_eof_reverse - vm_qrexec_gui/TC_00_AppVM_fedora-21/test_052_qrexec_vm_service_eof - vm_qrexec_gui/TC_00_AppVM_fedora-21/test_053_qrexec_vm_service_eof_reverse - vm_qrexec_gui/TC_00_AppVM_fedora-21/test_060_qrexec_exit_code_dom0 - vm_qrexec_gui/TC_00_AppVM_fedora-21/test_065_qrexec_exit_code_vm - vm_qrexec_gui/TC_00_AppVM_fedora-21/test_100_qrexec_filecopy - vm_qrexec_gui/TC_00_AppVM_fedora-21/test_110_qrexec_filecopy_deny - vm_qrexec_gui/TC_00_AppVM_fedora-21/test_120_qrexec_filecopy_self - vm_qrexec_gui/TC_20_DispVM_fedora-21/test_000_prepare_dvm - vm_qrexec_gui/TC_20_DispVM_fedora-21/test_010_simple_dvm_run - vm_qrexec_gui/TC_20_DispVM_fedora-21/test_020_gui_app - vm_qrexec_gui/TC_20_DispVM_fedora-21/test_030_edit_file - [user@dom0 ~]$ sudo -E python3 -m qubes.tests.run -v `python3 -m qubes.tests.run -l | grep fedora-21` +``` +[user@dom0 ~]$ python3 -m qubes.tests.run -l | grep fedora-21 +network/VmNetworking_fedora-21/test_000_simple_networking +network/VmNetworking_fedora-21/test_010_simple_proxyvm +network/VmNetworking_fedora-21/test_020_simple_proxyvm_nm +network/VmNetworking_fedora-21/test_030_firewallvm_firewall +network/VmNetworking_fedora-21/test_040_inter_vm +vm_qrexec_gui/TC_00_AppVM_fedora-21/test_000_start_shutdown +vm_qrexec_gui/TC_00_AppVM_fedora-21/test_010_run_gui_app +vm_qrexec_gui/TC_00_AppVM_fedora-21/test_050_qrexec_simple_eof +vm_qrexec_gui/TC_00_AppVM_fedora-21/test_051_qrexec_simple_eof_reverse +vm_qrexec_gui/TC_00_AppVM_fedora-21/test_052_qrexec_vm_service_eof +vm_qrexec_gui/TC_00_AppVM_fedora-21/test_053_qrexec_vm_service_eof_reverse +vm_qrexec_gui/TC_00_AppVM_fedora-21/test_060_qrexec_exit_code_dom0 +vm_qrexec_gui/TC_00_AppVM_fedora-21/test_065_qrexec_exit_code_vm +vm_qrexec_gui/TC_00_AppVM_fedora-21/test_100_qrexec_filecopy +vm_qrexec_gui/TC_00_AppVM_fedora-21/test_110_qrexec_filecopy_deny +vm_qrexec_gui/TC_00_AppVM_fedora-21/test_120_qrexec_filecopy_self +vm_qrexec_gui/TC_20_DispVM_fedora-21/test_000_prepare_dvm +vm_qrexec_gui/TC_20_DispVM_fedora-21/test_010_simple_dvm_run +vm_qrexec_gui/TC_20_DispVM_fedora-21/test_020_gui_app +vm_qrexec_gui/TC_20_DispVM_fedora-21/test_030_edit_file +[user@dom0 ~]$ sudo -E python3 -m qubes.tests.run -v `python3 -m qubes.tests.run -l | grep fedora-21` +``` Example test run: -![snapshot-tests2.png](/attachment/wiki/developers/snapshot-tests2.png) +![snapshot-tests2.png](/attachment/doc/snapshot-tests2.png) Tests are also compatible with nose2 test runner, so you can use this instead: - sudo systemctl stop qubesd; sudo -E nose2 -v --plugin nose2.plugins.loader.loadtests qubes.tests; sudo systemctl start qubesd - -This may be especially useful together with various nose2 plugins to store tests results (for example `nose2.plugins.junitxml`), to ease presenting results. This is what we use on [OpenQA]. +```bash +sudo systemctl stop qubesd; sudo -E nose2 -v --plugin nose2.plugins.loader.loadtests qubes.tests; sudo systemctl start qubesd +``` +This may be especially useful together with various nose2 plugins to store tests results (for example `nose2.plugins.junitxml`), to ease presenting results. This is what we use on [OpenQA](http://open.qa/). ### Unit testing inside a VM @@ -132,6 +135,7 @@ its dependency [qubes-core-qrexec](https://github.com/QubesOS/qubes-core-qrexec) The below example however will assume that you set up a build environment as described in the [Qubes Builder documentation](/doc/qubes-builder/). Assuming you cloned the `qubes-builder` repository to your home directory inside a fedora VM, you can use the following commands to run the unit tests: + ```{.bash} cd ~ sudo dnf install python3-pip lvm2 python35 python3-virtualenv @@ -158,30 +162,31 @@ the current stable branch. Test runs can be altered using environment variables: - - `DEFAULT_LVM_POOL` - LVM thin pool to use for tests, in `VolumeGroup/ThinPool` format - - `QUBES_TEST_PCIDEV` - PCI device to be used in PCI passthrough tests (for example sound card) - - `QUBES_TEST_TEMPLATES` - space separated list of templates to run tests on; if not set, all installed templates are tested - - `QUBES_TEST_LOAD_ALL` - load all tests (including tests for all templates) when relevant test modules are imported; this needs to be set for test runners not supporting [load_tests protocol](https://docs.python.org/3/library/unittest.html#load-tests-protocol) +- `DEFAULT_LVM_POOL` - LVM thin pool to use for tests, in `VolumeGroup/ThinPool` format +- `QUBES_TEST_PCIDEV` - PCI device to be used in PCI passthrough tests (for example sound card) +- `QUBES_TEST_TEMPLATES` - space separated list of templates to run tests on; if not set, all installed templates are tested +- `QUBES_TEST_LOAD_ALL` - load all tests (including tests for all templates) when relevant test modules are imported; this needs to be set for test runners not supporting [load_tests protocol](https://docs.python.org/3/library/unittest.html#load-tests-protocol) ### Adding a new test to core-admin -After adding a new unit test to [core-admin/qubes/tests](https://github.com/QubesOS/qubes-core-admin/tree/master/qubes/tests) you'll have to include it in [core-admin/qubes/tests/\_\_init\_\_.py](https://github.com/QubesOS/qubes-core-admin/tree/master/qubes/tests/__init__.py) +After adding a new unit test to [core-admin/qubes/tests](https://github.com/QubesOS/qubes-core-admin/tree/master/qubes/tests) you'll have to include it in [core-admin/qubes/tests/\_\_init\_\_.py](https://github.com/QubesOS/qubes-core-admin/tree/master/qubes/tests/__init__.py) #### Editing `__init__.py` + You'll also need to add your test at the bottom of the `__init__.py` file, in the method `def load_tests`, in the for loop with `modname`. Again, given the hypothetical `example.py` test: ~~~python - for modname in ( - 'qubes.tests.basic', - 'qubes.tests.dom0_update', - 'qubes.tests.network', - 'qubes.tests.vm_qrexec_gui', - 'qubes.tests.backup', - 'qubes.tests.backupcompatibility', - 'qubes.tests.regressions', - 'qubes.tests.example', # This is our newly added test - ): +for modname in ( + 'qubes.tests.basic', + 'qubes.tests.dom0_update', + 'qubes.tests.network', + 'qubes.tests.vm_qrexec_gui', + 'qubes.tests.backup', + 'qubes.tests.backupcompatibility', + 'qubes.tests.regressions', + 'qubes.tests.example', # This is our newly added test + ): ~~~ ### Testing PyQt applications @@ -213,7 +218,7 @@ class SomeTestCase(unittest.TestCase): # first test that actually use event loop will try to dereference (already # destroyed) objects, resulting in SEGV self.loop = quamash.QEventLoop(self.qtapp) - + def tearDown(self): [...] # process any pending events before destroying the object @@ -237,15 +242,13 @@ class SomeTestCase(unittest.TestCase): gc.collect() ~~~ +## Automated tests with openQA -Installation Tests with openQA ------------------------------- - -**URL:** +**URL:** **Tests:** -Manually testing the installation of Qubes OS is a time-consuming process. -We use [openQA] to automate this process. +Manually testing Qubes OS and its installation is a time-consuming process. +We use [OpenQA](http://open.qa/) to automate this process. It works by installing Qubes in KVM and interacting with it as a user would, including simulating mouse clicks and keyboard presses. Then, it checks the output to see whether various tests were passed, e.g. by comparing the virtual screen output to screenshots of a successful installation. @@ -254,8 +257,23 @@ KVM also supports nested virtualization, so HVM should theoretically work. In practice, however, either Xen or QEMU crashes when this is attempted. Nonetheless, PV works well, which is sufficient for automated installation testing. -Thanks to an anonymous donor, our openQA system is hosted in a datacenter on hardware that meets these requirements. +Thanks to present and past donors who have provided the infrastructure for Qubes' openQA system with hardware that meets these requirements. + +### Looking for patterns in tests + +In order to better visualize patterns in tests the [`openqa_investigator`](https://github.com/QubesOS/openqa-tests-qubesos/blob/master/utils/openqa_investigator.py) script can be used. +It feeds off of the openQA test data to make graph plots. Here is an example: + +![openqa-investigator-splitgpg-example.png](/attachment/doc/openqa-investigator-splitgpg-example.png) + +Some outputs: + - plot by tests + - plot by errors + - markdown -[unittest]: https://docs.python.org/3/library/unittest.html -[OpenQA]: http://open.qa/ +Some filters: + - filter by error + - filter by test name +Check out the script's help with `python3 openqa_investigator.py --help` +to see all available options. diff --git a/developer/debugging/mount-lvm-image.md b/developer/debugging/mount-lvm-image.md index ca9f090131..d5b7608078 100644 --- a/developer/debugging/mount-lvm-image.md +++ b/developer/debugging/mount-lvm-image.md @@ -1,13 +1,13 @@ --- +lang: en layout: doc -title: Mount LVM image permalink: /doc/mount-lvm-image/ +ref: 46 +title: How to mount LVM images --- -# How to mount LVM image +You want to read your LVM image (e.g., there is a problem where you can't start any VMs except dom0). -You want to read your LVM image (e.g., there is a problem where you can't start any VMs except dom0). - 1: make the image available for qubesdb. From dom0 terminal: @@ -25,7 +25,8 @@ From dom0 terminal: 3: Attach the device to your newly created disp VM -From the GUI, or from the command line: +From the GUI, or from the command line: + ```bash [user@dom0]$ qvm-block attach NEWLY_CREATED_DISPVM dom0:$dev ``` @@ -37,15 +38,17 @@ From the GUI, or from the command line: ``` 5: Umount and kill the VM -``` + +```bash [user@dispXXXX]$ umount /mnt/ ``` 6: Remove the image from qubesdb -``` + +```bash [user@dom0]$ qubesdb-rm /qubes-block-devices/$dev/ ``` # References -https://github.com/QubesOS/qubes-issues/issues/4687#issuecomment-451626625 +Please consult this issue's [comment](https://github.com/QubesOS/qubes-issues/issues/4687#issuecomment-451626625). diff --git a/developer/debugging/profiling.md b/developer/debugging/profiling.md index e20e3208f4..2932b4cbf1 100644 --- a/developer/debugging/profiling.md +++ b/developer/debugging/profiling.md @@ -1,22 +1,20 @@ --- +lang: en layout: doc -title: Profiling permalink: /doc/profiling/ redirect_from: - /en/doc/profiling/ - /doc/Profiling/ - /wiki/Profiling/ +ref: 48 +title: Python profiling --- -Profiling -========= - This is a python profiling primer. For the purpose of this document, `qubes-dev` is name of the domain used for postprocessing profiling stats. -Requirements ------------- +## Requirements ~~~ yum install gprof2dot graphviz @@ -30,10 +28,9 @@ mkdir -p ~/profiling qvm-run -p qubes-dev 'cat ~/profiling/Upload.sh' > ~/profiling/Upload.sh ~~~ -- WARNING: this will obviously be running third-party code which is not signed by ITL nor Fedora. You have been warned. +- WARNING: this will obviously be running third-party code which is not signed by ITL nor Fedora. You have been warned. -Workflow --------- +## Workflow ### Identify function responsible for some slow action @@ -43,17 +40,21 @@ You have to select the area in which you suspect less than optimal performance. Replace - def foo(self, bar): - # function content +```python +def foo(self, bar): + # function content +``` with - def foo(self, *args, **kwargs): - profile.runctx('self.real_foo(*args, **kwargs)', globals(), locals(), - time.strftime('/home/user/profiling/foo-%Y%m%d-%H%M%S.pstats')) +```python +def foo(self, *args, **kwargs): + profile.runctx('self.real_foo(*args, **kwargs)', globals(), locals(), + time.strftime('/home/user/profiling/foo-%Y%m%d-%H%M%S.pstats')) - def real_foo(self, bar): - # function content +def real_foo(self, bar): + # function content +``` ### Run application @@ -88,11 +89,10 @@ This creates `index.html` with all SVG graphics linked to TXT files, ready for u make REMOTE=example.com:public_html/qubes/profiling/ upload ~~~ -Example -------- +## Example This example is from `qubes-manager` (`qubesmanager/main.py`). -!["update\_table-20140424-170010.svg"](//attachment/wiki/Profiling/update_table-20140424-170010.svg) +!["update\_table-20140424-170010.svg"](//attachment/doc/update_table-20140424-170010.svg) It is apparent that the problem is around `get_disk_usage`, which calls something via `subprocess.call`. It does this 15 times, probably once per VM. diff --git a/developer/debugging/safe-remote-ttys.md b/developer/debugging/safe-remote-ttys.md index 0568df6dd5..b618311603 100644 --- a/developer/debugging/safe-remote-ttys.md +++ b/developer/debugging/safe-remote-ttys.md @@ -1,15 +1,13 @@ --- +lang: en layout: doc -title: Safe Remote Dom0 Terminals permalink: /doc/safe-remote-ttys/ redirect_from: - /en/doc/safe-remote-ttys/ -- /doc/safe-remote-ttys/ +ref: 49 +title: Safe remote dom0 terminals --- -Safe Remote Dom0 Terminals -========================== - If you do not have working graphics in Dom0, then using a terminal can be quite annoying! This was the case for the author while trying to debug PCI-passthrough of a machine's primary (only) GPU. diff --git a/developer/debugging/test-bench.md b/developer/debugging/test-bench.md index 6f2a9019e6..ca48a5eab6 100644 --- a/developer/debugging/test-bench.md +++ b/developer/debugging/test-bench.md @@ -1,73 +1,140 @@ --- +lang: en layout: doc -title: Test Bench permalink: /doc/test-bench/ redirect_from: - /en/doc/test-bench/ - /doc/TestBench/ - /wiki/TestBench/ +ref: 44 +title: How to set up a test bench --- -Test bench for Dom0 -=================== - This guide shows how to set up simple test bench that automatically test your code you're about to push. It is written especially for `core3` branch of `core-admin.git` repo, but some ideas are universal. We will set up a spare machine (bare metal, not a virtual) that will be hosting our experimental Dom0. We will communicate with it via Ethernet and SSH. This tutorial assumes you are familiar with [QubesBuilder](/doc/qubes-builder/) and you have it set up and running flawlessly. -Setting up the machine ----------------------- +> **Notice:** +> This setup intentionally weakens some security properties in the testing system. So make sure you understand the risks and use exclusively for testing. -First, do a clean install from ISO you built or grabbed elsewhere. +## Setting up the Machine -You have to fix network, because it is intentionally broken. This script should reenable your network card without depending on anything else. +### Install ISO +First, do a clean install from the `.iso` [you built](/doc/qubes-iso-building/) or grabbed elsewhere (for example [here](https://forum.qubes-os.org/t/qubesos-4-1-alpha-signed-weekly-builds/3601)). - #!/bin/sh +### Enabling Network Access in Dom0 - # adjust this for your NIC (run lspci) - BDF=0000:02:00.0 +Internet access is intentionally disabled by default in dom0. But to ease the deployment process we will give it access. The following steps should be done in `dom0`. - prog=$(basename $0) +> **Note:** the following assume you have only one network card. If you have two, pick one and leave the other attached to `sys-net`. - pciunbind() { - local path - path=/sys/bus/pci/devices/${1}/driver/unbind - if ! [ -w ${path} ]; then - echo "${prog}: Device ${1} not bound" - return 1 - fi - echo -n ${1} >${path} - } +1. Remove the network card (PCI device) from `sys-net` +2. Restart your computer (for the removal to take effect) +3. Install `dhcp-client` and `openssh-server` on your testbench's dom0. +4. Save the following script in `/home/user/bin/dom0_network.sh` and make it executable. It should enable your network card in dom0. *Be sure to adjust the script's variables to suit your needs.* - pcibind() { - local path - path=/sys/bus/pci/drivers/${2}/bind - if ! [ -w ${path} ]; then - echo "${prog}: Driver ${2} not found" - return 1 - fi - echo ${1} >${path} - } + ```bash + #!/bin/sh - pciunbind ${BDF} - pcibind ${BDF} e1000e + # adjust this for your NIC (run lspci) + BDF=0000:02:00.0 - dhclient + # adjust this for your network driver + DRIVER=e1000e -TODO: describe how to run this at every startup + prog=$(basename $0) -Now configure your DHCP server so your testbench gets static IP and connect your machine to your local network. You should ensure that your testbench can reach the Internet. + pciunbind() { + local path + path=/sys/bus/pci/devices/${1}/driver/unbind + if ! [ -w ${path} ]; then + echo "${prog}: Device ${1} not bound" + return 1 + fi + echo -n ${1} >${path} + } -Install `openssh-server` on your testbench: + pcibind() { + local path + path=/sys/bus/pci/drivers/${2}/bind + if ! [ -w ${path} ]; then + echo "${prog}: Driver ${2} not found" + return 1 + fi + echo ${1} >${path} + } -~~~ -yum install openssh-server -~~~ + pciunbind ${BDF} + pcibind ${BDF} ${DRIVER} + + sleep 1 + dhclient + ``` + +5. Configure your DHCP server so your testbench gets static IP and connect your machine to your local network. You should ensure that your testbench can reach the Internet. + +6. You'll need to run the above script on every startup. To automate this save the following systemd service `/etc/systemd/system/dom0-network-direct.service` + + ``` + [Unit] + Description=Connect network to dom0 + + [Service] + Type=oneshot + ExecStart=/home/user/bin/dom0_network.sh + + [Install] + WantedBy=multi-user.target + ``` + +7. Then, enable and start the SSH Server and the script on boot: -Ensure that sudo works without password from your user account (it should by default). + ```bash + sudo systemctl enable sshd + sudo systemctl start sshd + + sudo systemctl enable dom0-network-direct + sudo systemctl start dom0-network-direct + ``` -Development VM --------------- +> **Note:** If you want to install additional software in dom0 and your only network card was assigned to dom0, then _instead_ of the usual `sudo qubes-dom0-update ` now you run `sudo dnf --setopt=reposdir=/etc/yum.repos.d install `. + +### Install Tests and Their Dependencies + +A regular Qubes installation isn't ready to run the full suite of tests. For example, in order to run the [Split GPG tests](https://github.com/QubesOS/qubes-app-linux-split-gpg/blob/4bc201bb70c011119eed19df25dc5b46120d04ed/tests/splitgpg/tests.py) you need to have the `qubes-gpg-split-tests` package installed in your app qubes. + +Because of the above reason, some additional configurations need to be done to your testing environment. This can be done in an automated manner with the help of the [Salt](/doc/salt) configuration that provisions the [automated testing environment](/doc/automated-tests/). + +The following commands should work for you, but do keep in mind that the provisioning scripts are designed for the [openQA environment](https://openqa.qubes-os.org/) and not your specific local testing system. Run the following in `dom0`: + + ```bash + # For future reference the following commands are an adaptation of + # https://github.com/marmarek/openqa-tests-qubesos/blob/master/tests/update.pm + + # Install git + sudo qubes-dom0-update git || sudo dnf --setopt=reposdir=/etc/yum.repos.d install git + + # Download the openQA automated testing environment Salt configuration + git clone https://github.com/marmarek/openqa-tests-qubesos/ + cd openqa-tests-qubesos/extra-files + sudo cp -a system-tests/ /srv/salt/ + sudo qubesctl top.enable system-tests + + # Install the same configuration as the one in openQA + QUBES_VERSION=4.1 + PILLAR_DIR=/srv/pillar/base/update + sudo mkdir -p $PILLAR_DIR + printf 'update:\n qubes_ver: '$QUBES_VERSION'\n' | sudo tee $PILLAR_DIR/init.sls + printf "base:\n '*':\n - update\n" | sudo tee $PILLAR_DIR/init.top + sudo qubesctl top.enable update pillar=True + + # Apply states to dom0 and VMs + # NOTE: These commands can take several minutes (if not more) without showing output + sudo qubesctl --show-output state.highstate + sudo qubesctl --max-concurrency=2 --skip-dom0 --templates --show-output state.highstate + ``` + +## Development VM ### SSH @@ -87,7 +154,9 @@ Host testbench HostName 192.168.123.45 ~~~ -Then connect to your testbench and paste newly generated `id_ecdsa.pub` to `.ssh/authorized_keys` on testbench so you can log in without entering password every time. +#### Passwordless SSH Login + +To log to your testbench without entering password every time, copy your newly generated public key (`id_ecdsa.pub`) to `~/.ssh/authorized_keys` on your testbench. You can do this easily by running this command on `qubes-dev`: `ssh-copy-id -i ~/.ssh/id_ecdsa.pub user@192.168.123.45` (substituting with the actual username address of your testbench). ### Scripting @@ -95,47 +164,53 @@ This step is optional, but very helpful. Put these scripts somewhere in your `${ `qtb-runtests`: - #!/bin/sh +```bash +#!/bin/sh - ssh testbench python -m qubes.tests.run +ssh testbench python -m qubes.tests.run +``` `qtb-install`: - #!/bin/sh +```bash +#!/bin/sh - TMPDIR=/tmp/qtb-rpms +TMPDIR=/tmp/qtb-rpms - if [ $# -eq 0 ]; then - echo "usage: $(basename $0) ..." - exit 2 - fi +if [ $# -eq 0 ]; then + echo "usage: $(basename $0) ..." + exit 2 +fi - set -e +set -e - ssh testbench mkdir -p "${TMPDIR}" - scp "${@}" testbench:"${TMPDIR}" +ssh testbench mkdir -p "${TMPDIR}" +scp "${@}" testbench:"${TMPDIR}" || echo "check if you have 'scp' installed on your testbench" - while [ $# -gt 0 ]; do - ssh testbench sudo rpm -i --replacepkgs --replacefiles "${TMPDIR}/$(basename ${1})" - shift - done +while [ $# -gt 0 ]; do + ssh testbench sudo rpm -i --replacepkgs --replacefiles "${TMPDIR}/$(basename ${1})" + shift +done +``` `qtb-iterate`: - #!/bin/sh +```bash +#!/bin/sh - set -e +set -e - # substitute path to your builder installation - pushd ${HOME}/builder >/dev/null +# substitute path to your builder installation +pushd ${HOME}/builder >/dev/null - # the following are needed only if you have sources outside builder - #rm -rf qubes-src/core-admin - #make COMPONENTS=core-admin get-sources +# the following are needed only if you have sources outside builder +#rm -rf qubes-src/core-admin +#make COMPONENTS=core-admin get-sources - make core-admin - qtb-install qubes-src/core-admin/rpm/x86_64/qubes-core-dom0-*.rpm - qtb-runtests +make core-admin +qtb-install qubes-src/core-admin/rpm/x86_64/qubes-core-dom0-*.rpm +qtb-runtests +``` ### Hooking git @@ -143,14 +218,18 @@ I (woju) have those two git hooks. They ensure tests are passing (or are marked `core-admin/.git/hooks/pre-commit`: (you may retain also the default hook, here omitted for readability) - #!/bin/sh +```bash +#!/bin/sh - set -e +set -e - python -c "import sys, qubes.tests.run; sys.exit(not qubes.tests.run.main())" +python -c "import sys, qubes.tests.run; sys.exit(not qubes.tests.run.main())" +``` `core-admin/.git/hooks/pre-push`: - #!/bin/sh +```bash +#!/bin/sh - exec qtb-iterate +exec qtb-iterate +``` diff --git a/developer/debugging/vm-interface.md b/developer/debugging/vm-interface.md index 7227c41e10..d6fc2dff0c 100644 --- a/developer/debugging/vm-interface.md +++ b/developer/debugging/vm-interface.md @@ -1,53 +1,51 @@ --- +lang: en layout: doc -title: VM Configuration Interface permalink: /doc/vm-interface/ redirect_from: - /en/doc/vm-interface/ - /doc/VMInterface/ - /doc/SystemDoc/VMInterface/ - /wiki/SystemDoc/VMInterface/ +ref: 47 +title: Qube configuration interface --- -VM Configuration Interface -========================== - Qubes VM have some settings set by dom0 based on VM settings. There are multiple configuration channels, which includes: -- QubesDB -- XenStore (in Qubes 2, data the same as in QubesDB, keys without leading `/`) -- Qubes RPC (called at VM startup, or when configuration changed) -- GUI protocol - -QubesDB --------------------- - -### Keys exposed by dom0 to VM ### - -- `/qubes-vm-type` - VM type, the same as `type` field in `qvm-prefs`. One of `AppVM`, `ProxyVM`, `NetVM`, `TemplateVM`, `HVM`, `TemplateHVM` -- `/qubes-vm-updatable` - flag whether VM is updatable (whether changes in root.img will survive VM restart). One of `True`, `False` -- `/qubes-vm-persistence` - what data do persist between VM restarts: - - `full` - all disks - - `rw-only` - only `/rw` disk - - `none` - none -- `/qubes-timezone - name of timezone based on dom0 timezone. For example `Europe/Warsaw` -- `/qubes-keyboard` (deprecated in R4.1) - keyboard layout based on dom0 layout. Its syntax is suitable for `xkbcomp` command (after expanding escape sequences like `\n` or `\t`). This is meant only as some default value, VM can ignore this option and choose its own keyboard layout (this is what keyboard setting from Qubes Manager does). This entry is created as part of gui-daemon initialization (so not available when gui-daemon disabled, or not started yet). -- `/keyboard-layout` - keyboard layout based on GuiVM layout. Its syntax can be `layout+variant+options`, `layout+variant`, `layout++options` or simply `layout`. For example, `fr+oss`, `pl++compose:caps` or `fr`. This is meant only as some default value, VM can ignore this option and choose its own keyboard layout (this is what keyboard setting from Qubes Manager does). -- `/qubes-debug-mode` - flag whether VM has debug mode enabled (qvm-prefs setting). One of `1`, `0` -- `/qubes-service/SERVICE_NAME` - subtree for VM services controlled from dom0 (using the `qvm-service` command or Qubes Manager). One of `1`, `0`. Note that not every service will be listed here, if entry is missing, it means "use VM default". A list of currently supported services is in the `qvm-service` man page. -- `/qubes-netmask` - network mask (only when VM has netvm set); currently hardcoded "255.255.255.0" -- `/qubes-ip - IP address for this VM (only when VM has netvm set) -- `/qubes-gateway` - default gateway IP (only when VM has netvm set); VM should add host route to this address directly via eth0 (or whatever default interface name is) -- `/qubes-primary-dns` - primary DNS address (only when VM has netvm set) -- `/qubes-secondary-dns` - secondary DNS address (only when VM has netvm set) -- `/qubes-netvm-gateway` - same as `qubes-gateway` in connected VMs (only when VM serves as network backend - ProxyVM and NetVM) -- `/qubes-netvm-netmask` - same as `qubes-netmask` in connected VMs (only when VM serves as network backend - ProxyVM and NetVM) -- `/qubes-netvm-network` - network address (only when VM serves as network backend - ProxyVM and NetVM); can be also calculated from qubes-netvm-gateway and qubes-netvm-netmask -- `/qubes-netvm-primary-dns` - same as `qubes-primary-dns` in connected VMs (only when VM serves as network backend - ProxyVM and NetVM); traffic sent to this IP on port 53 should be redirected to primary DNS server -- `/qubes-netvm-secondary-dns` - same as `qubes-secondary-dns` in connected VMs (only when VM serves as network backend - ProxyVM and NetVM); traffic sent to this IP on port 53 should be redirected to secondary DNS server -- `/guivm-windows-prefix` - title prefix for any window not originating from another qube. This means windows of applications running in GuiVM itself - -#### Firewall rules in 3.x #### +- QubesDB +- XenStore (in Qubes 2, data the same as in QubesDB, keys without leading `/`) +- Qubes RPC (called at VM startup, or when configuration changed) +- GUI protocol + +## QubesDB + +### Keys exposed by dom0 to VM + +- `/qubes-vm-type` - VM type, the same as `type` field in `qvm-prefs`. One of `AppVM`, `ProxyVM`, `NetVM`, `TemplateVM`, `HVM`, `TemplateHVM` +- `/qubes-vm-updatable` - flag whether VM is updatable (whether changes in root.img will survive VM restart). One of `True`, `False` +- `/qubes-vm-persistence` - what data do persist between VM restarts: + - `full` - all disks + - `rw-only` - only `/rw` disk + - `none` - none +- `/qubes-timezone - name of timezone based on dom0 timezone. For example `Europe/Warsaw` +- `/qubes-keyboard` (deprecated in R4.1) - keyboard layout based on dom0 layout. Its syntax is suitable for `xkbcomp` command (after expanding escape sequences like `\n` or `\t`). This is meant only as some default value, VM can ignore this option and choose its own keyboard layout (this is what keyboard setting from Qubes Manager does). This entry is created as part of gui-daemon initialization (so not available when gui-daemon disabled, or not started yet). +- `/keyboard-layout` - keyboard layout based on GuiVM layout. Its syntax can be `layout+variant+options`, `layout+variant`, `layout++options` or simply `layout`. For example, `fr+oss`, `pl++compose:caps` or `fr`. This is meant only as some default value, VM can ignore this option and choose its own keyboard layout (this is what keyboard setting from Qubes Manager does). +- `/qubes-debug-mode` - flag whether VM has debug mode enabled (qvm-prefs setting). One of `1`, `0` +- `/qubes-service/SERVICE_NAME` - subtree for VM services controlled from dom0 (using the `qvm-service` command or Qubes Manager). One of `1`, `0`. Note that not every service will be listed here, if entry is missing, it means "use VM default". A list of currently supported services is in the `qvm-service` man page. +- `/qubes-netmask` - network mask (only when VM has netvm set); currently hardcoded "255.255.255.0" +- `/qubes-ip - IP address for this VM (only when VM has netvm set) +- `/qubes-gateway` - default gateway IP (only when VM has netvm set); VM should add host route to this address directly via eth0 (or whatever default interface name is) +- `/qubes-primary-dns` - primary DNS address (only when VM has netvm set) +- `/qubes-secondary-dns` - secondary DNS address (only when VM has netvm set) +- `/qubes-netvm-gateway` - same as `qubes-gateway` in connected VMs (only when VM serves as network backend - ProxyVM and NetVM) +- `/qubes-netvm-netmask` - same as `qubes-netmask` in connected VMs (only when VM serves as network backend - ProxyVM and NetVM) +- `/qubes-netvm-network` - network address (only when VM serves as network backend - ProxyVM and NetVM); can be also calculated from qubes-netvm-gateway and qubes-netvm-netmask +- `/qubes-netvm-primary-dns` - same as `qubes-primary-dns` in connected VMs (only when VM serves as network backend - ProxyVM and NetVM); traffic sent to this IP on port 53 should be redirected to primary DNS server +- `/qubes-netvm-secondary-dns` - same as `qubes-secondary-dns` in connected VMs (only when VM serves as network backend - ProxyVM and NetVM); traffic sent to this IP on port 53 should be redirected to secondary DNS server +- `/guivm-windows-prefix` - title prefix for any window not originating from another qube. This means windows of applications running in GuiVM itself + +#### Firewall rules in 3.x QubesDB is also used to configure firewall in ProxyVMs. Rules are stored in separate key for each target VM. Entries: @@ -63,7 +61,7 @@ VM after applying rules may signal some error, writing a message to `/qubes-iptables-error` key. This does not exclude any other way of communicating problems - like a popup. -#### Firewall rules in 4.x #### +#### Firewall rules in 4.x QubesDB is also used to configure firewall in ProxyVMs. Each rule is stored as a separate entry, grouped on target VM: @@ -85,19 +83,19 @@ Each rule is a single QubesDB entry, consisting of pairs `key=value` separated by space. QubesDB enforces limit on a single entry length - 3072 bytes. Possible options for a single rule: - - `action`, values: `accept`, `drop`; this is present in every rule - - `dst4`, value: destination IPv4 address with a mask; for example: `192.168.0.0/24` - - `dst6`, value: destination IPv6 address with a mask; for example: `2000::/3` - - `dsthost`, value: DNS hostname of destination host - - `proto`, values: `tcp`, `udp`, `icmp` - - `specialtarget`, value: One of predefined target, currently defined values: - - `dns` - such option should match DNS traffic to default DNS server (but +- `action`, values: `accept`, `drop`; this is present in every rule +- `dst4`, value: destination IPv4 address with a mask; for example: `192.168.0.0/24` +- `dst6`, value: destination IPv6 address with a mask; for example: `2000::/3` +- `dsthost`, value: DNS hostname of destination host +- `proto`, values: `tcp`, `udp`, `icmp` +- `specialtarget`, value: One of predefined target, currently defined values: + - `dns` - such option should match DNS traffic to default DNS server (but not any DNS server), on both TCP and UDP - - `dstports`, value: destination ports range separated with `-`, valid only +- `dstports`, value: destination ports range separated with `-`, valid only together with `proto=tcp` or `proto=udp`; for example `1-1024`, `80-80` - - `icmptype`, value: numeric (decimal) icmp message type, for example `8` for +- `icmptype`, value: numeric (decimal) icmp message type, for example `8` for echo request, valid only together with `proto=icmp` - - `dpi`, value: Deep Packet Inspection protocol (like: HTTP, SSL, SMB, SSH, SMTP) or the default 'NO' as no DPI, only packet filtering +- `dpi`, value: Deep Packet Inspection protocol (like: HTTP, SSL, SMB, SSH, SMTP) or the default 'NO' as no DPI, only packet filtering Options must appear in the rule in the order listed above. Duplicated options are forbidden. @@ -117,48 +115,46 @@ Example valid rules: - `action=drop proto=tcp specialtarget=dns` - drop DNS queries sent using TCP - `action=drop` -### Keys set by VM for passing info to dom0 ### +### Keys set by VM for passing info to dom0 -- `memory/meminfo` (**xenstore**) - used memory (updated by qubes-meminfo-writer), input information for qmemman; - - Qubes 3.x format: 6 lines (EOL encoded as `\n`), each in format "FIELD: VALUE kB"; fields: `MemTotal`, `MemFree`, `Buffers`, `Cached`, `SwapTotal`, `SwapFree`; meaning the same as in `/proc/meminfo` in Linux. - - Qubes 4.0+ format: used memory size in the VM, in kbytes -- `/qubes-block-devices` - list of block devices exposed by this VM, each device (subdirectory) should be named in a way that VM can attach the device based on it. Each should contain these entries: - - `desc` - device description (ASCII text) - - `size` - device size in bytes - - `mode` - default connection mode; `r` for read-only, `w` for read-write -- `/qubes-usb-devices` - list of USB devices exposed by this VM, each device (subdirectory) should contain: - - `desc` - device description (ASCII text) - - `usb-ver` - USB version (1, 2 or 3) +- `memory/meminfo` (**xenstore**) - used memory (updated by qubes-meminfo-writer), input information for qmemman; + - Qubes 3.x format: 6 lines (EOL encoded as `\n`), each in format "FIELD: VALUE kB"; fields: `MemTotal`, `MemFree`, `Buffers`, `Cached`, `SwapTotal`, `SwapFree`; meaning the same as in `/proc/meminfo` in Linux. + - Qubes 4.0+ format: used memory size in the VM, in kbytes +- `/qubes-block-devices` - list of block devices exposed by this VM, each device (subdirectory) should be named in a way that VM can attach the device based on it. Each should contain these entries: + - `desc` - device description (ASCII text) + - `size` - device size in bytes + - `mode` - default connection mode; `r` for read-only, `w` for read-write +- `/qubes-usb-devices` - list of USB devices exposed by this VM, each device (subdirectory) should contain: + - `desc` - device description (ASCII text) + - `usb-ver` - USB version (1, 2 or 3) -Qubes RPC ---------- +## Qubes RPC Services called by dom0 to provide some VM configuration: -- `qubes.SetMonitorLayout` - provide list of monitors, one per line. Each line contains four numbers: `width height X Y width_mm height_mm` (physical dimensions - `width_mm` and `height_mm` - are optional) -- `qubes.WaitForSession` - called to wait for full VM startup -- `qubes.GetAppmenus` - receive appmenus from given VM (template); TODO: describe format here -- `qubes.GetImageRGBA` - receive image/application icon. Protocol: +- `qubes.SetMonitorLayout` - provide list of monitors, one per line. Each line contains four numbers: `width height X Y width_mm height_mm` (physical dimensions - `width_mm` and `height_mm` - are optional) +- `qubes.WaitForSession` - called to wait for full VM startup +- `qubes.GetAppmenus` - receive appmenus from given VM (template); TODO: describe format here +- `qubes.GetImageRGBA` - receive image/application icon. Protocol: - 1. Caller sends name of requested icon. This can be one of: - * `xdgicon:NAME` - search for NAME in standard icons theme - * `-` - get icon data from stdin (the caller), can be prefixed with format name, for example `png:-` - * file name - 2. The service responds with image dimensions: width and height as + 1. Caller sends name of requested icon. This can be one of: + * `xdgicon:NAME` - search for NAME in standard icons theme + * `-` - get icon data from stdin (the caller), can be prefixed with format name, for example `png:-` + * file name + 2. The service responds with image dimensions: width and height as decimal numbers, separated with space and with EOL marker at the and; then image data in RGBA format (32 bits per pixel) -- `qubes.SetDateTime` - set VM time, called periodically by dom0 (can be +- `qubes.SetDateTime` - set VM time, called periodically by dom0 (can be triggered manually from dom0 by calling `qvm-sync-clock`). The service receives one line at stdin - time in format of `date -u -Iseconds`, for example `2015-07-31T16:10:43+0000`. -- `qubes.SetGuiMode` - called in HVM to switch between fullscreen and seamless +- `qubes.SetGuiMode` - called in HVM to switch between fullscreen and seamless GUI mode. The service receives a single word on stdin - either `FULLSCREEN` or `SEAMLESS` -- `qubes.ResizeDisk` - called to inform that underlying disk was resized. +- `qubes.ResizeDisk` - called to inform that underlying disk was resized. Name of disk image is passed on standard input (`root`, `private`, `volatile`, or other). This is used starting with Qubes 4.0. - Other Qrexec services installed by default: - `qubes.Backup` - store Qubes backup. The service receives location chosen by @@ -193,19 +189,19 @@ Other Qrexec services installed by default: can send icon for the same window multiple times to replace previous one (for example for animated icons) - `qubes.VMShell` - call any command in the VM; the command(s) is passed one per line - - `qubes.VMShell+WaitForSession` waits for full VM startup first + - `qubes.VMShell+WaitForSession` waits for full VM startup first - `qubes.VMExec` - call any command in the VM, without using shell, the command needs to be passed as argument and encoded as follows: - - the executable name and arguments are separated by `+` - - everything except alphanumeric characters, `.` and `_` needs to be + - the executable name and arguments are separated by `+` + - everything except alphanumeric characters, `.` and `_` needs to be escaped - - bytes are escaped as `-HH` (where `HH` is hex code, capital letters only) - - `-` itself can be escaped as `--` - - example: to run `ls -a /home/user`, use + - bytes are escaped as `-HH` (where `HH` is hex code, capital letters only) + - `-` itself can be escaped as `--` + - example: to run `ls -a /home/user`, use `qubes.VMExec+ls+--a+-2Fhome-2Fuser` - `qubes.VMExecGUI` - a variant of `qubes.VMExec` that waits for full VM startup first - + Services called in GuiVM: - `policy.Ask`, `policy.Notify` - confirmation prompt and notifications for @@ -224,7 +220,6 @@ abstraction. This will change in the future. Those tools are: Additionally, automatic tests extensively run various commands directly in VMs. We do not plan to change that. -GUI protocol ------------- +## GUI protocol GUI initialization includes passing the whole screen dimensions from dom0 to VM. This will most likely be overwritten by qubes.SetMonitorLayout Qubes RPC call. diff --git a/developer/debugging/windows-debugging.md b/developer/debugging/windows-debugging.md index d1b2dd186e..7c2c14fcf4 100644 --- a/developer/debugging/windows-debugging.md +++ b/developer/debugging/windows-debugging.md @@ -1,53 +1,56 @@ --- +lang: en layout: doc -title: Windows Debugging permalink: /doc/windows-debugging/ redirect_from: - /en/doc/windows-debugging/ - /doc/WindowsDebugging/ - /wiki/WindowsDebugging/ +ref: 50 +title: Windows debugging --- -Debugging Windows HVMs -====================== - Debugging Windows code can be tricky in a virtualized environment. The guide below assumes Xen hypervisor and Windows 7 VMs. User-mode debugging is usually straightforward if it can be done on one machine. Just duplicate your normal debugging environment in the VM. Things get complicated if you need to perform kernel debugging or troubleshoot problems that only manifest on system boot, user logoff or similar. For that you need two Windows VMs: the *host* and the *target*. The *host* will contain [WinDbg](https://msdn.microsoft.com/en-us/library/windows/hardware/ff551063(v=vs.85).aspx) installation, your source code and private symbols. The *target* will run the code being debugged. Both will be linked by virtual serial ports. -- First, you need to prepare separate copies of both *target* and *host* VM configuration files with some changes. Copy the files from **/var/lib/qubes/appvms/vmname/vmname.conf** to some convenient location, let's call them **host.conf** and **target.conf**. -- In both copied files add the following line at the end: `serial = 'pty'`. This will make Xen connect VM's serial ports to dom0's ptys. -- From now on you need to start both VMs like this: `qvm-start --custom-config=/your/edited/host.conf host` -- To connect both VM serial ports together you will either need [socat](http://www.dest-unreach.org/socat/) or a custom utility described later. -- To determine which dom0 pty corresponds to VM's serial port you need to read xenstore, example script below: +- First, you need to prepare separate copies of both *target* and *host* VM configuration files with some changes. Copy the files from **/var/lib/qubes/appvms/vmname/vmname.conf** to some convenient location, let's call them **host.conf** and **target.conf**. +- In both copied files add the following line at the end: `serial = 'pty'`. This will make Xen connect VM's serial ports to dom0's ptys. +- From now on you need to start both VMs like this: `qvm-start --custom-config=/your/edited/host.conf host` +- To connect both VM serial ports together you will either need [socat](http://www.dest-unreach.org/socat/) or a custom utility described later. +- To determine which dom0 pty corresponds to VM's serial port you need to read xenstore, example script below: - #!/bin/sh +```bash +#!/bin/sh - id1=$(xl domid "$1-dm") - tty1=$(xenstore-read /local/domain/${id1}/device/console/3/tty) - echo $tty1 +id1=$(xl domid "$1-dm") +tty1=$(xenstore-read /local/domain/${id1}/device/console/3/tty) +echo $tty1 +``` - Pass it a running VM name and it will output the corresponding pty name. +Pass it a running VM name and it will output the corresponding pty name. -- To connect both ptys you can use [socat](http://www.dest-unreach.org/socat/) like that: +- To connect both ptys you can use [socat](http://www.dest-unreach.org/socat/) like that: - #!/bin/sh +```bash +#!/bin/sh - id1=$(xl domid "$1-dm") - id2=$(xl domid "$2-dm") - tty1=$(xenstore-read /local/domain/${id1}/device/console/3/tty) - tty2=$(xenstore-read /local/domain/${id2}/device/console/3/tty) - socat $tty1,raw $tty2,raw +id1=$(xl domid "$1-dm") +id2=$(xl domid "$2-dm") +tty1=$(xenstore-read /local/domain/${id1}/device/console/3/tty) +tty2=$(xenstore-read /local/domain/${id2}/device/console/3/tty) +socat $tty1,raw $tty2,raw +``` - ...but there is a catch. Xen seems to process the traffic that goes through serial ports and changes all **0x0a** bytes into **0x0d, 0x0a** pairs (newline conversion). I didn't find a way to turn that off (setting ptys to raw mode didn't change anything) and it's not mentioned anywhere on the Internet, so maybe it's something on my system. If the above script works for you then you don't need anything more in dom0. +...but there is a catch. Xen seems to process the traffic that goes through serial ports and changes all **0x0a** bytes into **0x0d, 0x0a** pairs (newline conversion). I didn't find a way to turn that off (setting ptys to raw mode didn't change anything) and it's not mentioned anywhere on the Internet, so maybe it's something on my system. If the above script works for you then you don't need anything more in dom0. -- On the *target* system, run `bcdedit /set debug on` on the console to turn on kernel debugging. It defaults to the first serial port. -- On the *host* system, install [WinDbg](http://msdn.microsoft.com/en-us/library/windows/hardware/ff551063(v=vs.85).aspx) and start the kernel debug (Ctrl-K), choose **com1** as the debug port. -- Reboot the *target* VM. -- Run the above shell script in dom0. -- If everything is fine you should see the proper kernel debugging output in WinDbg. However, if you see something like that: +- On the *target* system, run `bcdedit /set debug on` on the console to turn on kernel debugging. It defaults to the first serial port. +- On the *host* system, install [WinDbg](http://msdn.microsoft.com/en-us/library/windows/hardware/ff551063(v=vs.85).aspx) and start the kernel debug (Ctrl-K), choose **com1** as the debug port. +- Reboot the *target* VM. +- Run the above shell script in dom0. +- If everything is fine you should see the proper kernel debugging output in WinDbg. However, if you see something like that: ~~~ Opened \\.\com1 @@ -55,7 +58,7 @@ Things get complicated if you need to perform kernel debugging or troubleshoot p Connected to Windows 7 7601 x64 target at (Wed Mar 19 20:35:43.262 2014 (UTC + 1:00)), ptr64 TRUE Kernel Debugger connection established. Symbol search path is: srv*c:\symbols*http://msdl.microsoft.com/download/symbols - Executable search path is: + Executable search path is: ... Retry sending the same data packet for 64 times. The transport connection between host kernel debugger and target Windows seems lost. please try resync with target, recycle the host debugger, or reboot the target Windows. @@ -73,115 +76,117 @@ Things get complicated if you need to perform kernel debugging or troubleshoot p ...then you're most likely a victim of the CRLF issue mentioned above. To get around it I wrote a small utility that basically does what socat would do and additionally corrects those replaced bytes in the stream. It's not pretty but it works: - #include - #include - #include - #include - - int fd1, fd2; - char mark = ' '; - - void out(unsigned char c) +```c +#include +#include +#include +#include + +int fd1, fd2; +char mark = ' '; + +void out(unsigned char c) +{ + static int count = 0; + static unsigned char buf[17] = {0}; + + // relay to ouptput port + write(fd2, &c, 1); + fprintf(stderr, "%c", mark); + + /* dump all data going over the line + if (count == 0) + fprintf(stderr, "%c", mark); + fprintf(stderr, "%02x ", c); + if (c >= 0x20 && c < 0x80) + buf[count] = c; + else + buf[count] = '.'; + count++; + if (count == 0x10) + { + count = 0; + fprintf(stderr, " %s\n", buf); + } + */ +} + +int main(int argc, char* argv[]) +{ + unsigned char c = 0; + struct termios tio; + ssize_t size; + + if (argc < 3) + { + fprintf(stderr, "Usage: %s pty1 pty2 [mark character]\n", argv[0]); + return EINVAL; + } + + fd1 = open(argv[1], O_RDONLY | O_NOCTTY); + if (fd1 <= 0) + { + perror("open fd1"); + return errno; + } + fd2 = open(argv[2], O_WRONLY | O_NOCTTY); + if (fd2 <= 0) + { + perror("open fd2"); + return errno; + } +/* + // This doesn't make any difference which supports the theory + // that it's Xen who corrupts the byte stream. + cfmakeraw(&tio); + if (tcsetattr(fd1, TCSANOW, &tio) < 0) + { + perror("tcsetattr 1"); + return errno; + } + if (tcsetattr(fd2, TCSANOW, &tio) < 0) + { + perror("tcsetattr 2"); + return errno; + } +*/ + if (argc == 4) + mark = argv[3][0]; + + while (1) + { + size = read(fd1, &c, 1); + if (size <= 0) + break; + +parse: + if (c == 0x0d) { - static int count = 0; - static unsigned char buf[17] = {0}; - - // relay to ouptput port - write(fd2, &c, 1); - fprintf(stderr, "%c", mark); - - /* dump all data going over the line - if (count == 0) - fprintf(stderr, "%c", mark); - fprintf(stderr, "%02x ", c); - if (c >= 0x20 && c < 0x80) - buf[count] = c; - else - buf[count] = '.'; - count++; - if (count == 0x10) + size = read(fd1, &c, 1); + if (size <= 0) { - count = 0; - fprintf(stderr, " %s\n", buf); + out(0x0d); + break; } - */ - } - - int main(int argc, char* argv[]) - { - unsigned char c = 0; - struct termios tio; - ssize_t size; - - if (argc < 3) + if (c == 0x0a) { - fprintf(stderr, "Usage: %s pty1 pty2 [mark character]\n", argv[0]); - return EINVAL; + out(0x0a); } - - fd1 = open(argv[1], O_RDONLY | O_NOCTTY); - if (fd1 <= 0) - { - perror("open fd1"); - return errno; - } - fd2 = open(argv[2], O_WRONLY | O_NOCTTY); - if (fd2 <= 0) - { - perror("open fd2"); - return errno; - } - /* - // This doesn't make any difference which supports the theory - // that it's Xen who corrupts the byte stream. - cfmakeraw(&tio); - if (tcsetattr(fd1, TCSANOW, &tio) < 0) - { - perror("tcsetattr 1"); - return errno; - } - if (tcsetattr(fd2, TCSANOW, &tio) < 0) - { - perror("tcsetattr 2"); - return errno; - } - */ - if (argc == 4) - mark = argv[3][0]; - - while (1) + else { - size = read(fd1, &c, 1); - if (size <= 0) - break; - - parse: - if (c == 0x0d) - { - size = read(fd1, &c, 1); - if (size <= 0) - { - out(0x0d); - break; - } - if (c == 0x0a) - { - out(0x0a); - } - else - { - out(0x0d); - goto parse; - } - } - else - out(c); + out(0x0d); + goto parse; } - - close(fd1); - close(fd2); - return 0; } + else + out(c); + } + + close(fd1); + close(fd2); + return 0; +} +``` > This utility is a unidirectional relay so you need to run two instances to get duplex communication, like: > @@ -193,7 +198,6 @@ Things get complicated if you need to perform kernel debugging or troubleshoot p > tty2=$(xenstore-read /local/domain/${id2}/device/console/3/tty) > ./ptycrlf ${tty1} ${tty2} - & > ./ptycrlf ${tty2} ${tty1} + & - > With this everything should be good: > > ~~~ @@ -202,7 +206,7 @@ Things get complicated if you need to perform kernel debugging or troubleshoot p > Connected to Windows 7 7601 x64 target at (Wed Mar 19 20:56:31.371 2014 (UTC + 1:00)), ptr64 TRUE > Kernel Debugger connection established. > Symbol search path is: srv*c:\symbols*http://msdl.microsoft.com/download/symbols -> Executable search path is: +> Executable search path is: > Windows 7 Kernel Version 7601 MP (1 procs) Free x64 > Built by: 7601.18247.amd64fre.win7sp1_gdr.130828-1532 > Machine Name: @@ -214,7 +218,7 @@ Things get complicated if you need to perform kernel debugging or troubleshoot p There are two main issues to be adopted to get all things to work in the R4.0. -## Add a virtual serial port ## +## Add a virtual serial port Qemu in the stub domain with virtual serial port added in a recommended way (``````) fails to start (Could not open '/dev/hvc1': No such device). It seems like a lack of multiple xen consoles support/configuration. The only way that I have found is to attach serial port explicitly to the available console. @@ -228,15 +232,17 @@ $ gunzip stubdom-linux-rootfs.gz $ cpio -i -d -H newc --no-absolute-filenames < stubdom-linux-rootfs $ rm stubdom-linux-rootfs ``` + 2. Edit Init script to remove last loop and to add "-serial /dev/hvc0" to the qemu command line. 3. Apply changes: + ```shell_session $ find . -print0 | cpio --null -ov --format=newc | gzip -9 > ../stubdom-linux-rootfs $ sudo mv ../stubdom-linux-rootfs /usr/lib/xen/boot ``` -## Connect two consoles ## +## Connect two consoles Run the following script: diff --git a/developer/general/continuous-integration.md b/developer/general/continuous-integration.md new file mode 100644 index 0000000000..611519e5c3 --- /dev/null +++ b/developer/general/continuous-integration.md @@ -0,0 +1,37 @@ +--- +lang: en +layout: doc +permalink: /doc/continuous-integration/ +title: Continuous integration (CI) +--- + +This page explains the [continuous integration +(CI)](https://en.wikipedia.org/wiki/Continuous_integration) infrastructure that +the Qubes OS Project uses. + +## Website and documentation + +The following commands may be useful as a way to interact with our CI +infrastructure on website +([qubesos.github.io](https://github.com/QubesOS/qubesos.github.io)) and +documentation ([qubes-doc](https://github.com/QubesOS/qubes-doc)) pull requests +(PRs). Note that special permissions may be required to use some of these +commands. These commands are generally issued by adding a comment to a PR +containing only the command. + +- `PipelineRetry`: Attempts to run the entire build pipeline over again. This + can be useful if CI incorrectly uses a stale branch instead of testing the PR + as if it were merged into `master`. + +- `PipelineRefresh`: Like `PipelineRetry`, except it only fetches the job status + from GitLab. It doesn't schedule a new build. + +- `TestDeploy`: Deploys a test website, which is a live version of the Qubes + website as if this PR had been merged. This can be useful for previewing a PR + on a live public website. **Note:** You must wait for the site to finish + building before issuing this command, or else it will deploy an empty + website. To find the URL of the test website, look for text similar to "This + branch was successfully deployed" and a button named something like "View + deployment." Note that there are two different testing sites: `wwwtest` is + manually updated, whereas `wwwpreview` is managed by the `TestDeploy` + command. diff --git a/developer/general/devel-books.md b/developer/general/devel-books.md index 4284b89323..d7ae9ae075 100644 --- a/developer/general/devel-books.md +++ b/developer/general/devel-books.md @@ -1,29 +1,31 @@ --- +lang: en layout: doc -title: Developer Books permalink: /doc/devel-books/ redirect_from: - /en/doc/devel-books/ - /doc/DevelBooks/ - /wiki/DevelBooks/ +ref: 32 +title: Developer books --- Below is a list of various books that might be useful in learning some basics needed for Qubes development. - A must-read about Xen internals: - * _[The Definitive Guide to the Xen Hypervisor](https://www.amazon.com/Definitive-Guide-Xen-Hypervisor/dp/013234971X)_, by David Chisnall + - _[The Definitive Guide to the Xen Hypervisor](https://www.amazon.com/Definitive-Guide-Xen-Hypervisor/dp/013234971X)_, by David Chisnall - Some good books about the Linux kernel: - * _[Linux Kernel Development](https://www.amazon.com/Linux-Kernel-Development-Robert-Love/dp/0672329468)_, by Robert Love - * _[Linux Device Drivers](https://www.amazon.com/Linux-Device-Drivers-Jonathan-Corbet/dp/0596005903)_, by Jonathan Corbet + - _[Linux Kernel Development](https://www.amazon.com/Linux-Kernel-Development-Robert-Love/dp/0672329468)_, by Robert Love + - _[Linux Device Drivers](https://www.amazon.com/Linux-Device-Drivers-Jonathan-Corbet/dp/0596005903)_, by Jonathan Corbet - Solid intro into Trusted Computing: - * _[Dynamics of a Trusted Platform](https://www.amazon.com/Dynamics-Trusted-Platform-Buildin-Grawrock/dp/1934053082)_, by David Grawrock (original Intel architect for TXT) + - _[Dynamics of a Trusted Platform](https://www.amazon.com/Dynamics-Trusted-Platform-Buildin-Grawrock/dp/1934053082)_, by David Grawrock (original Intel architect for TXT) - Good book about GIT: - * _[Pro Git](https://git-scm.com/book/en/v2)_, by Scott Chacon and Ben Straub (complete book available free online) + - _[Pro Git](https://git-scm.com/book/en/v2)_, by Scott Chacon and Ben Straub (complete book available free online) - Useful books about Python: - * _[Programming in Python 3](http://www.qtrac.eu/py3book.html)_, by Mark Summerfield - * _[Rapid GUI Programming with Python and Qt](http://www.qtrac.eu/pyqtbook.html)_, by Mark Summerfield + - _[Programming in Python 3](http://www.qtrac.eu/py3book.html)_, by Mark Summerfield + - _[Rapid GUI Programming with Python and Qt](http://www.qtrac.eu/pyqtbook.html)_, by Mark Summerfield (Although note that [Qt is being replaced by GTK](/doc/usability-ux/#gnome-kde-and-xfce) in Qubes code.) diff --git a/developer/general/doc-guidelines.md b/developer/general/doc-guidelines.md index 55f8d9a810..55845b994c 100644 --- a/developer/general/doc-guidelines.md +++ b/developer/general/doc-guidelines.md @@ -1,145 +1,294 @@ --- +lang: en layout: doc -title: Documentation Guidelines permalink: /doc/doc-guidelines/ redirect_from: - /en/doc/doc-guidelines/ - /wiki/DocStyle/ - /doc/DocStyle/ +ref: 30 +title: Documentation Guidelines --- -Documentation Guidelines -======================== - -All Qubes OS documentation pages are stored as plain text files in the dedicated [qubes-doc] repository. -By cloning and regularly pulling from this repo, users can maintain their own up-to-date offline copy of all Qubes documentation rather than relying solely on the web. - -The documentation is a community effort. Volunteers work hard trying to keep everything accurate and comprehensive. -If you notice a problem or some way it can be improved, please [edit the documentation][contribute]! - - -Security --------- - -All pull requests against [qubes-doc] must pass review prior to be merged, except in the case of [external documentation] (see [#4693]). -This process is designed to ensure that contributed text is accurate and non-malicious. -This process is a best effort that should provide a reasonable degree of assurance, but it is not foolproof. -For example, all text characters are checked for ANSI escape sequences. -However, binaries, such as images, are simply checked to ensure they appear or function the way they should when the website is rendered. -They are not further analyzed in an attempt to determine whether they are malicious. - -Once a pull request passes review, the reviewer should add a signed comment stating, "Passed review as of ``" (or similar). -The documentation maintainer then verifies that the pull request is mechanically sound (no merge conflicts, broken links, ANSI escapes, etc.). -If so, the documentation maintainer then merges the pull request, adds a PGP-signed tag to the latest commit (usually the merge commit), then pushes to the remote. -In cases in which another reviewer is not required, the documentation maintainer may review the pull request (in which case no signed comment is necessary, since it would be redundant with the signed tag). - - -Questions, problems, and improvements -------------------------------------- - -If you have a question about something you read in the documentation, please send it to the appropriate [mailing list][support]. -If you see that something in the documentation should be fixed or improved, please [contribute] the change yourself. -To report an issue with the documentation, please follow our standard [issue reporting guidelines][issue]. -(If you report an issue with the documentation, you will likely be asked to address it, unless there is a clear indication in your report that you are not willing or able to do so.) - - -How to Contribute ------------------ - -Editing the documentation is easy, so if you see that a change should be made, please contribute it! +All Qubes OS documentation pages are stored as plain text files in the +dedicated [qubes-doc](https://github.com/QubesOS/qubes-doc) repository. By +cloning and regularly pulling from this repo, users can maintain their own +up-to-date offline copy of all Qubes documentation rather than relying solely +on the web. + +The documentation is a community effort. Volunteers work hard trying to keep +everything accurate and comprehensive. If you notice a problem or some way it +can be improved, please [edit the documentation](#how-to-contribute)! + +## Security + +*Also see: [Should I trust this website?](/faq/#should-i-trust-this-website)* + +All pull requests (PRs) against +[qubes-doc](https://github.com/QubesOS/qubes-doc) must pass review prior to be +merged, except in the case of [external +documentation](/doc/#external-documentation) (see +[#4693](https://github.com/QubesOS/qubes-issues/issues/4693)). This process is +designed to ensure that contributed text is accurate and non-malicious. This +process is a best effort that should provide a reasonable degree of assurance, +but it is not foolproof. For example, all text characters are checked for ANSI +escape sequences. However, binaries, such as images, are simply checked to +ensure they appear or function the way they should when the website is +rendered. They are not further analyzed in an attempt to determine whether they +are malicious. + +Once a pull request passes review, the reviewer should add a signed comment +stating, "Passed review as of ``" (or similar). The +documentation maintainer then verifies that the pull request is mechanically +sound (no merge conflicts, broken links, ANSI escapes, etc.). If so, the +documentation maintainer then merges the pull request, adds a PGP-signed tag to +the latest commit (usually the merge commit), then pushes to the remote. In +cases in which another reviewer is not required, the documentation maintainer +may review the pull request (in which case no signed comment is necessary, +since it would be redundant with the signed tag). + +## Questions, problems, and improvements + +If you have a question about something you read in the documentation, please +send it to the appropriate [mailing list](/support/). If you see that something +in the documentation should be fixed or improved, please +[contribute](#how-to-contribute) the change yourself. To report an issue with +the documentation, please follow our standard [issue reporting +guidelines](/doc/issue-tracking/). (If you report an issue with the +documentation, you will likely be asked to address it, unless there is a clear +indication in your report that you are not willing or able to do so.) + +## How to contribute + +Editing the documentation is easy, so if you see that a change should be made, +please contribute it! A few notes before we get started: - * Since Qubes is a security-oriented project, every documentation change will be reviewed before it's accepted. - This allows us to maintain quality control and protect our users. - * We don't want you to spend time and effort on a contribution that we can't accept. - If your contribution would take a lot of time, please [file an issue][issue] for it first so that we can make sure we're on the same page before significant works begins. - * Alternatively, you may already have written content that doesn't conform to these guidelines, but you'd be willing to modify it so that it does. - In this case, you can still submit it by following the instructions below. - Just make a note in your pull request that you're aware of the changes that need to be made and that you're just asking for the content to be reviewed before you spend time making those changes. +* Since Qubes is a security-oriented project, every documentation change will + be reviewed before it's accepted. This allows us to maintain quality control + and protect our users. +* We don't want you to spend time and effort on a contribution that we can't + accept. If your contribution would take a lot of time, please [file an + issue](/doc/issue-tracking/) for it first so that we can make sure we're on + the same page before significant works begins. +* Alternatively, you may already have written content that doesn't conform to + these guidelines, but you'd be willing to modify it so that it does. In this + case, you can still submit it by following the instructions below. Just make + a note in your pull request (PR) that you're aware of the changes that need + to be made and that you're just asking for the content to be reviewed before + you spend time making those changes. + +As mentioned above, we keep all the documentation in a dedicated [Git +repository](https://github.com/QubesOS/qubes-doc) hosted on +[GitHub](https://github.com/). Thanks to GitHub's interface, you can edit the +documentation even if you don't know Git at all! The only thing you need is a +GitHub account, which is free. + +(**Note:** If you're already familiar with GitHub or wish to work from the +command line, you can skip the rest of this section. All you need to do to +contribute is to [fork and +clone](https://guides.github.com/activities/forking/) the +[qubes-doc](https://github.com/QubesOS/qubes-doc) repo, make your changes, then +[submit a pull +request](https://help.github.com/articles/using-pull-requests/).) + +Ok, let's start. Every documentation page has an "Edit this page" button. It +may be on the side (in the desktop layout): + +[![edit-button-desktop](/attachment/doc/03-button2.png)](/attachment/doc/03-button2.png) -As mentioned above, we keep all the documentation in a dedicated [Git repository][qubes-doc] hosted on [GitHub]. -Thanks to GitHub's interface, you can edit the documentation even if you don't know Git at all! -The only thing you need is a GitHub account, which is free. +Or at the bottom (in the mobile layout): -(**Note:** If you're already familiar with GitHub or wish to work from the command line, you can skip the rest of this section. -All you need to do to contribute is to [fork and clone][gh-fork] the [qubes-doc] repo, make your changes, then [submit a pull request][gh-pull].) +[![edit-button-mobile](/attachment/doc/02-button1.png)](/attachment/doc/02-button1.png) -Ok, let's start. -Every documentation page has an "Edit this page" button. -It may be on the side (in the desktop layout): +When you click on it, you'll be prompted for your GitHub username and password +(if you aren't already logged in). You can also create an account from here. -![edit-button-desktop](/attachment/wiki/doc-edit/03-button2.png) +[![github-sign-in](/attachment/doc/04-sign-in.png)](/attachment/doc/04-sign-in.png) -Or at the bottom (in the mobile layout): +If this is your first contribution to the documentation, you need to "fork" the +repository (make your own copy). It's easy --- just click the big green button +on the next page. This step is only needed the first time you make a +contribution. -![edit-button-mobile](/attachment/wiki/doc-edit/02-button1.png) +[![fork](/attachment/doc/05-fork.png)](/attachment/doc/05-fork.png) -When you click on it, you'll be prompted for your GitHub username and password (if you aren't already logged in). -You can also create an account from here. +Now you can make your modifications. You can also preview the changes to see +how they'll be formatted by clicking the "Preview changes" tab. If you want to +add images, please see [How to add images](#how-to-add-images). If you're +making formatting changes, please [render the site +locally](https://github.com/QubesOS/qubesos.github.io#instructions) to verify +that everything looks correct before submitting any changes. -![github-sign-in](/attachment/wiki/doc-edit/04-sign-in.png) +[![edit](/attachment/doc/06-edit.png)](/attachment/doc/06-edit.png) -If this is your first contribution to the documentation, you need to "fork" the repository (make your own copy). It's easy --- just click the big green button on the next page. -This step is only needed the first time you make a contribution. +Once you're finished, describe your changes at the bottom and click "Propose +file change". -![fork](/attachment/wiki/doc-edit/05-fork.png) +[![commit](/attachment/doc/07-commit-msg.png)](/attachment/doc/07-commit-msg.png) -Now you can make your modifications. -You can also preview the changes to see how they'll be formatted by clicking the "Preview changes" tab. -If you're making formatting changes, please [render the site locally] to verify that everything looks correct before submitting any changes. +After that, you'll see exactly what modifications you've made. At this stage, +those changes are still in your own copy of the documentation ("fork"). If +everything looks good, send those changes to us by pressing the "Create pull +request" button. -![edit](/attachment/wiki/doc-edit/06-edit.png) +[![pull-request](/attachment/doc/08-review-changes.png)](/attachment/doc/08-review-changes.png) -Once you're finished, describe your changes at the bottom and click "Propose file change". +You will be able to adjust the pull request message and title there. In most +cases, the defaults are ok, so you can just confirm by pressing the "Create +pull request" button again. -![commit](/attachment/wiki/doc-edit/07-commit-msg.png) +[![pull-request-confirm](/attachment/doc/09-create-pull-request.png)](/attachment/doc/09-create-pull-request.png) -After that, you'll see exactly what modifications you've made. -At this stage, those changes are still in your own copy of the documentation ("fork"). -If everything looks good, send those changes to us by pressing the "Create pull request" button. +If any of your changes should be reflected in the [documentation index (a.k.a. +table of contents)](/doc/) --- for example, if you're adding a new page, +changing the title of an existing page, or removing a page --- please see [How +to edit the documentation index](#how-to-edit-the-documentation-index). -![pull-request](/attachment/wiki/doc-edit/08-review-changes.png) +That's all! We will review your changes. If everything looks good, we'll pull +them into the official documentation. Otherwise, we may have some questions for +you, which we'll post in a comment on your pull request. (GitHub will +automatically notify you if we do.) If, for some reason, we can't accept your +pull request, we'll post a comment explaining why we can't. -You will be able to adjust the pull request message and title there. -In most cases, the defaults are ok, so you can just confirm by pressing the "Create pull request" button again. +[![done](/attachment/doc/10-done.png)](/attachment/doc/10-done.png) -![pull-request-confirm](/attachment/wiki/doc-edit/09-create-pull-request.png) +## How to edit the documentation index -That's all! -We will review your changes. -If everything looks good, we'll pull them into the official documentation. -Otherwise, we may have some questions for you, which we'll post in a comment on your pull request. -(GitHub will automatically notify you if we do.) -If, for some reason, we can't accept your pull request, we'll post a comment explaining why we can't. +The source file for the [documentation index (a.k.a. table of contents)](/doc/) +lives here: -![done](/attachment/wiki/doc-edit/10-done.png) + +Editing this file will change what appears on the documentation index. If your +pull request (PR) adds, removes, or edits anything that should be reflected in +the documentation index, please make sure you also submit an associated pull +request against this file. -How to add images ------------------ +## How to add images -To add an image to a page, use the following syntax in the main document: +To add an image to a page, use the following syntax in the main document. This +will make the image a hyperlink to the image file, allowing the reader to click +on the image in order to view the image by itself. ``` -![Image Title](/attachment/wiki/page-title/image-filename.png) +[![Image Title](/attachment/doc/image.png)](/attachment/doc/image.png) ``` -Then, submit your image(s) in a separate pull request to the [qubes-attachment] repository using the same path and filename. - - -Version-specific Documentation ------------------------------- - -We maintain only one set of documentation for Qubes OS. -We do not maintain a different set of documentation for each version of Qubes. -Our single set of Qubes OS documentation is updated on a continual, rolling basis. -Our first priority is to document all **current, stable releases** of Qubes. -Our second priority is to document the next, upcoming release (if any) that is currently in the beta or release candidate stage. - -In cases where a documentation page covers functionality that differs considerably between Qubes OS versions, the page should be subdivided into clearly-labeled sections that cover the different functionality in different versions: - -### Incorrect Example ### +Then, submit your image(s) in a separate pull request to the +[qubes-attachment](https://github.com/QubesOS/qubes-attachment) repository +using the same path and filename. This is the only permitted way to include +images. Do not link to images on other websites. + +## Organizational guidelines + +### Do not duplicate documentation + +Duplicating documentation is almost always a bad idea. There are many reasons +for this. The main one is that almost all documentation has to be updated as +some point. When similar documentation appears in more than one place, it is +very easy for it to get updated in one place but not the others (perhaps +because the person updating it doesn't realize it's in more than once place). +When this happens, the documentation as a whole is now inconsistent, and the +outdated documentation becomes a trap, especially for novice users. Such traps +are often more harmful than if the documentation never existed in the first +place. The solution is to **link** to existing documentation rather than +duplicating it. There are some exceptions to this policy (e.g., information +that is certain not to change for a very long time), but they are rare. + +### Core vs. external documentation + +Core documentation resides in the [Qubes OS Project's official +repositories](https://github.com/QubesOS/), mainly in +[qubes-doc](https://github.com/QubesOS/qubes-doc). External documentation can +be anywhere else (such as forums, community websites, and blogs), but there is +an especially large collection in the [Qubes +Community](https://github.com/Qubes-Community) project. External documentation +should not be submitted to [qubes-doc](https://github.com/QubesOS/qubes-doc). +If you've written a piece of documentation that is not appropriate for +[qubes-doc](https://github.com/QubesOS/qubes-doc), we encourage you to submit +it to the [Qubes Community](https://github.com/Qubes-Community) project +instead. However, *linking* to external documentation from +[qubes-doc](https://github.com/QubesOS/qubes-doc) is perfectly fine. Indeed, +the maintainers of the [Qubes Community](https://github.com/Qubes-Community) +project should regularly submit PRs against the documentation index (see [How +to edit the documentation index](#how-to-edit-the-documentation-index)) to add +and update Qubes Community links in the "External Documentation" section of the +documentation table of contents. + +The main difference between **core** (or **official**) and **external** (or +**community** or **unofficial**) documentation is whether it documents software +that is officially written and maintained by the Qubes OS Project. The purpose +of this distinction is to keep the core docs maintainable and high-quality by +limiting them to the software output by the Qubes OS Project. In other words, +we take responsibility for documenting all of the software we put out into the +world, but it doesn't make sense for us to take on the responsibility of +documenting or maintaining documentation for anything else. For example, Qubes +OS may use a popular Linux distribution for an official +[TemplateVM](/doc/templates/). However, it would not make sense for a +comparatively small project like ours, with modest funding and a lean +workforce, to attempt to document software belonging to a large, richly-funded +project with an army of paid and volunteer contributors, especially when they +probably already have documentation of their own. This is particularly true +when it comes to Linux in general. Although many users who are new to Qubes are +also new to Linux, it makes absolutely no sense for our comparatively tiny +project to try to document Linux in general when there is already a plethora of +documentation out there. + +Many contributors do not realize that there is a significant amount of work +involved in *maintaining* documentation after it has been written. They may +wish to write documentation and submit it to the core docs, but they see only +their own writing process and fail to consider that it will have to be kept +up-to-date and consistent with the rest of the docs for years afterward. +Submissions to the core docs also have to go through a review process to ensure +accuracy before being merged (see [security](#security)), which takes up +valuable time from the team. We aim to maintain high quality standards for the +core docs (style and mechanics, formatting), which also takes up a lot of time. +If the documentation involves anything external to the Qubes OS Project (such +as a website, platform, program, protocol, framework, practice, or even a +reference to a version number), the documentation is likely to become outdated +when that external thing changes. It's also important to periodically review +and update this documentation, especially when a new Qubes release comes out. +Periodically, there may be technical or policy changes that affect all the core +documentation. The more documentation there is relative to maintainers, the +harder all of this will be. Since there are many more people who are willing to +write documentation than to maintain it, these individually small incremental +additions amount to a significant maintenance burden for the project. + +On the positive side, we consider the existence of community documentation to +be a sign of a healthy ecosystem, and this is quite common in the software +world. The community is better positioned to write and maintain documentation +that applies, combines, and simplifies the official documentation, e.g., +tutorials that explain how to install and use various programs in Qubes, how to +create custom VM setups, and introductory tutorials that teach basic Linux +concepts and commands in the context of Qubes. In addition, just because the +Qubes OS Project has officially written and maintains some flexible framework, +such as `qrexec`, it does not make sense to include every tutorial that says +"here's how to do something cool with `qrexec`" in the core docs. Such +tutorials generally also belong in the community documentation. + +See [#4693](https://github.com/QubesOS/qubes-issues/issues/4693) for more +background information. + +### Version-specific documentation + +*See [#5308](https://github.com/QubesOS/qubes-issues/issues/5308) for potential +changes to this policy.* + +We maintain only one set of documentation for Qubes OS. We do not maintain a +different set of documentation for each version of Qubes. Our single set of +Qubes OS documentation is updated on a continual, rolling basis. Our first +priority is to document all **current, stable releases** of Qubes. Our second +priority is to document the next, upcoming release (if any) that is currently +in the beta or release candidate stage. + +In cases where a documentation page covers functionality that differs +considerably between Qubes OS versions, the page should be subdivided into +clearly-labeled sections that cover the different functionality in different +versions: + +### Incorrect Example ``` # Page Title # @@ -164,7 +313,7 @@ command introduced in 4.0: Once you foo, make sure to close the baz before fooing the next bar. ``` -### Correct Example ### +### Correct Example ``` # Page Title # @@ -183,7 +332,6 @@ To foo: Once you foo, make sure to close the baz before fooing the next bar. - ## Qubes 4.0 ## ### How to Foo ### @@ -201,124 +349,220 @@ general `qubes-baz` command: Once you foo, make sure to close the baz before fooing the next bar. ``` -Subdividing the page into clearly-labeled sections for each version has several benefits: - - * It preserves good content for older (but still supported) versions. - Many documentation contributors are also people who prefer to use the latest version. - Many of them are tempted to *replace* existing content that applies to an older, supported version with content that applies only to the latest version. - This is somewhat understandable. - Since they only use the latest version, they may be focused on their own experience, and they may even regard the older version as deprecated, even when it's actually still supported. - However, allowing this replacement of content would do a great disservice to those who still rely on the older, supported version. - In many cases, these users value the stability and reliability of the older, supported version. - With the older, supported version, there has been more time to fix bugs and make improvements in both the software and the documentation. - Consequently, much of the documentation content for this version may have gone through several rounds of editing, review, and revision. - It would be a tragedy for this content to vanish while the very set of users who most prize stability and reliability are depending on it. - * It's easy for readers to quickly find the information they're looking for, since they can go directly to the section that applies to their version. - * It's hard for readers to miss information they need, since it's all in one place. - In the incorrect example, information that the reader needs could be in any paragraph in the entire document, and there's no way to tell without reading the entire page. - In the correct example, the reader can simply skim the headings in order to know which parts of the page need to be read and which can be safely ignored. - The fact that some content is repeated in the two version-specific sections is not a problem, since no reader has to read the same thing twice. - Moreover, as one version gets updated, it's likely that the documentation for that version will also be updated. - Therefore, content that is initially duplicated between version-specific sections will not necessarily stay that way, and this is a good thing: - We want the documentation for a version that *doesn't* change to stay the same, and we want the documentation for a version that *does* change to change along with the software. - * It's easy for documentation contributors and maintainers to know which file to edit and update, since there's only one page for all Qubes OS versions. - Initially creating the new headings and duplicating content that applies to both is only a one-time cost for each page, and many pages don't even require this treatment, since they apply to all currently-supported Qubes OS versions. - -By contrast, an alternative approach, such as segregating the documentation into two different branches, would mean that contributions that apply to both Qubes versions would only end up in one branch, unless someone remembered to manually submit the same thing to the other branch and actually made the effort to do so. -Most of the time, this wouldn't happen. -When it did, it would mean a second pull request that would have to be reviewed. -Over time, the different branches would diverge in non-version-specific content. -Good general content that was submitted only to one branch would effectively disappear once that version was deprecated. -(Even if it were still on the website, no one would look at it, since it would explicitly be in the subdirectory of a deprecated version, and there would be a motivation to remove it from the website so that search results wouldn't be populated with out-of-date information.) - -For further discussion about version-specific documentation in Qubes, see [here][version-thread]. - - -Style Guidelines ----------------- - - * Familiarize yourself with the terms defined in the [glossary]. Use these - terms consistently and accurately throughout your writing. - - -Markdown Conventions --------------------- - -All the documentation is written in Markdown for maximum accessibility. -When making contributions, please try to observe the following style conventions: - - * Use spaces instead of tabs. - * In order to enable offline browsing, use relative paths (e.g., `/doc/doc-guidelines/` instead of `https://www.qubes-os.org/doc/doc-guidelines/`, except when the source text will be reproduced outside of the Qubes website repo. - Examples of exceptions: - * [QSBs] (intended to be read as plain text) - * [News] posts (plain text is reproduced on the [mailing lists][support]) - * URLs that appear inside code blocks (e.g., in comments and document templates) - * Files like `README.md` and `CONTRIBUTING.md` - * Insert a newline at, and only at, the end of each sentence, except when the text will be reproduced outside of the Qubes website repo (see previous item for examples). - * Rationale: This practice results in one sentence per line, which is most appropriate for source that consists primarily of natural language text. - It results in the most useful diffs and facilitates translation into other languages while mostly preserving source readability. - * If appropriate, make numerals in numbered lists match between Markdown source and HTML output. - * Rationale: In the event that a user is required to read the Markdown source directly, this will make it easier to follow, e.g., numbered steps in a set of instructions. - * Use hanging indentations - where appropriate. - * Use Atx-style headings: `# h1`, `##h 2`, `### h3`, etc. - * When writing code blocks, use [syntax highlighting](https://github.github.com/gfm/#info-string) where [possible](https://github.com/jneen/rouge/wiki/List-of-supported-languages-and-lexers) and use `[...]` for anything omitted. - * When providing command line examples: - * Tell the reader where to open a terminal (dom0 or a specific domU), and show the command along with its output (if any) in a code block, e.g.: - ~~~markdown - Open a terminal in dom0 and run: - ```shell_session - $ cd test - $ echo Hello - Hello - ``` - ~~~ - * Precede each command with the appropriate command prompt: - At a minimum, the prompt should contain a trailing `#` (for the user `root`) or `$` (for other users) on Linux systems and `>` on Windows systems, respectively. - * Don't try to add comments inside the code block. - For example, *don't* do this: - ~~~markdown - Open a terminal in dom0 and run: - ```shell_session - # Navigate to the new directory - $ cd test - # Generate a greeting - $ echo Hello - Hello - ``` - ~~~ - The `#` symbol preceding each comment is ambiguous with a root command prompt. - Instead, put your comments *outside* of the code block in normal prose. - * Use `[reference-style][ref]` links. - -`[ref]: https://daringfireball.net/projects/markdown/syntax#link` - -([This][md] is a great source for learning about Markdown.) - - -Git Conventions ---------------- - -Please try to write good commit messages, according to the -[instructions in our coding style guidelines][git-commit]. - - -[qubes-doc]: https://github.com/QubesOS/qubes-doc -[glossary]: /doc/glossary/ -[issue]: /doc/reporting-bugs/ -[contribute]: #how-to-contribute -[qubes-issues]: https://github.com/QubesOS/qubes-issues/issues -[gh-fork]: https://guides.github.com/activities/forking/ -[gh-pull]: https://help.github.com/articles/using-pull-requests/ -[GitHub]: https://github.com/ -[support]: /support/ -[version-example]: /doc/template/fedora/upgrade-25-to-26/ -[version-thread]: https://groups.google.com/d/topic/qubes-users/H9BZX4K9Ptk/discussion -[QSBs]: /security/bulletins/ -[News]: /news/ -[md]: https://daringfireball.net/projects/markdown/ -[git-commit]: /doc/coding-style/#commit-message-guidelines -[render the site locally]: https://github.com/QubesOS/qubesos.github.io#instructions -[qubes-attachment]: https://github.com/QubesOS/qubes-attachment -[external documentation]: /doc/#external-documentation -[#4693]: https://github.com/QubesOS/qubes-issues/issues/4693 +Subdividing the page into clearly-labeled sections for each version has several +benefits: + +* It preserves good content for older (but still supported) versions. Many + documentation contributors are also people who prefer to use the latest + version. Many of them are tempted to *replace* existing content that applies + to an older, supported version with content that applies only to the latest + version. This is somewhat understandable. Since they only use the latest + version, they may be focused on their own experience, and they may even + regard the older version as deprecated, even when it's actually still + supported. However, allowing this replacement of content would do a great + disservice to those who still rely on the older, supported version. In many + cases, these users value the stability and reliability of the older, + supported version. With the older, supported version, there has been more + time to fix bugs and make improvements in both the software and the + documentation. Consequently, much of the documentation content for this + version may have gone through several rounds of editing, review, and + revision. It would be a tragedy for this content to vanish while the very set + of users who most prize stability and reliability are depending on it. +* It's easy for readers to quickly find the information they're looking for, + since they can go directly to the section that applies to their version. +* It's hard for readers to miss information they need, since it's all in one + place. In the incorrect example, information that the reader needs could be + in any paragraph in the entire document, and there's no way to tell without + reading the entire page. In the correct example, the reader can simply skim + the headings in order to know which parts of the page need to be read and + which can be safely ignored. The fact that some content is repeated in the + two version-specific sections is not a problem, since no reader has to read + the same thing twice. Moreover, as one version gets updated, it's likely that + the documentation for that version will also be updated. Therefore, content + that is initially duplicated between version-specific sections will not + necessarily stay that way, and this is a good thing: We want the + documentation for a version that *doesn't* change to stay the same, and we + want the documentation for a version that *does* change to change along with + the software. +* It's easy for documentation contributors and maintainers to know which file + to edit and update, since there's only one page for all Qubes OS versions. + Initially creating the new headings and duplicating content that applies to + both is only a one-time cost for each page, and many pages don't even require + this treatment, since they apply to all currently-supported Qubes OS + versions. + +By contrast, an alternative approach, such as segregating the documentation +into two different branches, would mean that contributions that apply to both +Qubes versions would only end up in one branch, unless someone remembered to +manually submit the same thing to the other branch and actually made the effort +to do so. Most of the time, this wouldn't happen. When it did, it would mean a +second pull request that would have to be reviewed. Over time, the different +branches would diverge in non-version-specific content. Good general content +that was submitted only to one branch would effectively disappear once that +version was deprecated. (Even if it were still on the website, no one would +look at it, since it would explicitly be in the subdirectory of a deprecated +version, and there would be a motivation to remove it from the website so that +search results wouldn't be populated with out-of-date information.) + +For further discussion about version-specific documentation in Qubes, see +[here](https://groups.google.com/d/topic/qubes-users/H9BZX4K9Ptk/discussion). + +## Style guidelines + +* Familiarize yourself with the terms defined in the + [glossary](/doc/glossary/). Use these terms consistently and accurately + throughout your writing. +* Syntactically distinguish variables in commands. + + For example, this is ambiguous: + + ``` + $ qvm-run --dispvm=dvm-template --service qubes.StartApp+xterm + ``` + + It should instead be: + + ``` + $ qvm-run --dispvm= --service qubes.StartApp+xterm + ``` + + Note that we syntactically distinguish variables in three ways: + 1. Surrounding them in angled brackets (`< >`) + 2. Using underscores (`_`) between words + 3. Using all capital letters + +## Markdown conventions + +All the documentation is written in Markdown for maximum accessibility. When +making contributions, please try to observe the following style conventions: + +* Use spaces instead of tabs. +* Do not write HTML inside Markdown documents (except in rare, unavoidable + cases, such as alerts). In particular, never include HTML or CSS for styling, + formatting, or white space control. That belongs in the (S)CSS files instead. +* Link only to images in + [qubes-attachment](https://github.com/QubesOS/qubes-attachment) (see + [instructions above](#how-to-add-images)). Do not link to images on other + websites. +* In order to enable offline browsing and automatic onion redirection, always + use relative (rather than absolute) links, e.g., `/doc/doc-guidelines/` + instead of `https://www.qubes-os.org/doc/doc-guidelines/`. Examples of + exceptions: + * The signed plain text portions of [QSBs](/security/qsb/) and + [Canaries](/security/canary/) + * URLs that appear inside code blocks (e.g., in comments and document + templates) + * Files like `README.md` and `CONTRIBUTING.md` +* Hard wrap Markdown lines at 80 characters, unless the line can't be broken + (e.g., code or a URL). +* If appropriate, make numerals in numbered lists match between Markdown source + and HTML output. + * Rationale: In the event that a user is required to read the Markdown source + directly, this will make it easier to follow, e.g., numbered steps in a set + of instructions. +* Use hanging indentations where appropriate. +* Do not use `h1` headings (single `#` or `======` underline). These are + automatically generated from the `title:` line in the YAML frontmatter. +* Use Atx-style headings: , `##h 2`, `### h3`, etc. +* When writing code blocks, use [syntax + highlighting](https://github.github.com/gfm/#info-string) where + [possible](https://github.com/jneen/rouge/wiki/List-of-supported-languages-and-lexers) + and use `[...]` for anything omitted. +* When providing command line examples: + * Tell the reader where to open a terminal (dom0 or a specific domU), and + show the command along with its output (if any) in a code block, e.g.: + + ~~~markdown + Open a terminal in dom0 and run: + ```shell_session + $ cd test + $ echo Hello + Hello + ``` + ~~~ + + * Precede each command with the appropriate command prompt: At a minimum, the + prompt should contain a trailing `#` (for the user `root`) or `$` (for + other users) on Linux systems and `>` on Windows systems, respectively. + * Don't try to add comments inside the code block. For example, *don't* do + this: + + ~~~markdown + Open a terminal in dom0 and run: + ```shell_session + # Navigate to the new directory + $ cd test + # Generate a greeting + $ echo Hello + Hello + ``` + ~~~ + + The `#` symbol preceding each comment is ambiguous with a root command + prompt. Instead, put your comments *outside* of the code block in normal + prose. +* Use non-reference-style links like `[website](https://example.com/)`. Do + *not* use reference links like `[website][example]`, `[website][]` or + `[website]`. + +([This](https://daringfireball.net/projects/markdown/) is a great source for +learning about Markdown.) + +## Coding conventions + +These conventions apply to the website as a whole, including everything written +in HTML, CSS, YAML, and Liquid. These conventions are intended to keep the +codebase consistent when multiple collaborators are working on it. They should +be understood as a practical set of rules for maintaining order in this +specific codebase rather than as a statement of what is objectively right or +good. + +* Always use spaces. Never use tabs. +* Indent by exactly two (2) spaces. +* Whenever you add an opening tag, indent the following line. (Exception: If + you open and close the tag on the same line, do not indent the following + line.) +* Indent Liquid the same way as HTML. +* In general, the starting columns of every adjacent pair of lines should be no + more than two spaces apart (example below). +* No blank or empty lines. (Hint: When you feel you need one, add a comment on + that line instead.) +* Use comments to indicate the purposes of different blocks of code. This makes + the file easier to understand and navigate. +* Use descriptive variable names. Never use one or two letter variable names. + Avoid uncommon abbreviations and made-up words. +* In general, make it easy for others to read your code. Your future self will + thank you, and so will your collaborators! +* [Don't Repeat Yourself + (DRY)!](https://en.wikipedia.org/wiki/Don%27t_repeat_yourself) Instead of + repeating the same block of code multiple times, abstract it out into a + separate file and `include` that file where you need it. + +### Indentation example + +Here's an example that follows the indentation rules: + +{% raw %} +```html + + + + {% for item in secs.htmlsections[0].columns %} + + {% endfor %} + + {% for canary in site.data.sec-canary reversed %} + + + + + + {% endfor %} +
{{ item.title }}
{{ canary.date }}Qubes Canary #{{ canary.canary }}
+``` +{% endraw %} + +## Git conventions + +Please try to write good commit messages, according to the [instructions in our +coding style guidelines](/doc/coding-style/#commit-message-guidelines). diff --git a/developer/general/documentation-style-guide.md b/developer/general/documentation-style-guide.md new file mode 100644 index 0000000000..565587e91a --- /dev/null +++ b/developer/general/documentation-style-guide.md @@ -0,0 +1,445 @@ +--- +lang: en +layout: doc +permalink: /doc/documentation-style-guide/ +redirect_from: +- /doc/doc-guidelines/ +- /en/doc/doc-guidelines/ +- /wiki/DocStyle/ +- /doc/DocStyle/ +ref: 30 +title: Documentation style guide +--- + +_Also see [how to edit the documentation](/doc/how-to-edit-the-documentation/)._ + +Qubes OS documentation pages are stored as plain text Markdown files in the +[qubes-doc](https://github.com/QubesOS/qubes-doc) repository. By cloning and +regularly pulling from this repo, users can maintain their own up-to-date +offline copy of all Qubes documentation rather than relying solely on the web. + +The documentation is a volunteer community effort. People like you are +constantly working to make it better. If you notice something that can be fixed +or improved, please [edit the +documentation](/doc/how-to-edit-the-documentation/)! + +This page explains the standards we follow for writing, formatting, and +organizing the documentation. Please follow these guidelines and conventions +when editing the documentation. For the standards governing the website as a +whole, please see the [website style guide](/doc/website-style-guide). + +## Markdown conventions + +All the documentation is written in Markdown for maximum accessibility. When +making contributions, please observe the following style conventions. If you're +not familiar with Markdown syntax, +[this](https://daringfireball.net/projects/markdown/) is a great resource. + +### Hyperlink syntax + +Use non-reference-style links like `[website](https://example.com/)`. Do *not* +use reference-style links like `[website][example]`, `[website][]` or +`[website]`. This facilitates the localization process. + +### Relative vs. absolute links + +Always use relative rather than absolute paths for internal website links. For +example, use `/doc/documentation-style-guide/` instead of +`https://www.qubes-os.org/doc/documentation-style-guide/`. + +You may use absolute URLs in the following cases: + +- External links +- URLs that appear inside code blocks (e.g., in comments and document + templates, and the plain text reproductions of [QSBs](/security/qsb/) and + [Canaries](/security/canary/)), since they're not hyperlinks +- Git repo files like `README.md` and `CONTRIBUTING.md`, since they're not part + of the website itself but rather of the auxiliary infrastructure supporting + the website + +This rule is important because using absolute URLs for internal website links +breaks: + +- Serving the website offline +- Website localization +- Generating offline documentation +- Automatically redirecting Tor Browser visitors to the correct page on the + onion service mirror + +### Image linking + +See [how to add images](/doc/how-to-edit-the-documentation/#how-to-add-images) +for the required syntax. This will make the image a hyperlink to the image +file, allowing the reader to click on the image in order to view the full image +by itself. This is important. Following best practices, our website has a +responsive design, which allows the website to render appropriately across all +screen sizes. When viewing this page on a smaller screen, such as on a mobile +device, the image will automatically shrink down to fit the screen. If visitors +cannot click on the image to view it in full size, then, depending on their +device, they may have no way see the details in the image clearly. + +In addition, make sure to link only to images in the +[qubes-attachment](https://github.com/QubesOS/qubes-attachment) repository. Do +not attempt to link to images hosted on other websites. + +### HTML and CSS + +Do not write HTML inside Markdown documents (except in rare, unavoidable cases, +such as alerts). In particular, never include HTML or CSS for styling, +formatting, or white space control. That belongs in the (S)CSS files instead. + +### Headings + +Do not use `h1` headings (single `#` or `======` underline). These are +automatically generated from the `title:` line in the YAML front matter. + +Use Atx-style syntax for headings: `##h2`, `### h3`, etc. Do not use +underlining syntax (`-----`). + +### Indentation + +Use spaces instead of tabs. Use hanging indentations where appropriate. + +### Lists + +If appropriate, make numerals in numbered lists match between Markdown source +and HTML output. Some users read the Markdown source directly, and this makes +numbered lists easier to follow. + +### Code blocks + +When writing code blocks, use [syntax +highlighting](https://github.github.com/gfm/#info-string) where possible (see +[here](https://github.com/jneen/rouge/wiki/List-of-supported-languages-and-lexers) +for a list of supported languages). Use `[...]` for anything omitted. + +### Line wrapping + +Hard wrap Markdown lines at 80 characters, unless the line can't be broken +(e.g., code or a URL). + +## Writing guidelines + +### Correct use of terminology + +Familiarize yourself with the terms defined in the [glossary](/doc/glossary/). +Use these terms consistently and accurately throughout your writing. + +### Sentence case in headings + +Use sentence case (rather than title case) in headings for the reasons +explained +[here](https://www.sallybagshaw.com.au/articles/sentence-case-v-title-case/). +In particular, since the authorship of the Qubes documentation is decentralized +and widely distributed among users from around the world, many contributors +come from regions with different conventions for implementing title case, not +to mention that there are often differing style guide recommendations even +within a single region. It is much easier for all of us to implement sentence +case consistently across our growing body of pages, which is very important for +managing the ongoing maintenance burden and sustainability of the +documentation. + +### Writing command-line examples + +When providing command-line examples: + +- Tell the reader where to open a terminal (dom0 or a specific domU), and show + the command along with its output (if any) in a code block, e.g.: + + ~~~markdown + Open a terminal in dom0 and run: + ```shell_session + $ cd test + $ echo Hello + Hello + ``` + ~~~ + +- Precede each command with the appropriate command prompt: At a minimum, the + prompt should contain a trailing `#` (for the user `root`) or `$` (for other + users) on Linux systems and `>` on Windows systems, respectively. + +- Don't try to add comments inside the code block. For example, *don't* do + this: + + ~~~markdown + Open a terminal in dom0 and run: + ```shell_session + # Navigate to the new directory + $ cd test + # Generate a greeting + $ echo Hello + Hello + ``` + ~~~ + + The `#` symbol preceding each comment is ambiguous with a root command + prompt. Instead, put your comments *outside* of the code block in normal + prose. + +### Variable names in commands + +Syntactically distinguish variables in commands. For example, this is +ambiguous: + + $ qvm-run --dispvm=disposable-template --service qubes.StartApp+xterm + +It should instead be: + + $ qvm-run --dispvm= --service qubes.StartApp+xterm + +Note that we syntactically distinguish variables in three ways: + +1. Surrounding them in angled brackets (`< >`) +2. Using underscores (`_`) instead of spaces between words +3. Using all capital letters + +We have observed that many novices make the mistake of typing the surrounding +angled brackets (`< >`) on the command line, even after substituting the +desired real value between them. Therefore, in documentation aimed at novices, +we also recommend clarifying that the angled brackets should not be typed. This +can be accomplished in one of several ways: + +- Explicitly say something like "without the angled brackets." +- Provide an example command using real values that excludes the angled + brackets. +- If you know that almost all users will want to use (or should use) a specific + command containing all real values and no variables, you might consider + providing exactly that command and forgoing the version with variables. + Novices may not realize which parts of the command they can substitute with + different values, but if you've correctly judged that they should use the + command you've provided as is, then this shouldn't matter. + +## Organizational guidelines + +### Do not duplicate documentation + +Duplicating documentation is almost always a bad idea. There are many reasons +for this. The main one is that almost all documentation has to be updated as +some point. When similar documentation appears in more than one place, it is +very easy for it to get updated in one place but not the others (perhaps +because the person updating it doesn't realize it's in more than once place). +When this happens, the documentation as a whole is now inconsistent, and the +outdated documentation becomes a trap, especially for novice users. Such traps +are often more harmful than if the documentation never existed in the first +place. The solution is to **link** to existing documentation rather than +duplicating it. There are some exceptions to this policy (e.g., information +that is certain not to change for a very long time), but they are rare. + +### Core vs. external documentation + +Core documentation resides in the [Qubes OS Project's official +repositories](https://github.com/QubesOS/), mainly in +[qubes-doc](https://github.com/QubesOS/qubes-doc). External documentation can +be anywhere else (such as forums, community websites, and blogs), but there is +an especially large collection in the [Qubes +Community](https://github.com/Qubes-Community) project. External documentation +should not be submitted to [qubes-doc](https://github.com/QubesOS/qubes-doc). +If you've written a piece of documentation that is not appropriate for +[qubes-doc](https://github.com/QubesOS/qubes-doc), we encourage you to submit +it to the [Qubes Community](https://github.com/Qubes-Community) project +instead. However, *linking* to external documentation from +[qubes-doc](https://github.com/QubesOS/qubes-doc) is perfectly fine. Indeed, +the maintainers of the [Qubes Community](https://github.com/Qubes-Community) +project should regularly submit PRs against the documentation index (see [How +to edit the documentation +index](/doc/how-to-edit-the-documentation/#how-to-edit-the-documentation-index)) +to add and update Qubes Community links in the ["External +documentation"](/doc/#external-documentation) section of the documentation +table of contents. + +The main difference between **core** (or **official**) and **external** (or +**community** or **unofficial**) documentation is whether it documents software +that is officially written and maintained by the Qubes OS Project. The purpose +of this distinction is to keep the core docs maintainable and high-quality by +limiting them to the software output by the Qubes OS Project. In other words, +we take responsibility for documenting all of the software we put out into the +world, but it doesn't make sense for us to take on the responsibility of +documenting or maintaining documentation for anything else. For example, Qubes +OS may use a popular Linux distribution for an official +[TemplateVM](/doc/templates/). However, it would not make sense for a +comparatively small project like ours, with modest funding and a lean +workforce, to attempt to document software belonging to a large, richly-funded +project with an army of paid and volunteer contributors, especially when they +probably already have documentation of their own. This is particularly true +when it comes to Linux in general. Although many users who are new to Qubes are +also new to Linux, it makes absolutely no sense for our comparatively tiny +project to try to document Linux in general when there is already a plethora of +documentation out there. + +Many contributors do not realize that there is a significant amount of work +involved in *maintaining* documentation after it has been written. They may +wish to write documentation and submit it to the core docs, but they see only +their own writing process and fail to consider that it will have to be kept +up-to-date and consistent with the rest of the docs for years afterward. +Submissions to the core docs also have to [undergo a review +process](/doc/how-to-edit-the-documentation#security) to ensure accuracy before +being merged, which takes up valuable time from the team. We aim to maintain +high quality standards for the core docs (style and mechanics, formatting), +which also takes up a lot of time. If the documentation involves anything +external to the Qubes OS Project (such as a website, platform, program, +protocol, framework, practice, or even a reference to a version number), the +documentation is likely to become outdated when that external thing changes. +It's also important to periodically review and update this documentation, +especially when a new Qubes release comes out. Periodically, there may be +technical or policy changes that affect all the core documentation. The more +documentation there is relative to maintainers, the harder all of this will be. +Since there are many more people who are willing to write documentation than to +maintain it, these individually small incremental additions amount to a +significant maintenance burden for the project. + +On the positive side, we consider the existence of community documentation to +be a sign of a healthy ecosystem, and this is quite common in the software +world. The community is better positioned to write and maintain documentation +that applies, combines, and simplifies the official documentation, e.g., +tutorials that explain how to install and use various programs in Qubes, how to +create custom VM setups, and introductory tutorials that teach basic Linux +concepts and commands in the context of Qubes. In addition, just because the +Qubes OS Project has officially written and maintains some flexible framework, +such as `qrexec`, it does not make sense to include every tutorial that says +"here's how to do something cool with `qrexec`" in the core docs. Such +tutorials generally also belong in the community documentation. + +See [#4693](https://github.com/QubesOS/qubes-issues/issues/4693) for more +background information. + +### Release-specific documentation + +*See [#5308](https://github.com/QubesOS/qubes-issues/issues/5308) for pending +changes to this policy.* + +We maintain only one set of documentation for Qubes OS. We do not maintain a +different set of documentation for each release of Qubes. Our single set of +Qubes OS documentation is updated on a continual, rolling basis. Our first +priority is to document all **current, stable releases** of Qubes. Our second +priority is to document the next, upcoming release (if any) that is currently +in the beta or release candidate stage. + +In cases where a documentation page covers functionality that differs +considerably between Qubes OS releases, the page should be subdivided into +clearly-labeled sections that cover the different functionality in different +releases (examples below). + +In general, avoid mentioning specific Qubes versions in the body text of +documentation, as these references rapidly go out of date and become misleading +to readers. + +#### Incorrect Example + +``` +## How to Foo + +Fooing is the process by which one foos. There are both general and specific +versions of fooing, which vary in usefulness depending on your goals, but for +the most part, all fooing is fooing. + +To foo in Qubes 3.2: + + $ qvm-foo + +Note that this does not work in Qubes 4.0, where there is a special widget +for fooing, which you can find in the lower-right corner of the screen in +the Foo Manager. Alternatively, you can use the more general `qubes-baz` +command introduced in 4.0: + + $ qubes-baz --foo + +Once you foo, make sure to close the baz before fooing the next bar. +``` + +#### Correct Example + +``` +## Qubes 3.2 + +### How to Foo + +Fooing is the process by which one foos. There are both general and specific +versions of fooing, which vary in usefulness depending on your goals, but for +the most part, all fooing is fooing. + +To foo: + + $ qvm-foo + +Once you foo, make sure to close the baz before fooing the next bar. + +## Qubes 4.0 + +### How to Foo + +Fooing is the process by which one foos. There are both general and specific +versions of fooing, which vary in usefulness depending on your goals, but for +the most part, all fooing is fooing. + +There is a special widget for fooing, which you can find in the lower-right +corner of the screen in the Foo Manager. Alternatively, you can use the +general `qubes-baz` command: + + $ qubes-baz --foo + +Once you foo, make sure to close the baz before fooing the next bar. +``` + +Subdividing the page into clearly-labeled sections for each release has several +benefits: + +- It preserves good content for older (but still supported) releases. Many + documentation contributors are also people who prefer to use the latest + release. Many of them are tempted to *replace* existing content that applies + to an older, supported release with content that applies only to the latest + release. This is somewhat understandable. Since they only use the latest + release, they may be focused on their own experience, and they may even + regard the older release as deprecated, even when it's actually still + supported. However, allowing this replacement of content would do a great + disservice to those who still rely on the older, supported release. In many + cases, these users value the stability and reliability of the older, + supported release. With the older, supported release, there has been more + time to fix bugs and make improvements in both the software and the + documentation. Consequently, much of the documentation content for this + release may have gone through several rounds of editing, review, and + revision. It would be a tragedy for this content to vanish while the very set + of users who most prize stability and reliability are depending on it. +- It's easy for readers to quickly find the information they're looking for, + since they can go directly to the section that applies to their release. +- It's hard for readers to miss information they need, since it's all in one + place. In the incorrect example, information that the reader needs could be + in any paragraph in the entire document, and there's no way to tell without + reading the entire page. In the correct example, the reader can simply skim + the headings in order to know which parts of the page need to be read and + which can be safely ignored. The fact that some content is repeated in the + two release-specific sections is not a problem, since no reader has to read + the same thing twice. Moreover, as one release gets updated, it's likely that + the documentation for that release will also be updated. Therefore, content + that is initially duplicated between release-specific sections will not + necessarily stay that way, and this is a good thing: We want the + documentation for a release that *doesn't* change to stay the same, and we + want the documentation for a release that *does* change to change along with + the software. +- It's easy for documentation contributors and maintainers to know which file + to edit and update, since there's only one page for all Qubes OS releases. + Initially creating the new headings and duplicating content that applies to + both is only a one-time cost for each page, and many pages don't even require + this treatment, since they apply to all currently-supported Qubes OS + releases. + +By contrast, an alternative approach, such as segregating the documentation +into two different branches, would mean that contributions that apply to both +Qubes releases would only end up in one branch, unless someone remembered to +manually submit the same thing to the other branch and actually made the effort +to do so. Most of the time, this wouldn't happen. When it did, it would mean a +second pull request that would have to be reviewed. Over time, the different +branches would diverge in non-release-specific content. Good general content +that was submitted only to one branch would effectively disappear once that +release was deprecated. (Even if it were still on the website, no one would +look at it, since it would explicitly be in the subdirectory of a deprecated +release, and there would be a motivation to remove it from the website so that +search results wouldn't be populated with out-of-date information.) + +For further discussion about release-specific documentation in Qubes, see +[here](https://groups.google.com/d/topic/qubes-users/H9BZX4K9Ptk/discussion). + +## Git conventions + +Please follow our [Git commit message +guidelines](/doc/coding-style/#commit-message-guidelines). diff --git a/developer/general/gsoc.md b/developer/general/gsoc.md index 3241fcfb29..40df0c9314 100644 --- a/developer/general/gsoc.md +++ b/developer/general/gsoc.md @@ -1,38 +1,37 @@ --- -layout: sidebar -title: Google Summer of Code +lang: en +layout: doc permalink: /gsoc/ -redirect_from: /GSoC/ +redirect_from: +- /GSoC/ +ref: 33 +title: Google Summer of Code (GSoC) --- -2020 Google Summer of Code -================ ## Information for Students -Thank you for your interest in participating in the [Google Summer of Code program][gsoc-qubes] with the [Qubes OS team][team]. You can read more about the Google Summer of Code program at the [official website][gsoc] and the [official FAQ][gsoc-faq]. +Thank you for your interest in participating in the [Google Summer of Code program](https://summerofcode.withgoogle.com/organizations/4675790572093440/) with the [Qubes OS team](/team/). You can read more about the Google Summer of Code program at the [official website](https://summerofcode.withgoogle.com/) and the [official FAQ](https://developers.google.com/open-source/gsoc/faq). Being accepted as a Google Summer of Code student is quite competitive. Students wishing to participate in the Summer of Code must be aware that you will be required to produce code for Qubes OS for 3 months. Your mentors, Qubes developers, will dedicate a portion of their time towards mentoring you. Therefore, we seek candidates who are committed to helping Qubes long-term and are willing to do quality work and be proactive in communicating with your mentor. -You don't have to be a proven developer -- in fact, this whole program is meant to facilitate joining Qubes and other free and open source communities. The Qubes community maintains information about [contributing to Qubes development][contributing] and [how to send patches][patches]. In order to contribute code to the Qubes project, you must be able to [sign your code][code-signing]. +You don't have to be a proven developer -- in fact, this whole program is meant to facilitate joining Qubes and other free and open source communities. The Qubes community maintains information about [contributing to Qubes development](/doc/contributing/#contributing-code) and [how to send patches](/doc/source-code/#how-to-send-patches). In order to contribute code to the Qubes project, you must be able to [sign your code](/doc/code-signing/). -You should start learning the components that you plan on working on before the start date. Qubes developers are available on the [mailing lists][ml-devel] for help. The GSoC timeline reserves a lot of time for bonding with the project -- use that time wisely. Good communication is key, you should plan to communicate with your team daily and formally report progress and plans weekly. Students who neglect active communication will be failed. - -You can view the projects we had in 2017 in the [GSoC archive here][2017-archive]. +You should start learning the components that you plan on working on before the start date. Qubes developers are available on the [mailing lists](/support/#qubes-devel) for help. The GSoC timeline reserves a lot of time for bonding with the project -- use that time wisely. Good communication is key, you should plan to communicate with your team daily and formally report progress and plans weekly. Students who neglect active communication will be failed. ### Overview of Steps -- Join the [qubes-devel list][ml-devel] and introduce yourself, and meet your fellow developers -- Read [Google's instructions for participating][gsoc-participate] and the [GSoC Student Manual][gsoc-student] +- Join the [qubes-devel list](/support/#qubes-devel) and introduce yourself, and meet your fellow developers +- Read [Google's instructions for participating](https://developers.google.com/open-source/gsoc/) and the [GSoC Student Manual](https://developers.google.com/open-source/gsoc/resources/manual#student_manual) - Take a look at the list of ideas below - Come up with a project that you are interested in (and feel free to propose your own! Don't feel limited by the list below.) - Read the Student Proposal guidelines below - Write a first draft proposal and send it to the qubes-devel mailing list for review -- Submit proposal using [Google's web interface][gsoc-submit] ahead of the deadline (this requires a Google Account!) +- Submit proposal using [Google's web interface](https://summerofcode.withgoogle.com/) ahead of the deadline (this requires a Google Account!) - Submit proof of enrollment well ahead of the deadline Coming up with an interesting idea that you can realistically achieve in the time available to you (one summer) is probably the most difficult part. We strongly recommend getting involved in advance of the beginning of GSoC, and we will look favorably on applications from students who have already started to act like free and open source developers. -Before the summer starts, there are some preparatory tasks which are highly encouraged. First, if you aren't already, definitely start using Qubes as your primary OS as soon as possible! Also, it is encouraged that you become familiar and comfortable with the Qubes development workflow sooner than later. A good way to do this (and also a great way to stand out as an awesome applicant and make us want to accept you!) might be to pick up some issues from [qubes-issues][qubes-issues] (our issue-tracking repo) and submit some patches addressing them. Some suitable issues might be those with tags ["help wanted" and "P: minor"][qubes-issues-suggested] (although more significant things are also welcome, of course). Doing this will get you some practice with [qubes-builder][qubes-builder], our code-signing policies, and some familiarity with our code base in general so you are ready to hit the ground running come summer. +Before the summer starts, there are some preparatory tasks which are highly encouraged. First, if you aren't already, definitely start using Qubes as your primary OS as soon as possible! Also, it is encouraged that you become familiar and comfortable with the Qubes development workflow sooner than later. A good way to do this (and also a great way to stand out as an awesome applicant and make us want to accept you!) might be to pick up some issues from [qubes-issues](https://github.com/QubesOS/qubes-issues/issues) (our issue-tracking repo) and submit some patches addressing them. Some suitable issues might be those with tags ["help wanted" and "P: minor"](https://github.com/QubesOS/qubes-issues/issues?q=is%3Aissue%20is%3Aopen%20label%3A%22P%3A%20minor%22%20label%3A%22help%20wanted%22) (although more significant things are also welcome, of course). Doing this will get you some practice with [qubes-builder](/doc/qubes-builder/), our code-signing policies, and some familiarity with our code base in general so you are ready to hit the ground running come summer. ### Student proposal guidelines @@ -75,7 +74,7 @@ Other things to think about: ## Project Ideas -These project ideas were contributed by our developers and may be incomplete. If you are interested in submitting a proposal based on these ideas, you should contact the [qubes-devel mailing list][ml-devel] and associated GitHub issue to learn more about the idea. +These project ideas were contributed by our developers and may be incomplete. If you are interested in submitting a proposal based on these ideas, you should contact the [qubes-devel mailing list](/support/#qubes-devel) and associated GitHub issue to learn more about the idea. ``` ### Adding a Proposal @@ -92,105 +91,6 @@ If applicable, links to more information or discussions **Mentor**: Name and email address. ``` -### Template manager, new template distribution mechanism - -**Project**: Template manager, new template distribution mechanism - -**Brief explanation**: Template VMs currently are distributed using RPM -packages. There are multiple problems with that, mostly related to static -nature of RPM package (what files belong to the package). This means such -Template VM cannot be renamed, migrated to another storage (like LVM), etc. -Also we don't want RPM to automatically update template package itself (which -would override all the user changes there). More details: -[#2064](https://github.com/QubesOS/qubes-issues/issues/2064), -[#2534](https://github.com/QubesOS/qubes-issues/issues/2534), -[#3573](https://github.com/QubesOS/qubes-issues/issues/3573). - -**Expected results**: - - - Design new mechanism for distributing templates (possibly including some - package format - either reuse something already existing, or design - new one). The mechanism needs to handle: - - integrity protection (digital signatures), not parsing any data in dom0 - prior to signature verification - - efficient handling of large sparse files - - ability to deploy the template into various storage mechanisms (sparse - files, LVM thin volumes etc). - - template metadata, templates repository - enable the user to browse - available templates (probably should be done in dedicated VM, or DisposableVM) - - manual template removal by users (without it, see problems such - as [#5509](https://github.com/QubesOS/qubes-issues/issues/5509) - - Implement the above mechanism: - - tool to download named template - should perform download operation in - some VM (as dom0 have no network access), then transfer the data to dom0, - verify its integrity and then create Template VM and feed it's root - filesystem image with downloaded data. - - tool to browse templates repository - both CLI and GUI (preferably integrated - with existing Template Manager tool) - - integrate both tools - user should be able to choose some template to be - installed from repository browsing tool - see - [#1705](https://github.com/QubesOS/qubes-issues/issues/1705) for some idea - (this one lacks integrity verification, but a similar service could - be developed with that added) - - If new "package" format is developed, add support for it into - [linux-template-builder](https://github.com/QubesOS/qubes-linux-template-builder). - - Document the mechanism. - - Write unit tests and integration tests. - -**Knowledge prerequisite**: - - - Large files (disk images) handling (sparse files, archive formats) - - Bash and Python scripting - - Data integrity handling - digital signatures (gpg2, gpgv2) - - PyGTK - - RPM package format, (yum) repository basics - -**Mentor**: [Marek Marczykowski-Górecki](/team/) - -### USB passthrough to Windows qubes - -**Project**: USB passthrough to Windows qubes - -**Brief explanation**: Add ability to use individual USB devices in Windows qubes. Right now the only option to do that, is to assign the whole USB controller (PCI device), which applies to all the devices connected to it. USB passthrough on Qubes is based on USBIP project, with transport over qrexec instead of TCP/IP. - -**Expected results**: - - - Evaluate possible approaches (including flexibility, compatibility and performance), suggested ideas: - - use [USBIP for Windows](https://github.com/cezuni/usbip-win) and make it work with qrexec - similar as done for Linux - - use qrexec+USBIP in Linux-based stubdomain and plug it into USB emulation in qemu - - Choose one approach, write (very simple) design documentation - - Write relevant new code (applies mostly for usbip-win case) - - Plug the mechanism into Qubes core toolstack ([Devices API](https://dev.qubes-os.org/projects/core-admin/en/latest/qubes-devices.html)) - -**Knowledge prerequisite**: - -- basic USB architecture knowledge (buses, devices, interfaces, functions) -- Python and Bash scripting -- C -- Windows USB stack and/or qemu USB stack - -**Mentor**: [Marek Marczykowski-Górecki](/team/) - -### Dedicated Audio qube - -**Project**: Dedicated Audio qube - -**Brief explanation**: Moving audio subsystem from dom0 to a dedicated AudioVM and/or a preexisting VM (e.g sys-usb with attached usb audio device). This would allow using USB audio devices system-wide, without leaving a USB controller in dom0. [Relevant github issue](https://github.com/QubesOS/qubes-issues/issues/1590). - -**Expected results**: - -- Make audio virtualization components work with non-dom0 backend (in short: add configuration option for the backend, instead of assuming "dom0") -- Possibly per-qube setting what should be used as an AudioVM -- Make other audio-related tools (including GUI tools) work with the new setup, especially enabling/disabling microphone (`qvm-device mic`) and volume control - -**Knowledge prerequisite**: - -- Pulseaudio -- C -- Python - -**Mentor**: [Marek Marczykowski-Górecki](/team/) - ### Qubes as a Vagrant provider **Project**: Qubes as a Vagrant provider @@ -199,17 +99,17 @@ would override all the user changes there). More details: **Expected results**: - - Design how Vagrant Qubes provider should look like, including: - - [box format](https://www.vagrantup.com/docs/plugins/providers.html#box-format) - - method for running commands inside (ssh vs qvm-run) - - Write a Vagrant provider able to create/start/stop/etc a VM - - Document how to configure and use the provider, including required qrexec policy changes and possibly firewall rules - - Write integration tests +- Design how Vagrant Qubes provider should look like, including: + - [box format](https://www.vagrantup.com/docs/plugins/providers.html#box-format) + - method for running commands inside (ssh vs qvm-run) +- Write a Vagrant provider able to create/start/stop/etc a VM +- Document how to configure and use the provider, including required qrexec policy changes and possibly firewall rules +- Write integration tests **Knowledge prerequisite**: - - Ruby - - Vagrant concepts +- Ruby +- Vagrant concepts **Mentor**: [Wojtek Porczyk](/team/), [Marek Marczykowski-Górecki](/team/) @@ -225,7 +125,6 @@ would override all the user changes there). More details: - Implementation of the above mechanism. - Documentation how to configure it securely. - **Knowledge prerequisite**: - shell and/or python scripting @@ -252,10 +151,11 @@ Choose either of GUI agent, GUI daemon. Both are of similar complexity and each - implement tests for new GUI handling, similar to existing tests for X11 based GUI Relevant links: - - [Low level GUI documentation](/doc/gui/) - - [qubes-gui-agent-linux](https://github.com/qubesos/qubes-gui-agent-linux) - - [qubes-gui-daemon](https://github.com/qubesos/qubes-gui-daemon) - - [Use Wayland instead of X11 to increase performance](https://github.com/qubesos/qubes-issues/issues/3366) + +- [Low level GUI documentation](/doc/gui/) +- [qubes-gui-agent-linux](https://github.com/qubesos/qubes-gui-agent-linux) +- [qubes-gui-daemon](https://github.com/qubesos/qubes-gui-daemon) +- [Use Wayland instead of X11 to increase performance](https://github.com/qubesos/qubes-issues/issues/3366) **Knowledge prerequisite**: @@ -282,21 +182,21 @@ details: [#1552](https://github.com/QubesOS/qubes-issues/issues/1552), **Expected results**: - - Adjust set of VMs and templates included in live edition. - - Update and fix build scripts for recent Qubes OS version. - - Update startup script to mount appropriate directories as either - copy-on-write (device-mapper snapshot), or tmpfs. - - Optimize memory usage: should be possible to run sys-net, sys-firewall, and - at least two more VMs on 4GB machine. This include minimizing writes to - copy-on-write layer and tmpfs (disable logging etc). - - Research option to install the system from live image. If feasible add - this option. +- Adjust set of VMs and templates included in live edition. +- Update and fix build scripts for recent Qubes OS version. +- Update startup script to mount appropriate directories as either + copy-on-write (device-mapper snapshot), or tmpfs. +- Optimize memory usage: should be possible to run sys-net, sys-firewall, and + at least two more VMs on 4GB machine. This include minimizing writes to + copy-on-write layer and tmpfs (disable logging etc). +- Research option to install the system from live image. If feasible add + this option. **Knowledge prerequisite**: - - System startup sequence: bootloaders (isolinux, syslinux, grub, UEFI), initramfs, systemd. - - Python and Bash scripting - - Filesystems and block devices: loop devices, device-mapper, tmpfs, overlayfs, sparse files. +- System startup sequence: bootloaders (isolinux, syslinux, grub, UEFI), initramfs, systemd. +- Python and Bash scripting +- Filesystems and block devices: loop devices, device-mapper, tmpfs, overlayfs, sparse files. **Mentor**: [Frédéric Pierret](/team/) @@ -320,7 +220,6 @@ REMOVED as of January 2020: work is being done on this **Mentor**: [Thomas Leonard](mailto:talex5@gmail.com), [Marek Marczykowski-Górecki](/team/) --> - ### LogVM(s) **Project**: LogVM(s) @@ -333,72 +232,35 @@ immune to altering past entries. See **Expected results**: - - Design a _simple_ protocol for transferring logs. The less metadata (parsed - in log-collecting VM) the better. - - Implement log collecting service. Besides logs itself, should save - information about logs origin (VM name) and timestamp. The service should - _not_ trust sending VM in any of those. - - Implement log forwarder compatible with systemd-journald and rsyslog. A - mechanism (service/plugin) fetching logs in real time from those and sending - to log-collecting VM over qrexec service. - - Document the protocol. - - Write unit tests and integration tests. +- Design a _simple_ protocol for transferring logs. The less metadata (parsed + in log-collecting VM) the better. +- Implement log collecting service. Besides logs itself, should save + information about logs origin (VM name) and timestamp. The service should + _not_ trust sending VM in any of those. +- Implement log forwarder compatible with systemd-journald and rsyslog. A + mechanism (service/plugin) fetching logs in real time from those and sending + to log-collecting VM over qrexec service. +- Document the protocol. +- Write unit tests and integration tests. **Knowledge prerequisite**: - - syslog - - systemd - - Python/Bash scripting +- syslog +- systemd +- Python/Bash scripting **Mentor**: [Frédéric Pierret](/team/) - - ### Whonix IPv6 and nftables support + **Project**: Whonix IPv6 and nftables support **Brief explanation**: [T509](https://phabricator.whonix.org/T509) **Expected results**: -- Work at upstream Tor: An older version of https://trac.torproject.org/projects/tor/wiki/doc/TransparentProxy page was the origin of Whonix. Update that page for nftables / IPv6 support without mentioning Whonix. Then discuss that on the tor-talk mailing list for wider input. - https://trac.torproject.org/projects/tor/ticket/21397 -- implement corridor feature request add IPv6 support / port to nftables - https://github.com/rustybird/corridor/issues/39 +- Work at upstream Tor: An older version of [TransparentProxy](https://trac.torproject.org/projects/tor/wiki/doc/TransparentProxy) page was the origin of Whonix. Update that page for nftables / IPv6 support without mentioning Whonix. Then discuss that on the tor-talk mailing list for wider input. [here](https://trac.torproject.org/projects/tor/ticket/21397) +- implement corridor feature request add IPv6 support / port to nftables - [issue](https://github.com/rustybird/corridor/issues/39) - port [whonix-firewall](https://github.com/Whonix/whonix-firewall) to nftables - make connections to IPv6 Tor relays work - make connections to IPv6 destinations work @@ -411,29 +273,8 @@ details in [#2618](https://github.com/QubesOS/qubes-issues/issues/2618). **Mentor**: [Patrick Schleizer](/team/) -### Audio support for Qubes Windows Tools -**Project**: Audio support for Qubes Windows Tools - -**Brief explanation**: Add audio support for Windows HVMs via Qubes Windows Tools. [#2624](https://github.com/QubesOS/qubes-issues/issues/2624) - -**Expected results**: Windows HVMs should have an audio device that supports playback and recording. - -**Knowledge prerequisite**: C/C++ languages, familiarity with Windows API, possibly familiarity with Windows audio stack on the driver level. - -**Mentor**: [Rafał Wojdyła](/team/) - -### Improve Windows GUI agent performance and stability -**Project**: Improve Windows GUI agent performance and stability - -**Brief explanation**: Previous profiling has shown that the Windows GUI agent uses significant portion of VM's CPU time for mouse input simulation. This can be improved, as well as agent's stability in some cases (desktop/user switching, logon/logoff, domain-joined VMs, multiple monitors). Seamless GUI experience can be significantly improved, but that may require changes in the Qubes video driver. [#1044](https://github.com/QubesOS/qubes-issues/issues/1044) [#1045](https://github.com/QubesOS/qubes-issues/issues/1045) [#1500](https://github.com/QubesOS/qubes-issues/issues/1500) [#2138](https://github.com/QubesOS/qubes-issues/issues/2138) [#2487](https://github.com/QubesOS/qubes-issues/issues/2487) [#2589](https://github.com/QubesOS/qubes-issues/issues/2589) - -**Expected results**: Reduction of agent's CPU usage, improved stability. - -**Knowledge prerequisite**: C language, Familiarity with Windows API, especially the windowing stack. Familiarity with profiling and debugging tools for Windows. - -**Mentor**: [Rafał Wojdyła](/team/) - ### GUI agent for Windows 8/10 + **Project**: GUI agent for Windows 8/10 **Brief explanation**: Add support for Windows 8+ to the Qubes GUI agent and video driver. Starting from Windows 8, Microsoft requires all video drivers to conform to the WDDM display driver model which is incompatible with the current Qubes video driver. Unfortunately the WDDM model is much more complex than the old XPDM one and officially *requires* a physical GPU device (which may be emulated). Some progress has been made to create a full WDDM driver that *doesn't* require a GPU device, but the driver isn't working correctly yet. Alternatively, WDDM model supports display-only drivers which are much simpler but don't have access to system video memory and rendering surfaces (a key feature that would simplify seamless GUI mode). [#1861](https://github.com/QubesOS/qubes-issues/issues/1861) @@ -452,19 +293,19 @@ details in [#2618](https://github.com/QubesOS/qubes-issues/issues/2618). **Expected results**: - - A template for `autounattended.xml` file for Windows installer - the template should have placeholders for settings that need to be provided by the user. - - A tool for generating actual `autounattended.xml` file based on the template and user settings. - - A tool for launching Windows installation, given installation image and `autounattended.xml` file (can be the same as in the above point). - - (Optional) Unattended installation should also include Qubes Windows Tools. - - (Optional) A tool should be able to use Windows license embedded in ACPI tables - [related discussion](https://groups.google.com/d/msgid/qubes-devel/0b7fabae-f843-e7ce-40cf-193326cecdb0%40zrubi.hu) - - User documentation - - Automated tests (unit tests, integration tests) +- A template for `autounattended.xml` file for Windows installer - the template should have placeholders for settings that need to be provided by the user. +- A tool for generating actual `autounattended.xml` file based on the template and user settings. +- A tool for launching Windows installation, given installation image and `autounattended.xml` file (can be the same as in the above point). +- (Optional) Unattended installation should also include Qubes Windows Tools. +- (Optional) A tool should be able to use Windows license embedded in ACPI tables - [related discussion](https://groups.google.com/d/msgid/qubes-devel/0b7fabae-f843-e7ce-40cf-193326cecdb0%40zrubi.hu) +- User documentation +- Automated tests (unit tests, integration tests) **Knowledge prerequisite**: - - Python scripting - - Linux administration, including handling loop devices, partition tables, filesystems etc - - For optional features, C language and x86 architecture (ACPI tables) +- Python scripting +- Linux administration, including handling loop devices, partition tables, filesystems etc +- For optional features, C language and x86 architecture (ACPI tables) **Mentor**: [Rafał Wojdyła](/team/), [Marek Marczykowski-Górecki](/team/) @@ -474,33 +315,33 @@ details in [#2618](https://github.com/QubesOS/qubes-issues/issues/2618). **Brief explanation**: Integrating GNOME into Qubes dom0. This include: - - patching window manager to add colorful borders - - removing stuff not needed in dom0 (file manager(s), indexing services etc) - - adjusting menu for easy navigation (same applications in different VMs and such problems, dom0-related entries in one place) - - More info: [#1806](https://github.com/QubesOS/qubes-issues/issues/1806) +- patching window manager to add colorful borders +- removing stuff not needed in dom0 (file manager(s), indexing services etc) +- adjusting menu for easy navigation (same applications in different VMs and such problems, dom0-related entries in one place) +- More info: [#1806](https://github.com/QubesOS/qubes-issues/issues/1806) **Expected results**: - - Review existing support for other desktop environments (KDE, Xfce4, i3, awesome). - - Patch window manager to draw colorful borders (we use only server-side - decorations), there is already very similar patch in - [Cappsule project](https://github.com/cappsule/cappsule-gui). - - Configure GNOME to not make use of dom0 user home in visible way (no search - in files there, no file manager, etc). - - Configure GNOME to not look into external devices plugged in (no auto - mounting, device notifications etc). - - Package above modifications as rpms, preferably as extra configuration files - and/or plugins than overwriting existing files. Exceptions to this rule may - apply if no other option. - - Adjust comps.xml (in installer-qubes-os repo) to define package group with - all required packages. - - Document installation procedure. +- Review existing support for other desktop environments (KDE, Xfce4, i3, awesome). +- Patch window manager to draw colorful borders (we use only server-side + decorations), there is already very similar patch in + [Cappsule project](https://github.com/cappsule/cappsule-gui). +- Configure GNOME to not make use of dom0 user home in visible way (no search + in files there, no file manager, etc). +- Configure GNOME to not look into external devices plugged in (no auto + mounting, device notifications etc). +- Package above modifications as RPMs, preferably as extra configuration files + and/or plugins than overwriting existing files. Exceptions to this rule may + apply if no other option. +- Adjust comps.xml (in installer-qubes-os repo) to define package group with + all required packages. +- Document installation procedure. **Knowledge prerequisite**: - - GNOME architecture - - C language (patching metacity) - - Probably also javascript - for modifying GNOME shell extensions +- GNOME architecture +- C language (patching metacity) +- Probably also javascript - for modifying GNOME shell extensions **Mentor**: [Frédéric Pierret](/team/), [Marek Marczykowski-Górecki](/team/) @@ -508,7 +349,7 @@ details in [#2618](https://github.com/QubesOS/qubes-issues/issues/2618). **Project**: Qubes Converters -**Brief explanation**: One of the pioneering ideas of Qubes is to use disposable virtual machines to convert untrustworthy files (such as documents given to journalists by unknown and potentially malicious whistleblowers) into trustworthy files. See [Joanna's blog on the Qubes PDF Convert](http://theinvisiblethings.blogspot.co.uk/2013/02/converting-untrusted-pdfs-into-trusted.html) for details of the idea. Joanna has implemented a prototype for PDF documents. The goal of this project would be to generalize beyond the simple prototype to accommodate a wide variety of file formats, including Word documents, audio files, video files, spreadsheets, and so on. The converters should prioritise safety over faithful conversion. For example the Qubes PDF converter typically leads to lower quality PDFs (e.g. cut and paste is no longer possible), because this makes the conversion process safer. +**Brief explanation**: One of the pioneering ideas of Qubes is to use disposable virtual machines to convert untrustworthy files (such as documents given to journalists by unknown and potentially malicious whistleblowers) into trustworthy files. See [Joanna's blog on the Qubes PDF Convert](https://theinvisiblethings.blogspot.co.uk/2013/02/converting-untrusted-pdfs-into-trusted.html) for details of the idea. Joanna has implemented a prototype for PDF documents. The goal of this project would be to generalize beyond the simple prototype to accommodate a wide variety of file formats, including Word documents, audio files, video files, spreadsheets, and so on. The converters should prioritise safety over faithful conversion. For example the Qubes PDF converter typically leads to lower quality PDFs (e.g. cut and paste is no longer possible), because this makes the conversion process safer. **Expected results**: We expect that in the timeframe, it will be possible to implement many converters for many file formats. However, if any unexpected difficulties arise, we would prioritise a small number of safe and high quality converters over a large number of unsafe or unuseful converters. @@ -517,6 +358,7 @@ details in [#2618](https://github.com/QubesOS/qubes-issues/issues/2618). **Mentors**: Andrew Clausen and Jean-Philippe Ouellet ### Progress towards reproducible builds + **Project**: Progress towards reproducible builds **Brief explanation**: A long-term goal is to be able to build the entire OS and installation media in a completely bit-wise deterministic manner, but there are many baby steps to be taken along that path. See: @@ -543,24 +385,28 @@ Qubes currently only supports the x86_64 CPU architecture. Xen currently has add Some related discussion: - - [#4318](https://github.com/QubesOS/qubes-issues/issues/4318) on porting to ppc64. - - [#3894](https://github.com/QubesOS/qubes-issues/issues/3894) on porting to L4 microkernel. +- [#4318](https://github.com/QubesOS/qubes-issues/issues/4318) on porting to ppc64. +- [#3894](https://github.com/QubesOS/qubes-issues/issues/3894) on porting to L4 microkernel. **Expected results**: - - Add cross-compilation support to qubes-builder and related components. - - Make aarch64 specific adjustments to Qubes toolstacks/manager (including passthrough of devices from device tree to guest domains). - - Aarch64 specific integration and unit tests. - - Production of generic u-boot or uefi capable image/iso for target hardware. +- Add cross-compilation support to qubes-builder and related components. +- Make aarch64 specific adjustments to Qubes toolstacks/manager (including passthrough of devices from device tree to guest domains). +- Aarch64 specific integration and unit tests. +- Production of generic u-boot or uefi capable image/iso for target hardware. **Knowledge prerequisite**: - - Libvirt and Qubes toolstacks (C and python languages). - - Xen debugging. - - General ARM architecture knowledge. - +- Libvirt and Qubes toolstacks (C and python languages). +- Xen debugging. +- General ARM architecture knowledge. + **Mentor**: [Marek Marczykowski-Górecki](/team/) + + ### Android development in Qubes **Project**: Research running Android in Qubes VM (probably HVM) and connecting it to Android Studio @@ -603,22 +451,23 @@ Since it's software emulation it's rather slow. Details, reference: [#2233](https://github.com/QubesOS/qubes-issues/issues/2233) **Expected results**: - - a simple way of setting up Android qubes with hardware emulation + +- a simple way of setting up Android qubes with hardware emulation (distributed as a template or as a salt, handling various modern Android versions) - - figuring out and implementing an easy and secure way to connect an Android +- figuring out and implementing an easy and secure way to connect an Android qube to a development qube with Android studio - - documentation and tests +- documentation and tests **Knowledge prerequisite**: -**Mentor**: Inquire on [qubes-devel][ml-devel]. +**Mentor**: Inquire on [qubes-devel](/support/#qubes-devel). ### Admin API Fuzzer **Project**: Develop a [Fuzzer](https://en.wikipedia.org/wiki/Fuzzing) for the -[Qubes OS Admin API](https://www.qubes-os.org/doc/admin-api/). +[Qubes OS Admin API](/doc/admin-api/). -**Brief explanation**: The [Qubes OS Admin API](https://www.qubes-os.org/doc/admin-api/) +**Brief explanation**: The [Qubes OS Admin API](/doc/admin-api/) enables VMs to execute privileged actions on other VMs or dom0 - if allowed by the Qubes OS RPC policy. Programming errors in the Admin API however may cause these access rights to be more permissive than anticipated by the programmer. @@ -627,34 +476,110 @@ Since the Admin API is continuously growing and changing, continuous security as A [Fuzzer](https://en.wikipedia.org/wiki/Fuzzing) would help to automate part of these assessments. **Expected results**: - - fully automated & extensible Fuzzer for parts of the Admin API - - user & developer documentation + +- fully automated & extensible Fuzzer for parts of the Admin API +- user & developer documentation **Prerequisites**: - - basic Python understanding - - some knowledge about fuzzing & existing fuzzing frameworks (e.g. [oss-fuzz](https://github.com/google/oss-fuzz/tree/master/projects/qubes-os)) - - a hacker's curiosity + +- basic Python understanding +- some knowledge about fuzzing & existing fuzzing frameworks (e.g. [oss-fuzz](https://github.com/google/oss-fuzz/tree/master/projects/qubes-os)) +- a hacker's curiosity **Mentor**: Inquire on [qubes-devel][ml-devel]. +### Secure Boot support + +**Project**: Add support for protecting boot binaries with Secure Boot technology, using user-generated keys. + +**Brief explanation**: Since recently, Xen supports "unified EFI boot" which allows to sign not only Xen binary itself, but also dom0 kernel and their parameters. While the base technology is there, enabling it is a painful and complex process. The goal of this project is to integrate configuration of this feature into Qubes, automating as much as possible. See discussion in [issue #4371](https://github.com/QubesOS/qubes-issues/issues/4371) + +**Expected results**: + - a tool to prepare relevant boot files for unified Xen EFI boot - this includes collecting Xen, dom0 kernel, initramfs, config file, and possibly few more (ucode update?); the tool should then sign the file with user provided key (preferably propose to generate it too) + - integrate it with updates mechanism, so new Xen or dom0 kernel will be picked up automatically + - include a fallback configuration that can be used for troubleshooting (main unified Xen EFI intentionally does not allow to manipulate parameters at boot time) + +**Knowledge prerequisite**: + - basic understanding of Secure Boot + - Bash and Python scripting + +**Mentor**: [Marek Marczykowski-Górecki](/team/) + +### Reduce logging of Disposable VMs + +**Project**: Reduce logging of Disposable VMs + +**Brief explanation**: Partial metadata of a DisposableVM is stored in the dom0 filesystem. This applies to various logs, GUI status files etc. There should be an option to hide as much of that as possible - including bypassing some logging, and removing various state files, or at the very least obfuscating any hints what is running inside DisposableVM. More details at [issue #4972](https://github.com/QubesOS/qubes-issues/issues/4972) + +**Expected results**: A DisposableVM should not leave logs hinting what was running inside. + +**Knowledge prerequisite**: + - Python scripting + - Basic knowledge of Linux system services management (systemd, syslog etc) + +**Mentor**: [Marek Marczykowski-Górecki](/team/) + +## Past Projects + +You can view the projects we had in 2017 in the [GSoC 2017 archive](https://summerofcode.withgoogle.com/archive/2017/organizations/5074771758809088/). We also participated in GSoC 2020, and you can see the project in the [GSoC 2020 archive](https://summerofcode.withgoogle.com/archive/2020/organizations/4924517870206976/). + +Here are some successful projects which have been implemented in the past by Google Summer of Code participants. + +### Template manager, new template distribution mechanism + +**Project**: Template manager, new template distribution mechanism + +**Brief explanation**: Template VMs currently are distributed using RPM +packages. There are multiple problems with that, mostly related to static +nature of RPM package (what files belong to the package). This means such +Template VM cannot be renamed, migrated to another storage (like LVM), etc. +Also we don't want RPM to automatically update template package itself (which +would override all the user changes there). More details: +[#2064](https://github.com/QubesOS/qubes-issues/issues/2064), +[#2534](https://github.com/QubesOS/qubes-issues/issues/2534), +[#3573](https://github.com/QubesOS/qubes-issues/issues/3573). + +**Expected results**: + +- Design new mechanism for distributing templates (possibly including some + package format - either reuse something already existing, or design + new one). The mechanism needs to handle: + - integrity protection (digital signatures), not parsing any data in dom0 + prior to signature verification + - efficient handling of large sparse files + - ability to deploy the template into various storage mechanisms (sparse + files, LVM thin volumes etc). + - template metadata, templates repository - enable the user to browse + available templates (probably should be done in dedicated VM, or DisposableVM) + - manual template removal by users (without it, see problems such + as [#5509](https://github.com/QubesOS/qubes-issues/issues/5509) +- Implement the above mechanism: + - tool to download named template - should perform download operation in + some VM (as dom0 have no network access), then transfer the data to dom0, + verify its integrity and then create Template VM and feed it's root + filesystem image with downloaded data. + - tool to browse templates repository - both CLI and GUI (preferably integrated + with existing Template Manager tool) + - integrate both tools - user should be able to choose some template to be + installed from repository browsing tool - see + [#1705](https://github.com/QubesOS/qubes-issues/issues/1705) for some idea + (this one lacks integrity verification, but a similar service could + be developed with that added) +- If new "package" format is developed, add support for it into + [linux-template-builder](https://github.com/QubesOS/qubes-linux-template-builder). +- Document the mechanism. +- Write unit tests and integration tests. + +**Knowledge prerequisite**: + +- Large files (disk images) handling (sparse files, archive formats) +- Bash and Python scripting +- Data integrity handling - digital signatures (gpg2, gpgv2) +- PyGTK +- RPM package format, (yum) repository basics + +**Mentor**: [Marek Marczykowski-Górecki](/team/) + ---- We adapted some of the language here about GSoC from the [KDE GSoC page](https://community.kde.org/GSoC). - -[2017-archive]: https://summerofcode.withgoogle.com/archive/2017/organizations/5074771758809088/ -[gsoc-qubes]: https://summerofcode.withgoogle.com/organizations/4675790572093440/ -[gsoc]: https://summerofcode.withgoogle.com/ -[team]: /team/ -[gsoc-faq]: https://developers.google.com/open-source/gsoc/faq -[contributing]: /doc/contributing/#contributing-code -[patches]: /doc/source-code/#how-to-send-patches -[code-signing]: /doc/code-signing/ -[ml-devel]: /support/#qubes-devel -[gsoc-participate]: https://developers.google.com/open-source/gsoc/ -[gsoc-student]: https://developers.google.com/open-source/gsoc/resources/manual#student_manual -[how-to-gsoc]: http://teom.org/blog/kde/how-to-write-a-kick-ass-proposal-for-google-summer-of-code/ -[gsoc-submit]: https://summerofcode.withgoogle.com/ -[mailing-lists]: /support/ -[qubes-issues]: https://github.com/QubesOS/qubes-issues/issues -[qubes-issues-suggested]: https://github.com/QubesOS/qubes-issues/issues?q=is%3Aissue%20is%3Aopen%20label%3A%22P%3A%20minor%22%20label%3A%22help%20wanted%22 -[qubes-builder]: /doc/qubes-builder/ diff --git a/developer/general/gsod.md b/developer/general/gsod.md index aa5f1262df..82a6be1019 100644 --- a/developer/general/gsod.md +++ b/developer/general/gsod.md @@ -1,14 +1,12 @@ --- -layout: sidebar -title: Google Season of Docs +lang: en +layout: doc permalink: /gsod/ +ref: 242 +title: Google Season of Docs (GSoD) --- -# 2020 Google Season of Docs - -Thank you for your interest in participating in the [2020 Google Season of Docs][gsod] program with the [Qubes OS team][team]. You can read more about the Google Season of Docs in the official [guides][gsod-doc] and [FAQ][gsod-faq]. - -You can view the project we had in 2019 in the [2019 GSoD archive][2019-qubes-gsod] and the [2019 writer's report][2019-qubes-report]. +Thank you for your interest in participating in the [2021 Google Season of Docs](https://developers.google.com/season-of-docs/) program with the [Qubes OS team](/team/). You can read more about the Google Season of Docs in the official [guides](https://developers.google.com/season-of-docs/docs/) and [FAQ](https://developers.google.com/season-of-docs/docs/faq). ## Project Ideas List @@ -31,6 +29,7 @@ Here's a suggested template for adding project ideas: **Mentor**: Name and email address. ``` + ### Offline documentation **Project**: Offline documentation @@ -39,13 +38,13 @@ Here's a suggested template for adding project ideas: **Expected results**: - - Review [past discussions on the issue](https://github.com/QubesOS/qubes-issues/issues/1019) - - Recommend workflow and platform for displaying offline documentation - - Test workflow and platform to ensure usability and functionality +- Review [past discussions on the issue](https://github.com/QubesOS/qubes-issues/issues/1019) +- Recommend workflow and platform for displaying offline documentation +- Test workflow and platform to ensure usability and functionality **Knowledge prerequisite**: - - [Markdown][markdown] +- [Markdown][markdown] **Mentor**: [Marek Marczykowski-Górecki][team] @@ -57,12 +56,12 @@ Here's a suggested template for adding project ideas: **Expected results**: - - Review [past discussions on the issue](https://github.com/QubesOS/qubes-issues/issues/1774) - - Provide visual mock-ups and proposed text +- Review [past discussions on the issue](https://github.com/QubesOS/qubes-issues/issues/1774) +- Provide visual mock-ups and proposed text **Knowledge prerequisite**: - - some experience with Anaconda would be helpful +- some experience with Anaconda would be helpful **Mentor**: [Marek Marczykowski-Górecki][team] @@ -88,56 +87,88 @@ Additionally, terminology is used inconsistently. **Mentor**: [Marek Marczykowski-Górecki][team] +### Installation Guide for Qubes OS on Virtual Machines + +**Project**: Installation Guide for Qubes OS on Virtual Machines. + +**Brief explanation**: The Qubes OS is missing an installation guide for virtual machines. Users are installing an outdated and unsupported version of Qubes OS (3.2) instead of the supported version. There is unofficial [existing installation guide] for Qubes OS on a virtual box but it is misleading and lacks documentation. Usually, users face some errors and bugs while installing Qubes OS on a virtual machine.[virtual box issue] + +**Expected results**: + +- Provide a new option of installation guide for users working on virtual machines. +- Review existing problems and provide solutions to them. +- Giving a warning for using outdated versions. + +**Knowledge prerequisite**: + +- Experience in virtual boxes and machines. +- Basic Knowledge about Fedora linux architecture. +- [Markdown][markdown] + +**Mentor**: [Marek Marczykowski-Górecki][team] + +## Past Projects + +You can view the project we had in 2019 in the [2019 GSoD archive](https://developers.google.com/season-of-docs/docs/2019/participants/project-qubes) and the [2019 writer's report](https://refre.ch/report-qubesos/). + +You can also view the project we had in 2020 in the [2020 GSoD archive](https://developers.google.com/season-of-docs/docs/2020/participants/project-qubesos-c1e0) and the [2020 writer's report](https://gist.github.com/PROTechThor/bfe9b8b28295d88c438b6f6c754ae733). + +Here are some successful projects which have been implemented in the past by Google Season of Docs participants. + ### Consolidate troubleshooting guides **Project**: Consolidate troubleshooting guides **Brief explanation**: Troubleshooting guides are scattered across many pages and sometimes incomplete, leading to repeatedly posting the same instruction over and over when helping users to diagnose problems. -This could be helped by writing consolidated guide with with a clear list of symptom-action layout. +This could be helped by writing a consolidated guide with a clear list of symptom-action layout. **Expected results**: -- Review existing [troubleshooting guides](https://www.qubes-os.org/doc/#troubleshooting) -- Review [issues][doc-issues] containing common troubleshooting steps (checking specific logs etc) +- Review existing [troubleshooting guides](/doc/#troubleshooting) +- Review [issues](https://github.com/QubesOS/qubes-issues/issues?q=is%3Aopen+is%3Aissue+label%3A%22C%3A+doc%22) containing common troubleshooting steps (checking specific logs etc) - Propose updated, consolidated troubleshooting documentation, including its layout **Knowledge prerequisite**: -- [Markdown][markdown] +- [Markdown](https://daringfireball.net/projects/markdown/) -**Mentor**: [Marek Marczykowski-Górecki][team] +**Mentor**: [Marek Marczykowski-Górecki](/team/) -### Installation Guide for Qubes OS on Virtual Machines +### Improve Getting Started page -**Project**: Installation Guide for Qubes OS on Virtual Machines. +**Project**: Improve Getting Started page -**Brief explanation**: The Qubes OS is missing an installation guide for virtual machines. Users are installing an outdated and unsupported version of Qubes OS (3.2) instead of the supported version. There is unofficial [existing installation guide] for Qubes OS on a virtual box but it is misleading and lacks documentation. Usually, users face some errors and bugs while installing Qubes OS on a virtual machine.[virtual box issue] +**Brief explanation**: The [Getting Started page](https://www.qubes-os.org/doc/getting-started/) is the place a new user would go to understand better how to use Qubes. It is currently has old screenshots not using the default desktop environment and could have much better flow. In addition, this improved page content may end up being served more directly to the user via the [offline documentation](https://github.com/QubesOS/qubes-issues/issues/1019) or the firstboot guide. **Expected results**: - -Provide a new option of installation guide for users working on virtual machines. - -Review existing problems and provide solutions to them. - -Giving a warning for using outdated versions. + +- Review the existing page and website, similar pages for other OSes +- Provide visual mock-ups and proposed text **Knowledge prerequisite**: - - Experience in virtual boxes and machines. - - Basic Knowledge about Fedora linux architecture. - - [Markdown][markdown] - -**Mentor**: [Marek Marczykowski-Górecki][team] -[gsod]: https://developers.google.com/season-of-docs/ -[team]: /team/ -[gsod-doc]: https://developers.google.com/season-of-docs/docs/ -[gsod-faq]: https://developers.google.com/season-of-docs/docs/faq -[gsod.md]: https://github.com/QubesOS/qubes-doc/blob/master/developer/general/gsod.md -[gsod-2020-thread]: https://groups.google.com/d/msgid/qubes-project/aac9b148-4081-ebd8-cb9d-9a9191033484%40qubes-os.org -[Documentation Guidelines]: /doc/doc-guidelines/ -[Help, Support, and Mailing Lists]: /support/ -[intro]: /intro/ -[getting started]: /getting-started/ -[markdown]: https://daringfireball.net/projects/markdown/ -[doc-issues]: https://github.com/QubesOS/qubes-issues/issues?q=is%3Aopen+is%3Aissue+label%3A%22C%3A+doc%22 -[2019-qubes-gsod]: https://developers.google.com/season-of-docs/docs/2019/participants/project-qubes -[2019-qubes-report]: https://refre.ch/report-qubesos/ -[existing installation guide]: https://www.youtube.com/watch?v=mATI8Lht0Js -[virtual box issue]: https://www.virtualbox.org/ticket/16771 +- basic Qubes OS knowledge +- [Markdown](https://daringfireball.net/projects/markdown/) + +**Mentor**: [Michael Carbone](/team/) + +### Rewrite qrexec documentation + +**Project**: Rewrite qrexec documentation + +**Brief explanation**: Current qrexec (qubes remote exec) documentation is hard to follow, important informations are hidden within a wall of text. +Some parts are split into multiple sections, for example version specific to avoid duplication, but it doesn't help reading it. +Additionally, protocol documentation describes only few specific use cases, instead of being clear and precise protocol specification. +Fixing this last point may require very close cooperation with developers, as the current documentation doesn't multiple corner cases (that's one of the issue with its current shape). + +**Expected results**: + +- Review existing [qrexec documentation](https://www.qubes-os.org/doc/qrexec3/) and an [issue about it](https://github.com/QubesOS/qubes-issues/issues/1392) +- Propose updated, consolidated admin documentation (policy writing, adding services) +- Propose consolidated protocol specification, based on the current documentation, and cooperation with developers + +**Knowledge prerequisite**: + +- [Markdown](https://daringfireball.net/projects/markdown/) + +**Mentor**: [Marek Marczykowski-Górecki](/team/) diff --git a/developer/general/how-to-edit-the-documentation.md b/developer/general/how-to-edit-the-documentation.md new file mode 100644 index 0000000000..f51fe51287 --- /dev/null +++ b/developer/general/how-to-edit-the-documentation.md @@ -0,0 +1,198 @@ +--- +lang: en +layout: doc +permalink: /doc/how-to-edit-the-documentation/ +title: How to edit the documentation +--- + +_Also see the [documentation style guide](/doc/documentation-style-guide/)._ + +Qubes OS documentation pages are stored as plain text Markdown files in the +[qubes-doc](https://github.com/QubesOS/qubes-doc) repository. By cloning and +regularly pulling from this repo, users can maintain their own up-to-date +offline copy of all Qubes documentation rather than relying solely on the web. + +The documentation is a volunteer community effort. People like you are +constantly working to make it better. If you notice something that can be fixed +or improved, please follow the steps below to open a pull request! + +## How to submit a pull request + +We keep all the documentation in +[qubes-doc](https://github.com/QubesOS/qubes-doc), a dedicated Git repository +hosted on [GitHub](https://github.com/). Thanks to GitHub's easy web interface, +you can edit the documentation even if you're not familiar with Git or the +command line! All you need is a free GitHub account. + +A few notes before we get started: + +- Since Qubes is a security-oriented project, every documentation change will + be [reviewed](#security) before it's accepted. This allows us to maintain + quality control and protect our users. + +- To give your contribution a better chance of being accepted, please follow + our [documentation style guide](/doc/documentation-style-guide/). + +- We don't want you to spend time and effort on a contribution that we can't + accept. If your contribution would take a lot of time, please [file an + issue](/doc/issue-tracking/) for it first so that we can make sure we're on + the same page before significant works begins. + +- Alternatively, you may already have written content that doesn't conform to + these guidelines, but you'd be willing to modify it so that it does. In this + case, you can still submit it by following the instructions below. Just make + a note in your pull request (PR) that you're aware of the changes that need + to be made and that you're just asking for the content to be reviewed before + you spend time making those changes. + +- Finally, if you've written something that doesn't belong in qubes-doc but + that would be beneficial to the Qubes community, please consider adding it to + the [external + documentation](/doc/documentation-style-guide/#core-vs-external-documentation). + +(**Advanced users:** If you're already familiar with GitHub or wish to work +from the command line, you can skip the rest of this section. All you need to +do to contribute is to [fork and +clone](https://guides.github.com/activities/forking/) the +[qubes-doc](https://github.com/QubesOS/qubes-doc) repo, make your changes, then +[submit a pull +request](https://help.github.com/articles/using-pull-requests/).) + +Ok, let's begin. Every documentation page has a "Page Source on GitHub" button. +Depending on the size of your screen, it may either be on the side (larger +screens) or on the bottom (smaller screens). + +[![page-source-button](/attachment/doc/doc-pr_01_page-source-button.png)](/attachment/doc/doc-pr_01_page-source-button.png) + +When you click on it, you'll be taken to the source file --- usually a Markdown +(`.md`) file --- on GitHub. On this page, there will be a button to edit the +file. + +[![github-edit](/attachment/doc/doc-pr_02_github-edit.png)](/attachment/doc/doc-pr_02_github-edit.png) + +You'll be prompted to sign in with your GitHub username and password +(if you aren't already logged in). You can also create a free account from here. + +[![github-sign-in](/attachment/doc/doc-pr_03_sign-in.png)](/attachment/doc/doc-pr_03_sign-in.png) + +If this is your first contribution to the documentation, you need to "fork" the +repository (make your own copy). It's easy --- just click the big green button +on the next page. This step is only needed the first time you make a +contribution. + +[![fork](/attachment/doc/doc-pr_04_fork.png)](/attachment/doc/doc-pr_04_fork.png) + +Now you can make your modifications. You can also preview the changes to see +how they'll be formatted by clicking the "Preview changes" tab. If you want to +add images, please see [How to add images](#how-to-add-images). If you're +making formatting changes, please [render the site +locally](https://github.com/QubesOS/qubesos.github.io#instructions) to verify +that everything looks correct before submitting any changes. + +[![edit](/attachment/doc/doc-pr_05_edit.png)](/attachment/doc/doc-pr_05_edit.png) + +Once you're finished, describe your changes at the bottom and click "Propose +file change". + +[![commit](/attachment/doc/doc-pr_06_commit-msg.png)](/attachment/doc/doc-pr_06_commit-msg.png) + +After that, you'll see exactly what modifications you've made. At this stage, +those changes are still in your own copy of the documentation ("fork"). If +everything looks good, send those changes to us by pressing the "Create pull +request" button. + +[![pull-request](/attachment/doc/doc-pr_07_review-changes.png)](/attachment/doc/doc-pr_07_review-changes.png) + +You will be able to adjust the pull request message and title there. In most +cases, the defaults are ok, so you can just confirm by pressing the "Create +pull request" button again. However, if you're not ready for your PR to be +reviewed or merged yet, please [make a draft PR +instead](https://github.blog/2019-02-14-introducing-draft-pull-requests/). + +[![pull-request-confirm](/attachment/doc/doc-pr_08_create-pull-request.png)](/attachment/doc/doc-pr_08_create-pull-request.png) + +If any of your changes should be reflected in the [documentation index (a.k.a. +table of contents)](/doc/) --- for example, if you're adding a new page, +changing the title of an existing page, or removing a page --- please see [How +to edit the documentation index](#how-to-edit-the-documentation-index). + +That's all! We will review your changes. If everything looks good, we'll pull +them into the official documentation. Otherwise, we may have some questions for +you, which we'll post in a comment on your pull request. (GitHub will +automatically notify you if we do.) If, for some reason, we can't accept your +pull request, we'll post a comment explaining why we can't. + +[![done](/attachment/doc/doc-pr_09_done.png)](/attachment/doc/doc-pr_09_done.png) + +## How to edit the documentation index + +The source file for the [documentation index (a.k.a. table of contents)](/doc/) +is +[doc-index.yml](https://github.com/QubesOS/qubesos.github.io/blob/master/_data/doc-index.yml). + +Editing this file will change what appears on the documentation index. If your +pull request (PR) adds, removes, or edits anything that should be reflected in +the documentation index, please make sure you also submit an associated pull +request against this file. + +## How to add images + +To add an image to a page, use the following syntax in the main document (see +[here](/doc/documentation-style-guide/#image-linking) for why this syntax is +important). + +``` +[![Image Title](/attachment/doc/image.png)](/attachment/doc/image.png) +``` + +Then, submit your image(s) in a separate pull request to the +[qubes-attachment](https://github.com/QubesOS/qubes-attachment) repository +using the same path and filename. This is the only permitted way to include +images. Do not link to images on other websites. + +## Serving the website locally + +You can serve the website offline on your local machine by following [these +instructions](https://github.com/QubesOS/qubesos.github.io#instructions). This +can be useful for making sure that your changes render the way you expect, +especially when your changes affect formatting, images, tables, styling, etc. + +## Security + +*Also see: [Should I trust this website?](/faq/#should-i-trust-this-website)* + +All pull requests (PRs) against +[qubes-doc](https://github.com/QubesOS/qubes-doc) must pass review prior to be +merged, except in the case of [external +documentation](/doc/#external-documentation) (see +[#4693](https://github.com/QubesOS/qubes-issues/issues/4693)). This process is +designed to ensure that contributed text is accurate and non-malicious. This +process is a best effort that should provide a reasonable degree of assurance, +but it is not foolproof. For example, all text characters are checked for ANSI +escape sequences. However, binaries, such as images, are simply checked to +ensure they appear or function the way they should when the website is +rendered. They are not further analyzed in an attempt to determine whether they +are malicious. + +Once a pull request passes review, the reviewer should add a signed comment +stating, "Passed review as of ``" (or similar). The +documentation maintainer then verifies that the pull request is mechanically +sound (no merge conflicts, broken links, ANSI escapes, etc.). If so, the +documentation maintainer then merges the pull request, adds a PGP-signed tag to +the latest commit (usually the merge commit), then pushes to the remote. In +cases in which another reviewer is not required, the documentation maintainer +may review the pull request (in which case no signed comment is necessary, +since it would be redundant with the signed tag). + +## Questions, problems, and improvements + +If you have a question about something you read in the documentation or about +how to edit the documentation, please post it on the +[forum](https://forum.qubes-os.org/) or send it to the appropriate [mailing +list](/support/). If you see that something in the documentation should be +fixed or improved, please [contribute](#how-to-submit-a-pull-request) the +change yourself. To report an issue with the documentation, please follow our +standard [issue reporting guidelines](/doc/issue-tracking/). (If you report an +issue with the documentation, you will likely be asked to submit a pull request +for it, unless there is a clear indication in your report that you are not +willing or able to do so.) diff --git a/developer/general/join.md b/developer/general/join.md index ef195556bf..3e5e036a1d 100644 --- a/developer/general/join.md +++ b/developer/general/join.md @@ -1,13 +1,11 @@ --- -layout: sidebar -title: Join +lang: en +layout: doc permalink: /join/ +ref: 26 +title: Join --- -Joining the Qubes OS Team -========================= - The Qubes OS Project does not currently have any open positions. This page will be updated when open positions become available. -In the meantime, there are many different ways you can [contribute to the Qubes OS project](/doc/contributing/). - +In the meantime, there are many different ways you can [contribute to the Qubes OS project](/doc/contributing/). diff --git a/developer/general/package-contributions.md b/developer/general/package-contributions.md index 0c8bd9d5ec..f73c9bd25e 100644 --- a/developer/general/package-contributions.md +++ b/developer/general/package-contributions.md @@ -1,114 +1,102 @@ --- +lang: en layout: doc -title: Package Contributions permalink: /doc/package-contributions/ +ref: 29 +title: Package contributions --- -Package Contributions -===================== - _This page is for developers who wish to contribute packages. -If you want to install contributed packages, please see [installing contributed packages]._ +If you want to install contributed packages, please see [installing contributed packages](/doc/installing-contributed-packages/)._ We're very grateful to the talented and hard-working community members who contribute software packages to Qubes OS. This page explains the inclusion criteria and procedures for such packages, as well as the roles and responsibilities of those involved. Inclusion Criteria ------------------ + In order to be accepted, packages must: - * In no way weaken the security of Qubes OS. - * Be published under an open-source license (read about the [Qubes OS License]). - * Follow our [coding guidelines]. - * Be thoroughly tested. - * Have a clearly-defined use case for Qubes users. - * Not be unduly burdensome to review. +* In no way weaken the security of Qubes OS. +* Be published under an open-source license (read about the [Qubes OS License](/doc/license/)). +* Follow our [coding guidelines](/doc/coding-style/). +* Be thoroughly tested. +* Have a clearly-defined use case for Qubes users. +* Not be unduly burdensome to review. (Please note that we always reserve the right to add criteria to this list.) Contribution Procedure ---------------------- -Before you start putting serious work into a package, we recommend that you discuss your idea with the Qubes developers and the broader community on the [qubes-devel mailing list]. + +Before you start putting serious work into a package, we recommend that you discuss your idea with the Qubes developers and the broader community on the [qubes-devel mailing list](/support/#qubes-devel). Once you have a package that's ready to become part of Qubes OS, please follow this procedure: - 1. Ensure that your package satisfies the [Inclusion Criteria]. - 2. If your code isn't already on GitHub, create a GitHub repo that contains your code. You can have a look to an example package called [qubes-skeleton]. - 3. If you haven't already, [sign your code][sig]. - 4. Create an issue in [qubes-issues] with the title `[Contribution] your-package-name`. - Include a link to your repo, a brief description of your package, and a brief explanation of why you think it should be included in Qubes. - Please note that the Qubes core developers are very busy. - If they are under heavy load when you submit your contribution, it may be a very long time before they have time to review your package. - If this happens, please do not be discouraged. - If you think they may have forgotten about your pending contribution, you may "bump" your request by commenting on your issue, but please do this *very* sparingly (i.e., no more than once a month). - We appreciate your understanding! - 5. You may be asked followup questions. - If we decide to accept your contribution, you will be invited to join the [QubesOS-contrib] organization on GitHub as public recognition of your contribution (but without push access; see [Review Procedure]), and [QubesOS-contrib] will fork your repo. - If we decide not to accept your contribution, we will state the reason and close the issue. +1. Ensure that your package satisfies the [Inclusion Criteria](#inclusion-criteria). +2. If your code isn't already on GitHub, create a GitHub repo that contains your code. You can have a look to an example package called [qubes-skeleton](https://github.com/QubesOS-contrib/qubes-skeleton). +3. If you haven't already, [sign your code](/doc/code-signing/). +4. Create an issue in [qubes-issues](https://github.com/QubesOS/qubes-issues/issues/) with the title `[Contribution] your-package-name`. + Include a link to your repo, a brief description of your package, and a brief explanation of why you think it should be included in Qubes. + Please note that the Qubes core developers are very busy. + If they are under heavy load when you submit your contribution, it may be a very long time before they have time to review your package. + If this happens, please do not be discouraged. + If you think they may have forgotten about your pending contribution, you may "bump" your request by commenting on your issue, but please do this *very* sparingly (i.e., no more than once a month). + We appreciate your understanding! +5. You may be asked followup questions. + If we decide to accept your contribution, you will be invited to join the [QubesOS-contrib](https://github.com/QubesOS-contrib) organization on GitHub as public recognition of your contribution (but without push access; see [Review Procedure](#review-procedure)), and [QubesOS-contrib](https://github.com/QubesOS-contrib) will fork your repo. + If we decide not to accept your contribution, we will state the reason and close the issue. Update Procedure ---------------- + *Anyone* can provide an update (patch) to a contributed package, not just the person who contributed that package! The update procedure is the same for everyone, including the original package contributor. -If you would like to update an already-contributed package (specifically, a fork owned by [QubesOS-contrib]), please submit a [signed][sig], fast-forwardable pull request to that repo with your changes. -Please note that your pull request **must** be both [signed][sig] and fast-forwardable, or else it will be closed without further review. +If you would like to update an already-contributed package (specifically, a fork owned by [QubesOS-contrib](https://github.com/QubesOS-contrib)), please submit a [signed](/doc/code-signing/), fast-forwardable pull request to that repo with your changes. +Please note that your pull request **must** be both [signed](/doc/code-signing/) and fast-forwardable, or else it will be closed without further review. One or more reviewers may post comments on your pull request. Please be prepared to read and respond to these comments. Review Procedure ---------------- -This review procedure covers both original package contributions (see [Contribution Procedure]) and all subsequent updates to those packages, including updates from the original package contributor (see [Update Procedure]). -All changes will be reviewed by a Qubes Core Reviewer (QCR) and the [Package Maintainer] (PM). + +This review procedure covers both original package contributions (see [Contribution Procedure](#contribution-procedure)) and all subsequent updates to those packages, including updates from the original package contributor (see [Update Procedure](#update-procedure)). +All changes will be reviewed by a Qubes Core Reviewer (QCR) and the [Package Maintainer](#package-maintainers) (PM). In all cases, the QCR will be a core Qubes developer. In some cases, the QCR and the PM will be the same person. For example, if someone contributes a package, then disappears, and no suitable replacement has been found, then it is likely that a core Qubes developer will play both the QCR and PM roles for that package, at least until another suitable candidate volunteers to become the PM for that package. The review procedure is as follows: - 1. Someone, S, wishes to make a change to a package, P. - 2. S submits a fast-forwardable pull request against the fork of P's repo owned by [QubesOS-contrib]. - 3. The PM reviews the pull request. - If the the pull request passes the PM's review, the PM adds a [signed][sig] *comment* on the pull request stating that it has passed review. - (In cases in which S = PM, the PM can simply add a [signed][sig] *tag* to the HEAD commit prior to submitting the pull request.) - If the pull request does not pass the PM's review, the PM leaves a comment on the pull request explaining why not. - 4. The QCR reviews the pull request. - If the pull request passes the QCR's review, the QCR pushes a [signed][sig] tag to the HEAD commit stating that it has passed review and fast-forward merges the pull request. - If the pull request does not pass the QCR's review, the QCR leaves a comment on the pull request explaining why not, and the QCR may decide to close the pull request. - -In all the cases, the first condition to be validated by the QCR's review is to ensure that the contribution **will not** hijack any core packages of [QubesOS] and of course, none of the [QubesOS-contrib] packages too. More precisely, particular attention to the whole build pipeline will be made with a specific review of: - - Package dependencies, - - Build scripts (including downloaded ones), - - All downloaded components should be verified against static hash, - - RPM/DEB installation scripts (e.g. looking at constraints who would hijack other packages), - - Makefiles, - - Package build [reproducible] - -and any steps which would result in partial/total compromise of legitimate components. For this part, you can have a look to an example package called [qubes-skeleton]. +1. Someone, S, wishes to make a change to a package, P. +2. S submits a fast-forwardable pull request against the fork of P's repo owned by [QubesOS-contrib](https://github.com/QubesOS-contrib). +3. The PM reviews the pull request. + If the the pull request passes the PM's review, the PM adds a [signed](/doc/code-signing/) *comment* on the pull request stating that it has passed review. + (In cases in which S = PM, the PM can simply add a [signed](/doc/code-signing/) *tag* to the HEAD commit prior to submitting the pull request.) + If the pull request does not pass the PM's review, the PM leaves a comment on the pull request explaining why not. +4. The QCR reviews the pull request. + If the pull request passes the QCR's review, the QCR pushes a [signed](/doc/code-signing/) tag to the HEAD commit stating that it has passed review and fast-forward merges the pull request. + If the pull request does not pass the QCR's review, the QCR leaves a comment on the pull request explaining why not, and the QCR may decide to close the pull request. + +In all the cases, the first condition to be validated by the QCR's review is to ensure that the contribution **will not** hijack any core packages of [QubesOS](https://github.com/QubesOS) and of course, none of the [QubesOS-contrib](https://github.com/QubesOS-contrib) packages too. More precisely, particular attention to the whole build pipeline will be made with a specific review of: + +* Package dependencies, +* Build scripts (including downloaded ones), +* All downloaded components should be verified against static hash, +* RPM/DEB installation scripts (e.g. looking at constraints who would hijack other packages), +* Makefiles, +* Package build [reproducible](https://reproducible-builds.org/) + +and any steps which would result in partial/total compromise of legitimate components. For this part, you can have a look to an example package called [qubes-skeleton](https://github.com/QubesOS-contrib/qubes-skeleton). Package Maintainers ------------------- + If you contribute a package, we assume that you will be the maintainer of that package, unless you tell us otherwise. As the maintainer of the package, it is your privilege and responsibility to: - * [Review][Review Procedure] each pull request made against the package. - * Decide when the package has reached a new version, and notify the Qubes core developers when this occurs. +* [Review](#review-procedure) each pull request made against the package. +* Decide when the package has reached a new version, and notify the Qubes core developers when this occurs. If you do not wish to be the maintainer of your package, please let us know. If you do not act on your maintainer duties for a given package for an extended period of time and after at least one reminder, we will assume that you no longer wish to be the maintainer for that package. - - -[installing contributed packages]: /doc/installing-contributed-packages/ -[Inclusion Criteria]: #inclusion-criteria -[Contribution Procedure]: #contribution-procedure -[Update Procedure]: #update-procedure -[Review Procedure]: #review-procedure -[Package Maintainer]: #package-maintainers -[Qubes OS License]: /doc/license/ -[sig]: /doc/code-signing/ -[coding guidelines]: /doc/coding-style/ -[qubes-devel mailing list]: /support/#qubes-devel -[QubesOS]: https://github.com/QubesOS -[QubesOS-contrib]: https://github.com/QubesOS-contrib -[qubes-issues]: https://github.com/QubesOS/qubes-issues/issues/ -[reproducible]: https://reproducible-builds.org/ -[qubes-skeleton]: https://github.com/QubesOS-contrib/qubes-skeleton diff --git a/developer/general/research.md b/developer/general/research.md new file mode 100644 index 0000000000..74ac3c1884 --- /dev/null +++ b/developer/general/research.md @@ -0,0 +1,17 @@ +--- +lang: en +layout: site +permalink: /research/ +redirect_from: +- /doc/qubes-research/ +- /en/doc/qubes-research/ +- /doc/QubesResearch/ +- /wiki/QubesResearch/ +ref: 139 +title: Research +--- + +Here are links to various research papers, projects, videos, and blog posts +related to Qubes OS. + +{% include research.html %} diff --git a/developer/general/style-guide.md b/developer/general/style-guide.md deleted file mode 100644 index 795a0c5e9d..0000000000 --- a/developer/general/style-guide.md +++ /dev/null @@ -1,96 +0,0 @@ ---- -layout: doc -title: Style-guide -permalink: /doc/style-guide/ ---- - -Style Guide -=========== - -## Fonts - -Currently Qubes OS is using the following fonts for our website, branding, and other public facing (non-OS) materials. The OS itself uses what is normal for a user's desktop environment of choice. - -
-{% for font in site.data.styleguide.fonts %} -
-
-
Custom Qubes Font
-
-
- Family: {{font.family}}
-
-
-{% endfor %} -
- ---- - -## Colors - -The following **grayscale** colors are currently used on the Qubes website and documentation, and they will eventually match colors within the OS itself. - -
-{% for color in site.data.styleguide.colors %} - {% if color.type == "grayscale" %} -
-
- {{color.name}} - #{{color.hex | downcase}} -
- {% endif %} -{% endfor %} -
- -The following **colors** are currently being used on the Qubes website and documentation, and they will eventually match the colors within the OS itself! - -
-{% for color in site.data.styleguide.colors %} - {% if color.type == "colors" %} -
-
- {{color.name}} - #{{color.hex | downcase}} -
- {% endif %} -{% endfor %} -
- ---- - -## Icons - -Currently, all the icons on the Qubes-OS.org website are generated using [FontAwesome](http://fortawesome.github.io/Font-Awesome/). - -*As more custom work is done to generate icons for the operating system itself, they will be added here!* - ---- - -## Logos - -The following is a collection of various sizes and versions of the Qubes logo used both in the OS itself and on our website. -The artwork is licensed under Creative Commons Attribution-ShareAlike 4.0 International (CC BY-SA 4.0). -The code is licensed under GNU GPLv2. -GPLv2 and the source code can be [downloaded here](https://github.com/QubesOS/qubes-artwork). - -
-{% for logo in site.data.styleguide.logos %} - {% for version in logo.versions %} -
-
-
- -
-
-
-

- Image: {{logo.image}}
- Size: {{version.size}}
- Format: {{version.format}}
- Download: this image -

-
-
- {% endfor %} -{% endfor %} -
diff --git a/developer/general/usability-ux.md b/developer/general/usability-ux.md index d703ec76e3..5e0c53592b 100644 --- a/developer/general/usability-ux.md +++ b/developer/general/usability-ux.md @@ -1,15 +1,14 @@ --- +lang: en layout: doc -title: Usability & UX permalink: /doc/usability-ux/ +ref: 31 +title: Usability & UX --- -Usability & UX -============== - Software that is too complicated to use, is often unused. Because we want as many people as possible to benefit from its unique security properties, the usability and user experience of Qubes OS is an utmost priority! -We ask anyone developing for Qubes OS to please read through this guide to better understand the user experience we strive to achieve. We also ask them to review [our style guide](/doc/style-guide/) for other design related information. +We ask anyone developing for Qubes OS to please read through this guide to better understand the user experience we strive to achieve. We also ask them to review [our visual style guide](/doc/visual-style-guide/) for other design related information. --- @@ -39,7 +38,6 @@ Perhaps the most common cause of mistakes is complexity. If there is a configura In making software easy to use, it is crucial to be mindful of [cognitive load](https://en.wikipedia.org/wiki/Cognitive_load) which dictates that *"humans are generally able to hold only seven +/- two units of information in short-term memory."* Making sure your interfaces don't pass this short-term memory limit is perhaps the most important factor in helping a user feel comfortable instead of overwhelmed. - --- ## Easy to Understand diff --git a/developer/general/visual-style-guide.md b/developer/general/visual-style-guide.md new file mode 100644 index 0000000000..ceef871df8 --- /dev/null +++ b/developer/general/visual-style-guide.md @@ -0,0 +1,9 @@ +--- +lang: en +layout: doc +permalink: /doc/visual-style-guide/ +redirect_from: +- /doc/style-guide/ +ref: 27 +title: Visual style guide +--- diff --git a/developer/general/website-style-guide.md b/developer/general/website-style-guide.md new file mode 100644 index 0000000000..6c5a7de828 --- /dev/null +++ b/developer/general/website-style-guide.md @@ -0,0 +1,78 @@ +--- +lang: en +layout: doc +permalink: /doc/website-style-guide/ +title: Website style guide +--- + +This page explains the standards we follow for building and maintaining the +website. Please follow these guidelines and conventions when modifying the +website. For the standards governing the documentation in particular, please +see the [documentation style guide](/doc/documentation-style-guide/). + +## Coding conventions + +The following conventions apply to the website as a whole, including everything +written in HTML, CSS, YAML, and Liquid. These conventions are intended to keep +the codebase consistent when multiple collaborators are working on it. They +should be understood as a practical set of rules for maintaining order in this +specific codebase rather than as a statement of what is objectively right or +good. + +### General practices + +- Use comments to indicate the purposes of different blocks of code. This makes + the file easier to understand and navigate. + +- Use descriptive variable names. Never use one or two letter variable names. + Avoid obscure abbreviations and made-up words. + +- In general, make it easy for others to read your code. Your future self will + thank you, and so will your collaborators! + +- [Don't Repeat Yourself + (DRY)!](https://en.wikipedia.org/wiki/Don%27t_repeat_yourself) Instead of + repeating the same block of code multiple times, abstract it out into a + separate file and `include` that file where you need it. + +### Whitespace + +- Always use spaces. Never use tabs. + +- Each indentation step should be exactly two (2) spaces. + +- Whenever you add an opening tag, indent the following line. (Exception: If + you open and close the tag on the same line, do not indent the following + line.) + +- Indent Liquid the same way as HTML. + +- In general, the starting columns of every adjacent pair of lines should be no + more than two spaces apart (example below). + +- No blank or empty lines. (Hint: When you feel you need one, add a comment on + that line instead.) + +#### Indentation example + +Here's an example that follows the indentation rules: + +{% raw %} +```html + + + + {% for item in secs.htmlsections[0].columns %} + + {% endfor %} + + {% for canary in site.data.sec-canary reversed %} + + + + + + {% endfor %} +
{{ item.title }}
{{ canary.date }}Qubes Canary #{{ canary.canary }}
+``` +{% endraw %} diff --git a/developer/releases/1.0/release-notes.md b/developer/releases/1.0/release-notes.md deleted file mode 100644 index 74b125ff50..0000000000 --- a/developer/releases/1.0/release-notes.md +++ /dev/null @@ -1,56 +0,0 @@ ---- -layout: doc -title: Qubes R1.0 Release Notes -permalink: /doc/releases/1.0/release-notes/ -redirect_from: -- /en/doc/releases/1.0/release-notes/ ---- - -Qubes R1.0 Release Notes -======================== - -Detailed release notes in [this blog post](http://blog.invisiblethings.org/2012/09/03/introducing-qubes-10.html). - -Known issues ------------- - -- Installer might not support some USB keyboards (\#230). This seems to include all the Mac Book keyboards (most PC laptops have PS2 keyboards and are not affected). - -- If you don't enable Composition (System Setting -\> Desktop -\> Enable desktop effects), which you really should do, then the KDE task bar might get ugly (e.g. half of it might be black). This is some KDE bug that we don't plan to fix. - -- Some keyboard layout set by KDE System Settings can cause [keyboard not working at all](https://groups.google.com/group/qubes-devel/browse_thread/thread/77d076b65dda7226). If you hit this issue, you can switch to console (by console login option) and manually edit `/etc/X11/xorg.conf.d/00-system-setup-keyboard.conf` (and `/etc/sysconfig/keyboard`) and place correct keyboard layout settings (details in linked thread). You can check if specific keyboard layout settings are proper using `setxkbmap` tool. - -- On systems with more than 8GB of RAM there is problem with DisposableVM. To fix it, limit maximum memory allocation for DispVM to 3GB - - ~~~ - qvm-prefs -s fedora-17-x64-dvm maxmem 3072 - qvm-create-default-dvm --default-template --default-script - ~~~ - -- On some systems the KDE Window Manager might freeze upon resuming from S3 sleep when compositing is enabled (and the only method to log in to the system if this happens is to switch to a text console, enter your user's password, kill the kwin process, go back to the Xorg console, log in, and start a new instance of kwin using Konsole application :) If you experience such problems, make sure to disable compositing before putting the system into sleep by pressing Alt-Ctrl-F12 (and then enabling it back once you log in after resume) -- this way you should never see this problem again. - -Downloads ---------- - -See [Qubes Downloads](/doc/QubesDownloads/). - -Installation instructions -------------------------- - -See [Installation Guide](/doc/installation-guide/). - -Upgrading ---------- - -### From Qubes 1.0-rc1 - -If you're already running Qubes 1.0-rc1, you don't need to reinstall, it's just enough to update the packages in your Dom0 and the template VM(s). The easiest way for doing this is to click on the Update Button in the Qubes Manger -- one click when you selected Dom0, and one click for each of your template VM (by default there is just one template). - -### From Qubes 1.0 Beta 3 - -If you have Qubes Beta 3 currently installed on your system, you must reinstall from scratch, as we offer no direct upgrade option in the installer (sorry). However, we do offer tools for smooth migration of your AppVMs. In order to do that, please backup your AppVMs using the `qvm-backup` tool [as usual](/doc/backup-restore/). Then, after you install Qubes 1.0 rc1, you can restore them using `qvm-backup-restore` tool. However, because we have changed the default template in RC1, you should tell qvm-back-restore about that by passing `--replace-template` option: - -~~~ -qvm-backup-restore --replace-template=fedora-15-x64:fedora-17-x64 -~~~ - diff --git a/developer/releases/1_0/release-notes.md b/developer/releases/1_0/release-notes.md new file mode 100644 index 0000000000..39c6ea37dd --- /dev/null +++ b/developer/releases/1_0/release-notes.md @@ -0,0 +1,50 @@ +--- +lang: en +layout: doc +permalink: /doc/releases/1.0/release-notes/ +redirect_from: +- /en/doc/releases/1.0/release-notes/ +ref: 18 +title: Qubes R1.0 release notes +--- + +Detailed release notes in [this blog post](https://blog.invisiblethings.org/2012/09/03/introducing-qubes-10.html). + +## Known issues + +- Installer might not support some USB keyboards (\#230). This seems to include all the Mac Book keyboards (most PC laptops have PS2 keyboards and are not affected). + +- If you don't enable Composition (System Setting -\> Desktop -\> Enable desktop effects), which you really should do, then the KDE task bar might get ugly (e.g. half of it might be black). This is some KDE bug that we don't plan to fix. + +- Some keyboard layout set by KDE System Settings can cause [keyboard not working at all](https://groups.google.com/group/qubes-devel/browse_thread/thread/77d076b65dda7226). If you hit this issue, you can switch to console (by console login option) and manually edit `/etc/X11/xorg.conf.d/00-system-setup-keyboard.conf` (and `/etc/sysconfig/keyboard`) and place correct keyboard layout settings (details in linked thread). You can check if specific keyboard layout settings are proper using `setxkbmap` tool. + +- On systems with more than 8GB of RAM there is problem with DisposableVM. To fix it, limit maximum memory allocation for DispVM to 3GB + + ~~~ + qvm-prefs -s fedora-17-x64-dvm maxmem 3072 + qvm-create-default-dvm --default-template --default-script + ~~~ + +- On some systems the KDE Window Manager might freeze upon resuming from S3 sleep when compositing is enabled (and the only method to log in to the system if this happens is to switch to a text console, enter your user's password, kill the kwin process, go back to the Xorg console, log in, and start a new instance of kwin using Konsole application :) If you experience such problems, make sure to disable compositing before putting the system into sleep by pressing Alt-Ctrl-F12 (and then enabling it back once you log in after resume) -- this way you should never see this problem again. + +## Downloads + +See [Qubes Downloads](/doc/QubesDownloads/). + +## Installation instructions + +See [Installation Guide](/doc/installation-guide/). + +## Upgrading + +### From Qubes 1.0-rc1 + +If you're already running Qubes 1.0-rc1, you don't need to reinstall, it's just enough to update the packages in your Dom0 and the template VM(s). The easiest way for doing this is to click on the Update Button in the Qubes Manger -- one click when you selected Dom0, and one click for each of your template VM (by default there is just one template). + +### From Qubes 1.0 Beta 3 + +If you have Qubes Beta 3 currently installed on your system, you must reinstall from scratch, as we offer no direct upgrade option in the installer (sorry). However, we do offer tools for smooth migration of your AppVMs. In order to do that, please backup your AppVMs using the `qvm-backup` tool [as usual](/doc/backup-restore/). Then, after you install Qubes 1.0 rc1, you can restore them using `qvm-backup-restore` tool. However, because we have changed the default template in RC1, you should tell qvm-back-restore about that by passing `--replace-template` option: + +~~~ +qvm-backup-restore --replace-template=fedora-15-x64:fedora-17-x64 +~~~ diff --git a/developer/releases/2.0/release-notes.md b/developer/releases/2_0/release-notes.md similarity index 59% rename from developer/releases/2.0/release-notes.md rename to developer/releases/2_0/release-notes.md index c1d01f4161..e0a5751dd9 100644 --- a/developer/releases/2.0/release-notes.md +++ b/developer/releases/2_0/release-notes.md @@ -1,18 +1,16 @@ --- +lang: en layout: doc -title: Qubes R2.0 Release Notes permalink: /doc/releases/2.0/release-notes/ redirect_from: - /en/doc/releases/2.0/release-notes/ +ref: 25 +title: Qubes R2.0 release notes --- -Qubes R2.0 Release Notes -======================== +Detailed release notes in [this blog post](https://blog.invisiblethings.org/2014/09/26/announcing-qubes-os-release-2.html) -Detailed release notes in [this blog post](http://blog.invisiblethings.org/2014/09/26/announcing-qubes-os-release-2.html) - -New features since 1.0 ----------------------- +## New features since 1.0 * Support for generic fully virtualized VMs (without qemu in the TCB!) * Support for Windows-based AppVMs integration (clipboard, file exchange, qrexec, pv drivers) @@ -28,39 +26,35 @@ New features since 1.0 * Support for dynamic screen resolution change * Dom0 distribution upgraded to Fedora 20 -Known issues ------------- +## Known issues -- On some graphics cards the Xfce4 Window Manager (one of the two supported Dom0 Windows Managers in Qubes R2, the other being KDE) might behave "strangely", e.g. decorations might not be drawn sometimes. Also the accompanying lightdm login manager might incorrectly display the wallpaper. If you're facing those problems, it's advisable to use the KDE Window Manager and kdm instead of Xfce4 and lightdm (this is default if one chooses the KDE only installation option in the installer). +* On some graphics cards the Xfce4 Window Manager (one of the two supported Dom0 Windows Managers in Qubes R2, the other being KDE) might behave "strangely", e.g. decorations might not be drawn sometimes. Also the accompanying lightdm login manager might incorrectly display the wallpaper. If you're facing those problems, it's advisable to use the KDE Window Manager and kdm instead of Xfce4 and lightdm (this is default if one chooses the KDE only installation option in the installer). -- Some icons in the Qubes Manager application might not be drawn correctly when using the Xfce4 environment in Dom0. If this bothers you, please use the KDE environment instead. +* Some icons in the Qubes Manager application might not be drawn correctly when using the Xfce4 environment in Dom0. If this bothers you, please use the KDE environment instead. -- If your GPU is not correctly supported by the Dom0 kernel (e.g. the 3D desktop effects do not run smoothly) then you might experience "heaviness" with Windows 7-based AppVMs. In that case, please solve the problem with your GPU support in Dom0 in the first place (by using a different kernel), or install Qubes OS on a different system. +* If your GPU is not correctly supported by the Dom0 kernel (e.g. the 3D desktop effects do not run smoothly) then you might experience "heaviness" with Windows 7-based AppVMs. In that case, please solve the problem with your GPU support in Dom0 in the first place (by using a different kernel), or install Qubes OS on a different system. -- Under some circumstances, Qubes backup can create broken backup, without any visible message (\#902). It is advisable to verify a backup to spot the problem. If you encounter this problem, backup VM directory manually. +* Under some circumstances, Qubes backup can create broken backup, without any visible message (\#902). It is advisable to verify a backup to spot the problem. If you encounter this problem, backup VM directory manually. -- System shutdown sometimes is very slow (\#903). To mitigate the problem, shutdown all the VMs first. +* System shutdown sometimes is very slow (\#903). To mitigate the problem, shutdown all the VMs first. -- For other known issues take a look at [our trac tickets](https://wiki.qubes-os.org/query?status=accepted&status=assigned&status=new&status=reopened&type=defect&milestone=Release+2.1+(post+R2)&col=id&col=summary&col=status&col=type&col=priority&col=milestone&col=component&order=priority) +* For other known issues take a look at [our trac tickets](https://wiki.qubes-os.org/query?status=accepted&status=assigned&status=new&status=reopened&type=defect&milestone=Release+2.1+(post+R2)&col=id&col=summary&col=status&col=type&col=priority&col=milestone&col=component&order=priority) It is advised to install updates just after system installation to apply bug fixes for (some of) the above problems. -Downloads ---------- +## Downloads See [Qubes Downloads](/doc/QubesDownloads/). -Installation instructions -------------------------- +## Installation instructions See [Installation Guide](/doc/installation-guide/). -Upgrading ---------- +## Upgrading ### From Qubes R2 rc1 -Upgrading from Qubes R2 rc1 should be a simple matter of installing updates for [dom0](/doc/software-update-dom0/) and [VMs](/doc/software-update-vm/). +Upgrading from Qubes R2 rc1 should be a simple matter of installing updates for [dom0](/doc/how-to-install-software-in-dom0/) and [VMs](/doc/software-update-vm/). ### From Qubes R2 beta 3 and older diff --git a/developer/releases/3.0/release-notes.md b/developer/releases/3_0/release-notes.md similarity index 90% rename from developer/releases/3.0/release-notes.md rename to developer/releases/3_0/release-notes.md index ec3d030d0a..e31b4a276c 100644 --- a/developer/releases/3.0/release-notes.md +++ b/developer/releases/3_0/release-notes.md @@ -1,18 +1,18 @@ --- +lang: en layout: doc -title: Qubes R3.0 Release Notes permalink: /doc/releases/3.0/release-notes/ redirect_from: - /en/doc/releases/3.0/release-notes/ +ref: 19 +title: Qubes R3.0 release notes --- -Qubes R3.0 Release Notes -======================== +### Qubes R3.0 Release Notes This Qubes OS release is dedicated to the memory of Caspar Bowden. -New features since 2.0 ----------------------- +## New features since 2.0 * HAL (Hypervisor Abstraction Layer) - based on libvirt, opens a whole new possibilities of using different hypervisors. Currently Qubes OS uses Xen. @@ -27,8 +27,7 @@ New features since 2.0 templates using DispVM. * Automated tests - makes much easier to find bugs, before its even shipped to users -Known issues ------------- +## Known issues * Windows Tools: `qvm-block` does not work @@ -42,18 +41,15 @@ Known issues It is advised to install updates just after system installation to apply bug fixes for (some of) the above problems. -Downloads ---------- +## Downloads See [Qubes Downloads](/doc/QubesDownloads/). -Installation instructions -------------------------- +## Installation instructions See [Installation Guide](/doc/installation-guide/). -Upgrading ---------- +## Upgrading ### From R3.0 release candidate @@ -64,4 +60,3 @@ If you are using Qubes R3.0rc1, R3.0rc2 or R3.0rc3, just install system updates, The easiest and safest way to upgrade to Qubes R3.0 is to install it from scratch and use [qubes backup and restore tools](/doc/backup-restore/) for migrating of all of the user VMs. Users of Qubes R2 can upgrade using [experimental procedure](/doc/upgrade-to-r3.0/). - diff --git a/developer/releases/3.0/schedule.md b/developer/releases/3_0/schedule.md similarity index 80% rename from developer/releases/3.0/schedule.md rename to developer/releases/3_0/schedule.md index 403895f6d9..255ce7080a 100644 --- a/developer/releases/3.0/schedule.md +++ b/developer/releases/3_0/schedule.md @@ -1,14 +1,13 @@ --- +lang: en layout: doc -title: Qubes R3.0 Release Schedule permalink: /doc/releases/3.0/schedule/ redirect_from: - /en/doc/releases/3.0/schedule/ +ref: 20 +title: Qubes R3.0 release schedule --- -Qubes R3.0 Release Schedule -=========================== - | Date | Stage | | -----------:| ------------------------------------- | | 5 Sep 2015 | current-testing freeze before 3.0-rc3 | diff --git a/developer/releases/3.1/release-notes.md b/developer/releases/3_1/release-notes.md similarity index 72% rename from developer/releases/3.1/release-notes.md rename to developer/releases/3_1/release-notes.md index 160d1b9003..35d1bcd4b6 100644 --- a/developer/releases/3.1/release-notes.md +++ b/developer/releases/3_1/release-notes.md @@ -1,30 +1,27 @@ --- +lang: en layout: doc -title: Qubes R3.1 release notes permalink: /doc/releases/3.1/release-notes/ +ref: 16 +title: Qubes R3.1 release notes --- -Qubes R3.1 release notes -======================== +## New features since 3.0 -New features since 3.0 ----------------------- - -* Management Stack based of Salt Stack in dom0 - [documentation][salt-doc] +* Management Stack based of Salt Stack in dom0 - [documentation](/doc/salt/) * Out of the box Whonix setup * UEFI support * LIVE edition (still alpha, not part of R3.1-rc1) * Updated GPU drivers in dom0 * Colorful window application icons (instead of just colorful lock icon) -* PV Grub support ([documentation][pvgrub-doc]) -* Out of the box USB VM setup, including [handling USB mouse][input-proxy] +* PV Grub support ([documentation](/doc/managing-vm-kernels/)) +* Out of the box USB VM setup, including [handling USB mouse](https://github.com/QubesOS/qubes-app-linux-input-proxy/blob/master/README.md) * Xen upgraded to 4.6, for better hardware support (especially Skylake platform) * Improve updates proxy flexibility - especially repositories served over HTTPS -You can get detailed description in [completed github issues][github-release-notes] +You can get detailed description in [completed github issues](https://github.com/QubesOS/qubes-issues/issues?q=is%3Aissue+sort%3Aupdated-desc+milestone%3A%22Release+3.1%22+label%3Arelease-notes+is%3Aclosed) -Known issues ------------- +## Known issues * Installation image does not fit on DVD, requires either DVD DL, or USB stick (5GB or more) @@ -38,18 +35,15 @@ Known issues It is advised to install updates just after system installation to apply bug fixes for (some of) the above problems. -Downloads ---------- +## Downloads See [Qubes Downloads](/downloads/). -Installation instructions -------------------------- +## Installation instructions See [Installation Guide](/doc/installation-guide/). -Upgrading ---------- +## Upgrading ### From R3.0 @@ -69,8 +63,3 @@ for migrating of all of the user VMs. Alternatively you can [upgrade to R3.0 using](/doc/releases/3.0/release-notes/#upgrading) first, then follow the instructions above. This will be time consuming process. - -[salt-doc]: /doc/salt/ -[pvgrub-doc]: /doc/managing-vm-kernel/ -[input-proxy]: https://github.com/QubesOS/qubes-app-linux-input-proxy/blob/master/README.md -[github-release-notes]: https://github.com/QubesOS/qubes-issues/issues?q=is%3Aissue+sort%3Aupdated-desc+milestone%3A%22Release+3.1%22+label%3Arelease-notes+is%3Aclosed diff --git a/developer/releases/3.1/schedule.md b/developer/releases/3_1/schedule.md similarity index 76% rename from developer/releases/3.1/schedule.md rename to developer/releases/3_1/schedule.md index ede408ed8f..93cd69c7c0 100644 --- a/developer/releases/3.1/schedule.md +++ b/developer/releases/3_1/schedule.md @@ -1,14 +1,13 @@ --- +lang: en layout: doc -title: Qubes R3.1 Release Schedule permalink: /doc/releases/3.1/schedule/ redirect_from: - /en/doc/releases/3.1/schedule/ +ref: 17 +title: Qubes R3.1 release schedule --- -Qubes R3.1 Release Schedule -=========================== - This schedule is based on [Version Scheme](/doc/version-scheme/#release-schedule). | Date | Stage | @@ -18,4 +17,4 @@ This schedule is based on [Version Scheme](/doc/version-scheme/#release-schedule | 12 Jan 2016 | 3.1-rc2 release | | 26 Jan 2016 | decide whether 3.1-rc2 is the final 3.1 | | 9 Feb 2016 | current-testing freeze before 3.1-rc3 | -| 16 Feb 2016
23 Feb 2016 | 3.1-rc3 release | +| ~~16 Feb 2016~~
23 Feb 2016 | 3.1-rc3 release | diff --git a/developer/releases/3.2/release-notes.md b/developer/releases/3_2/release-notes.md similarity index 56% rename from developer/releases/3.2/release-notes.md rename to developer/releases/3_2/release-notes.md index 2411915387..547dff271a 100644 --- a/developer/releases/3.2/release-notes.md +++ b/developer/releases/3_2/release-notes.md @@ -1,28 +1,25 @@ --- +lang: en layout: doc -title: Qubes R3.2 release notes permalink: /doc/releases/3.2/release-notes/ +ref: 21 +title: Qubes R3.2 release notes --- -Qubes R3.2 release notes -======================== +## New features since 3.1 -New features since 3.1 ----------------------- - -* Management Stack extended to support in-VM configuration - [documentation][salt-doc] -* PV USB - [documentation][usb] +* Management Stack extended to support in-VM configuration - [documentation](/doc/salt/) +* PV USB - [documentation](/doc/usb/) * Dom0 update to Fedora 23 for better hardware support * Kernel 4.4.x * Default desktop environment switched to Xfce4 * KDE 5 support (but it is no longer the default one) -* Tiling window managers support: awesome, [i3][i3] -* More flexible Qubes RPC services - [related ticket][qrexec-argument], [documentation][qrexec-doc] +* Tiling window managers support: awesome, [i3](/doc/i3/) +* More flexible Qubes RPC services - [related ticket](https://github.com/QubesOS/qubes-issues/issues/1876), [documentation](/doc/qrexec/#service-policies-with-arguments) -You can get detailed description in [completed github issues][github-release-notes] +You can get detailed description in [completed github issues](https://github.com/QubesOS/qubes-issues/issues?q=is%3Aissue+sort%3Aupdated-desc+milestone%3A%22Release+3.2%22+label%3Arelease-notes+is%3Aclosed) -Known issues ------------- +## Known issues * [Fedora 23 reached EOL in December 2016](https://fedoraproject.org/wiki/End_of_life). There is a [manual procedure to upgrade your VMs](/news/2018/01/06/fedora-26-upgrade/). @@ -34,44 +31,31 @@ Known issues It is advised to install updates just after system installation to apply bug fixes for (some of) the above problems. -Downloads ---------- +## Downloads See [Qubes Downloads](/downloads/). -Installation instructions -------------------------- +## Installation instructions See [Installation Guide](/doc/installation-guide/). After installation, [manually upgrade to Fedora 26](/news/2018/01/06/fedora-26-upgrade/). -Upgrading ---------- +## Upgrading ### From R3.1 The easiest and safest way to upgrade to Qubes R3.2 is to install it from -scratch and use [qubes backup and restore tools][backup] for +scratch and use [qubes backup and restore tools](/doc/backup-restore/) for migrating of all of the user VMs. Users of Qubes R3.1 can also upgrade using [this -procedure][upgrade]. +procedure](/doc/upgrade-to-r3.2/). ### From R3.0 or earlier When upgrading from earlier versions the easiest and safest way is to install -it from scratch and use [qubes backup and restore tools][backup] +it from scratch and use [qubes backup and restore tools](/doc/backup-restore/) for migrating of all of the user VMs. -Alternatively you can [upgrade to R3.1 using][upgrade-r3.1] first, then follow +Alternatively you can [upgrade to R3.1 using](/doc/releases/3.1/release-notes/#upgrading) first, then follow the instructions above. This will be time consuming process. - -[salt-doc]: /doc/salt/ -[usb]: /doc/usb/ -[i3]: /doc/i3/ -[upgrade]: /doc/upgrade-to-r3.2/ -[upgrade-r3.1]: /doc/releases/3.1/release-notes/#upgrading -[backup]: /doc/backup-restore/ -[qrexec-argument]: https://github.com/QubesOS/qubes-issues/issues/1876 -[qrexec-doc]: /doc/qrexec/#service-policies-with-arguments -[github-release-notes]: https://github.com/QubesOS/qubes-issues/issues?q=is%3Aissue+sort%3Aupdated-desc+milestone%3A%22Release+3.2%22+label%3Arelease-notes+is%3Aclosed diff --git a/developer/releases/3.2/schedule.md b/developer/releases/3_2/schedule.md similarity index 62% rename from developer/releases/3.2/schedule.md rename to developer/releases/3_2/schedule.md index 05dc5599b4..36cedd2364 100644 --- a/developer/releases/3.2/schedule.md +++ b/developer/releases/3_2/schedule.md @@ -1,23 +1,22 @@ --- +lang: en layout: doc -title: Qubes R3.2 Release Schedule permalink: /doc/releases/3.2/schedule/ redirect_from: - /en/doc/releases/3.2/schedule/ +ref: 22 +title: Qubes R3.2 release schedule --- -Qubes R3.2 Release Schedule -=========================== - This schedule is based on [Version Scheme](/doc/version-scheme/#release-schedule). | Date | Stage | | -----------:| --------------------------------------- | | 18 Jun 2016 | 3.2-rc1 release | | 2 Jul 2016 | decide whether 3.2-rc1 is the final 3.2 | -| 16 Jul 2016
20 Jul 2016 | current-testing freeze before 3.2-rc2 | -| 23 Jul 2016
27 Jul 2016 | 3.2-rc2 release | -| 5 Aug 2016
9 Aug 2016 | decide whether 3.2-rc2 is the final 3.2 | +| ~~16 Jul 2016~~
20 Jul 2016 | current-testing freeze before 3.2-rc2 | +| ~~23 Jul 2016~~
27 Jul 2016 | 3.2-rc2 release | +| ~~5 Aug 2016~~
9 Aug 2016 | decide whether 3.2-rc2 is the final 3.2 | | 24 Aug 2016 | current-testing freeze before 3.2-rc3 | | 31 Aug 2016 | 3.2-rc3 release | | 29 Sep 2016 | 3.2 release | diff --git a/developer/releases/4.0/release-notes.md b/developer/releases/4.0/release-notes.md deleted file mode 100644 index 3033692423..0000000000 --- a/developer/releases/4.0/release-notes.md +++ /dev/null @@ -1,129 +0,0 @@ ---- -layout: doc -title: Qubes R4.0 release notes -permalink: /doc/releases/4.0/release-notes/ ---- - -Qubes R4.0 release notes -======================== - -New features since 3.2 ----------------------- - -* Core management scripts rewrite with better structure and extensibility, [API documentation][api-doc] -* [Admin API][admin-api] allowing strictly controlled managing from non-dom0 -* All `qvm-*` command-line tools rewritten, some options have changed -* Renaming VM directly is prohibited, there is GUI to clone under new name and remove old VM -* Use [PVH][qsb-37] and [HVM][hvm-switch] by default to [mitigate Meltdown & Spectre][qsb-37] and lower the [attack surface on Xen][qsb-24] -* Create USB VM by default -* [Multiple DisposableVMs templates support][dispvm-ticket] -* New [backup format][backup-format] using scrypt key-derivation function -* Non-encrypted backups no longer supported -* [split VM packages][packages-split], for better support minimal, specialized templates -* [Qubes Manager decomposition][manager-ticket] - domains and devices widgets instead of full Qubes Manager; devices widget support also USB -* [More flexible firewall interface][vm-interface] for ease unikernel integration -* Template VMs do not have network interface by default, [qrexec-based updates proxy][qrexec-proxy] is used instead -* More flexible IP addressing for VMs - [custom IP][custom-ip], [hidden from the IP][hide-ip] -* More flexible Qubes RPC policy - [related ticket][qrexec-policy-keywords], [documentation][qrexec-doc] -* [New Qubes RPC confirmation window][qrexec-confirm], including option to specify destination VM -* [New storage subsystem design][storage] -* Dom0 update to Fedora 25 for better hardware support -* Kernel 4.9.x - -You can get detailed description in [completed github issues][github-release-notes] - -Security Notes --------------- - -* PV VMs migrated from 3.2 to 4.0-rc4 or later are automatically set to PVH mode in order to protect against Meltdown (see [QSB #37][qsb-37]). - However, PV VMs migrated from any earlier 4.0 release candidate (RC1, RC2, or RC3) are not automatically set to PVH mode. - These must be set manually. - -* The following steps may need to be applied in dom0 and Fedora 26 TemplateVMs in order to receive updates (see [#3737]). - - Steps for dom0 updates: - - 1. Open the Qubes Menu by clicking on the "Q" icon in the top-left corner of the screen. - 2. Select `Terminal Emulator`. - 3. In the window that opens, enter this command: - - sudo nano /etc/yum.repos.d/qubes-dom0.repo - - 4. This opens the nano text editor. Change all four instances of `http` to `https`. - 5. Press `CTRL+X`, then `Y`, then `ENTER` to save changes and exit. - 6. Check for updates normally. - - Steps for Fedora 26 TemplateVM updates: - - 1. Open the Qubes Menu by clicking on the "Q" icon in the top-left corner of the screen. - 2. Select `Template: fedora-26`, then `fedora-26: Terminal`. - 3. In the window that opens, enter the command for your version: - - [Qubes 3.2] sudo gedit /etc/yum.repos.d/qubes-r3.repo - [Qubes 4.0] sudo gedit /etc/yum.repos.d/qubes-r4.repo - - 4. This opens the gedit text editor in a window. Change all four instances of `http` to `https`. - 5. Click the "Save" button in the top-right corner of the window. - 6. Close the window. - 7. Check for updates normally. - 8. Shut down the TemplateVM. - -Known issues ------------- - -* Locale using coma as decimal separator [crashes qubesd][locale-bug]. Either install with different locale (English (United States) for example), or manually apply fix explained in that issue. - -* In the middle of installation, [keyboard layout reset to US][keyboard-layout-bug]. Be careful what is the current layout while setting default user password (see upper right screen corner). - -* On some laptops (for example Librem 15v2), touchpad do not work directly after installation. Reboot the system to fix the issue. - -* List of USB devices may contain device identifiers instead of name - -* With R4.0.1, which ships kernel-4.19, you may never reach the anaconda startup and be block on an idle black screen with blinking cursor. You can try to add `plymouth.ignore-serial-consoles` in the grub installer boot menu right after `quiet rhgb`. With legacy mode, you can do it directly when booting the DVD or USB key. In UEFI mode, follow the same procedure described for [disabling](https://www.qubes-os.org/doc/uefi-troubleshooting/#installation-freezes-before-getting-to-anaconda-qubes-40) `nouveau` module (related [solved issue](https://github.com/QubesOS/qubes-issues/issues/3849) in further version of Qubes). - -* For other known issues take a look at [our tickets](https://github.com/QubesOS/qubes-issues/issues?q=is%3Aopen+is%3Aissue+milestone%3A%22Release+4.0%22+label%3Abug) - -It is advised to install updates just after system installation to apply bug fixes for (some of) the above problems. - -Downloads ---------- - -See [Qubes Downloads](/downloads/). - -Installation instructions -------------------------- - -See [Installation Guide](/doc/installation-guide/). - -Upgrading ---------- - -There is no in-place upgrade path from earlier Qubes versions. The only -supported option to upgrade to Qubes R4.0 is to install it from scratch and use -[qubes backup and restore tools][backup] for migrating of all of the user VMs. -We also provide [detailed instruction][upgrade-to-r4.0] for this procedure. - - -[backup]: /doc/backup-restore/ -[github-release-notes]: https://github.com/QubesOS/qubes-issues/issues?q=is%3Aissue+sort%3Aupdated-desc+milestone%3A%22Release+4.0%22+label%3Arelease-notes+is%3Aclosed -[custom-ip]: https://github.com/QubesOS/qubes-issues/issues/1477 -[hide-ip]: https://github.com/QubesOS/qubes-issues/issues/1143 -[packages-split]: https://github.com/QubesOS/qubes-issues/issues/2771 -[hvm-switch]: https://github.com/QubesOS/qubes-issues/issues/2185 -[manager-ticket]: https://github.com/QubesOS/qubes-issues/issues/2132 -[dispvm-ticket]: https://github.com/QubesOS/qubes-issues/issues/2253 -[qrexec-proxy]: https://github.com/QubesOS/qubes-issues/issues/1854 -[qrexec-policy-keywords]: https://github.com/QubesOS/qubes-issues/issues/865 -[qrexec-confirm]: https://github.com/QubesOS/qubes-issues/issues/910 -[qrexec-doc]: /doc/qrexec/#specifying-vms-tags-types-targets-etc -[storage]: https://github.com/QubesOS/qubes-issues/issues/1842 -[vm-interface]: /doc/vm-interface/ -[admin-api]: /news/2017/06/27/qubes-admin-api/ -[qsb-24]: https://github.com/QubesOS/qubes-secpack/blob/master/QSBs/qsb-024-2016.txt -[qsb-37]: https://github.com/QubesOS/qubes-secpack/blob/master/QSBs/qsb-037-2018.txt -[backup-format]: /doc/backup-emergency-restore-v4/ -[api-doc]: https://dev.qubes-os.org/projects/qubes-core-admin/en/latest/ -[upgrade-to-r4.0]: /doc/upgrade-to-r4.0/ -[locale-bug]: https://github.com/QubesOS/qubes-issues/issues/3753 -[keyboard-layout-bug]: https://github.com/QubesOS/qubes-issues/issues/3352 -[#3737]: https://github.com/QubesOS/qubes-issues/issues/3737 diff --git a/developer/releases/4_0/release-notes.md b/developer/releases/4_0/release-notes.md new file mode 100644 index 0000000000..b80a1959d6 --- /dev/null +++ b/developer/releases/4_0/release-notes.md @@ -0,0 +1,107 @@ +--- +lang: en +layout: doc +permalink: /doc/releases/4.0/release-notes/ +ref: 23 +title: Qubes R4.0 release notes +--- + +New features since 3.2 +---------------------- + +* Core management scripts rewrite with better structure and extensibility, [API documentation](https://dev.qubes-os.org/projects/qubes-core-admin/en/latest/) +* [Admin API](/news/2017/06/27/qubes-admin-api/) allowing strictly controlled managing from non-dom0 +* All `qvm-*` command-line tools rewritten, some options have changed +* Renaming VM directly is prohibited, there is GUI to clone under new name and remove old VM +* Use [PVH](https://github.com/QubesOS/qubes-secpack/blob/master/QSBs/qsb-037-2018.txt) and [HVM](https://github.com/QubesOS/qubes-issues/issues/2185) by default to [mitigate Meltdown & Spectre](https://github.com/QubesOS/qubes-secpack/blob/master/QSBs/qsb-037-2018.txt) and lower the [attack surface on Xen](https://github.com/QubesOS/qubes-secpack/blob/master/QSBs/qsb-024-2016.txt) +* Create USB VM by default +* [Multiple DisposableVMs templates support](https://github.com/QubesOS/qubes-issues/issues/2253) +* New [backup format](/doc/backup-emergency-restore-v4/) using scrypt key-derivation function +* Non-encrypted backups no longer supported +* [split VM packages](https://github.com/QubesOS/qubes-issues/issues/2771), for better support minimal, specialized templates +* [Qubes Manager decomposition](https://github.com/QubesOS/qubes-issues/issues/2132) - domains and devices widgets instead of full Qubes Manager; devices widget support also USB +* [More flexible firewall interface](/doc/vm-interface/) for ease unikernel integration +* Template VMs do not have network interface by default, [qrexec-based updates proxy](https://github.com/QubesOS/qubes-issues/issues/1854) is used instead +* More flexible IP addressing for VMs - [custom IP](https://github.com/QubesOS/qubes-issues/issues/1477), [hidden from the IP](https://github.com/QubesOS/qubes-issues/issues/1143) +* More flexible Qubes RPC policy - [related ticket](https://github.com/QubesOS/qubes-issues/issues/865), [documentation](/doc/qrexec/#specifying-vms-tags-types-targets-etc) +* [New Qubes RPC confirmation window](https://github.com/QubesOS/qubes-issues/issues/910), including option to specify destination VM +* [New storage subsystem design](https://github.com/QubesOS/qubes-issues/issues/1842) +* Dom0 update to Fedora 25 for better hardware support +* Kernel 4.9.x + +You can get detailed description in [completed github issues](https://github.com/QubesOS/qubes-issues/issues?q=is%3Aissue+sort%3Aupdated-desc+milestone%3A%22Release+4.0%22+label%3Arelease-notes+is%3Aclosed) + +Security Notes +-------------- + +* PV VMs migrated from 3.2 to 4.0-rc4 or later are automatically set to PVH mode in order to protect against Meltdown (see [QSB #37](https://github.com/QubesOS/qubes-secpack/blob/master/QSBs/qsb-037-2018.txt)). + However, PV VMs migrated from any earlier 4.0 release candidate (RC1, RC2, or RC3) are not automatically set to PVH mode. + These must be set manually. + +* The following steps may need to be applied in dom0 and Fedora 26 TemplateVMs in order to receive updates (see [#3737](https://github.com/QubesOS/qubes-issues/issues/3737)). + + Steps for dom0 updates: + + 1. Open the Qubes Menu by clicking on the "Q" icon in the top-left corner of the screen. + 2. Select `Terminal Emulator`. + 3. In the window that opens, enter this command: + + ``` + sudo nano /etc/yum.repos.d/qubes-dom0.repo + ``` + + 4. This opens the nano text editor. Change all four instances of `http` to `https`. + 5. Press `CTRL+X`, then `Y`, then `ENTER` to save changes and exit. + 6. Check for updates normally. + + Steps for Fedora 26 TemplateVM updates: + + 1. Open the Qubes Menu by clicking on the "Q" icon in the top-left corner of the screen. + 2. Select `Template: fedora-26`, then `fedora-26: Terminal`. + 3. In the window that opens, enter the command for your version: + + ``` + [Qubes 3.2] sudo gedit /etc/yum.repos.d/qubes-r3.repo + [Qubes 4.0] sudo gedit /etc/yum.repos.d/qubes-r4.repo + ``` + + 4. This opens the gedit text editor in a window. Change all four instances of `http` to `https`. + 5. Click the "Save" button in the top-right corner of the window. + 6. Close the window. + 7. Check for updates normally. + 8. Shut down the TemplateVM. + +Known issues +------------ + +* Locale using coma as decimal separator [crashes qubesd](https://github.com/QubesOS/qubes-issues/issues/3753). Either install with different locale (English (United States) for example), or manually apply fix explained in that issue. + +* In the middle of installation, [keyboard layout reset to US](https://github.com/QubesOS/qubes-issues/issues/3352). Be careful what is the current layout while setting default user password (see upper right screen corner). + +* On some laptops (for example Librem 15v2), touchpad do not work directly after installation. Reboot the system to fix the issue. + +* List of USB devices may contain device identifiers instead of name + +* With R4.0.1, which ships kernel-4.19, you may never reach the anaconda startup and be block on an idle black screen with blinking cursor. You can try to add `plymouth.ignore-serial-consoles` in the grub installer boot menu right after `quiet rhgb`. With legacy mode, you can do it directly when booting the DVD or USB key. In UEFI mode, follow the same procedure described for [disabling](/doc/uefi-troubleshooting/#installation-freezes-before-displaying-installer) `nouveau` module (related [solved issue](https://github.com/QubesOS/qubes-issues/issues/3849) in further version of Qubes). + +* For other known issues take a look at [our tickets](https://github.com/QubesOS/qubes-issues/issues?q=is%3Aopen+is%3Aissue+milestone%3A%22Release+4.0%22+label%3Abug) + +It is advised to install updates just after system installation to apply bug fixes for (some of) the above problems. + +Downloads +--------- + +See [Qubes Downloads](/downloads/). + +Installation instructions +------------------------- + +See [Installation Guide](/doc/installation-guide/). + +Upgrading +--------- + +There is no in-place upgrade path from earlier Qubes versions. The only +supported option to upgrade to Qubes R4.0 is to install it from scratch and use +[qubes backup and restore tools](/doc/backup-restore/) for migrating of all of the user VMs. +We also provide [detailed instruction](/doc/upgrade-to-r4.0/) for this procedure. diff --git a/developer/releases/4.0/schedule.md b/developer/releases/4_0/schedule.md similarity index 54% rename from developer/releases/4.0/schedule.md rename to developer/releases/4_0/schedule.md index ca2a3c5ce5..6268a9999b 100644 --- a/developer/releases/4.0/schedule.md +++ b/developer/releases/4_0/schedule.md @@ -1,28 +1,27 @@ --- +lang: en layout: doc -title: Qubes R4.0 Release Schedule permalink: /doc/releases/4.0/schedule/ redirect_from: - /en/doc/releases/4.0/schedule/ +ref: 24 +title: Qubes R4.0 release schedule --- -Qubes R4.0 Release Schedule -=========================== - This schedule is based on [Version Scheme](/doc/version-scheme/#release-schedule). | Date | Stage | | -----------:| --------------------------------------- | | 31 Jul 2017 | 4.0-rc1 release | -| 28 Aug 2017
11 Sep 2017
9 Oct 2017
18 Oct 2017 | current-testing freeze before 4.0-rc2 | -| 4 Sep 2017
18 Sep 2017
16 Oct 2017
23 Oct 2017 | 4.0-rc2 release | -| 6 Nov 2017 | decide whether 4.0-rc2 is the final 4.0 | +| ~~28 Aug 2017~~
~~11 Sep 2017~~
~~9 Oct 2017~~
18 Oct 2017 | current-testing freeze before 4.0-rc2 | +| ~~4 Sep 2017~~
~~18 Sep 2017~~
~~16 Oct 2017~~
23 Oct 2017 | 4.0-rc2 release | +| 6 Nov 2017 | decide whether 4.0-rc2 is the final 4 | 20 Nov 2017 | current-testing freeze before 4.0-rc3 | | 27 Nov 2017 | 4.0-rc3 release | | 11 Dec 2017 | decide whether 4.0-rc3 is the final 4.0 | | 1 Jan 2018 | current-testing freeze before 4.0-rc4 | -| 8 Jan 2018
31 Jan 2018 | 4.0-rc4 release | -| 22 Jan 2018
14 Feb 2018 | decide whether 4.0-rc4 is the final 4.0 | +| ~~8 Jan 2018~~
31 Jan 2018 | 4.0-rc4 release | +| ~~22 Jan 2018~~
14 Feb 2018 | decide whether 4.0-rc4 is the final 4.0 | | 27 Feb 2018 | current-testing freeze before 4.0-rc5 | | 6 Mar 2018 | 4.0-rc5 release | | 20 Mar 2018 | decide whether 4.0-rc5 is the final 4.0 | diff --git a/developer/releases/4_1/release-notes.md b/developer/releases/4_1/release-notes.md new file mode 100644 index 0000000000..8ee3684633 --- /dev/null +++ b/developer/releases/4_1/release-notes.md @@ -0,0 +1,88 @@ +--- +layout: doc +title: Qubes OS 4.1 release notes +permalink: /doc/releases/4.1/release-notes/ +--- + +## New features and improvements since Qubes 4.0 + +- Optional qubes-remote-support package now available from repositories + (strictly opt-in, no package installed by default; no new ports or network + connections open by default; requires explicit connection initiation by the + user, then requires sharing a code word with the remote party before a + connection can be established; see + [#6364](https://github.com/QubesOS/qubes-issues/issues/6364) for more + information) +- Qubes firewall reworked to be more defensive (see + [#5540](https://github.com/QubesOS/qubes-issues/issues/5540) for details) +- Xen upgraded to version 4.14 +- Dom0 operating system upgraded to Fedora 32 +- Default desktop environment upgraded to Xfce 4.14 +- Upgraded default template releases +- Experimental support for GUI running outside of dom0 (hybrid mode GUI domain + without real GPU passthrough; see + [#5662](https://github.com/QubesOS/qubes-issues/issues/5662) for details) +- Experimental support for audio server running outside of dom0 ("Audio domain") +- sys-firewall and sys-usb are now disposables by default +- UEFI boot now loads GRUB, which in turn loads Xen, making the boot path + similar to legacy boot and allowing the user to modify boot parameters or + choose an alternate boot menu entry +- New qrexec policy format (see + [#4370](https://github.com/QubesOS/qubes-issues/issues/4370) for details) +- qrexec protocol improvements (see + [#4909](https://github.com/QubesOS/qubes-issues/issues/4909) for details) +- New qrexec-policy daemon +- Simplified using in-qube kernels +- Clarified disposable-related terminology and properties +- Default kernelopts can now be specified by a kernel package +- Improved support for high-resolution displays +- Improved notifications when a system drive runs out of free space +- Support for different cursor shapes +- "Paranoid mode" backup restore option now properly supported using + disposables +- Users can now choose between Debian and Fedora in the installer +- Certain files and applications are now opened in disposables, e.g., + Thunderbird email attachments +- New graphical interface for managing testing repository updates +- New "Cute Qube" icon family (replaces padlock icons) +- Disposable qube types now use the disposable icon +- New Template Manager tool for installing, removing, and updating templates + (meanwhile, the tool previously known as the "Template Manager," which was + for mass template switching, has been integrated into the Qube Manager) +- The "file" storage driver has been deprecated in Qubes 4.1 and will be + removed in Qubes 4.2 +- `property-del` event renamed to `property-reset` to avoid confusion +- qrexec no longer supports non-executable files in `/etc/qubes-rpc` +- qrexec components have been reorganized into the core-qrexec repository +- The `qvm-pool` argument parser has been rewritten and improved +- Removed the need for the out-of-tree u2mfn kernel module +- Qrexec services can now run as a socket server +- Improved template distribution mechanism +- Now possible to restart qrexec-agent +- The term "VM" has largely been replaced by "qube" +- GUI daemon is now configured using `qvm-features` tool, + `/etc/qubes/guid.conf` file is no longer used +- `qvm-run` tool got `--no-shell` option to run a single command without using + a shell inside the qube + +For a full list, including more detailed descriptions, please see +[here](https://github.com/QubesOS/qubes-issues/issues?q=is%3Aissue+sort%3Aupdated-desc+milestone%3A%22Release+4.1%22+label%3A%22release+notes%22+is%3Aclosed). + +## Known issues + +For a full list of known 4.1 issues with open bug reports, please see +[here](https://github.com/QubesOS/qubes-issues/issues?q=is%3Aopen+is%3Aissue+milestone%3A%22Release+4.1%22+label%3A%22T%3A+bug%22). +We strongly recommend [updating Qubes OS](/doc/how-to-update/) immediately +after installation in order to apply any and all available bug fixes. + +## Download + +See [downloads](/downloads/). + +## Installation instructions + +See the [installation guide](/doc/installation-guide/). + +## Upgrading + +Please see [how to upgrade to Qubes 4.1](/doc/upgrade/4.1/). diff --git a/developer/releases/4_1/schedule.md b/developer/releases/4_1/schedule.md new file mode 100644 index 0000000000..aaf3358c89 --- /dev/null +++ b/developer/releases/4_1/schedule.md @@ -0,0 +1,24 @@ +--- +lang: en +layout: doc +permalink: /doc/releases/4.1/schedule/ +redirect_from: +- /en/doc/releases/4.1/schedule/ +title: Qubes R4.1 release schedule +--- + +The table below is based on our [release schedule +policy](/doc/version-scheme/#release-schedule). + +| Date | Stage | +| ----------:| ----------------------------------------- | +| 2021-10-11 | 4.1.0-rc1 release | +| 2021-11-08 | current-testing freeze before 4.1.0-rc2 | +| 2021-11-15 | 4.1.0-rc2 release | +| 2021-11-29 | decide whether 4.1.0-rc2 is the final 4.1 | +| 2021-12-13 | current-testing freeze before 4.1.0-rc3 | +| 2021-12-20 | 4.1.0-rc3 release | +| 2022-01-03 | decide whether 4.1.0-rc3 is the final 4.1 | +| 2022-01-11 | current-testing freeze before 4.1.0-rc4 | +| 2022-01-18 | 4.1.0-rc4 release | +| 2022-01-31 | decide whether 4.1.0-rc4 is the final 4.1 | diff --git a/developer/releases/notes.md b/developer/releases/notes.md index cb1119f9db..1be2115db3 100644 --- a/developer/releases/notes.md +++ b/developer/releases/notes.md @@ -1,16 +1,15 @@ --- +lang: en layout: doc -title: Release Notes permalink: /doc/releases/notes/ +ref: 13 +title: Release notes --- -Release Notes -============= - - * [Qubes R1.0 release notes](/doc/releases/1.0/release-notes/) - * [Qubes R2.0 release notes](/doc/releases/2.0/release-notes/) - * [Qubes R3.0 release notes](/doc/releases/3.0/release-notes/) - * [Qubes R3.1 release notes](/doc/releases/3.1/release-notes/) - * [Qubes R3.2 release notes](/doc/releases/3.2/release-notes/) - * [Qubes R4.0 release notes](/doc/releases/4.0/release-notes/) - +* [Qubes R1.0 release notes](/doc/releases/1.0/release-notes/) +* [Qubes R2.0 release notes](/doc/releases/2.0/release-notes/) +* [Qubes R3.0 release notes](/doc/releases/3.0/release-notes/) +* [Qubes R3.1 release notes](/doc/releases/3.1/release-notes/) +* [Qubes R3.2 release notes](/doc/releases/3.2/release-notes/) +* [Qubes R4.0 release notes](/doc/releases/4.0/release-notes/) +* [Qubes R4.1 release notes](/doc/releases/4.1/release-notes/) diff --git a/developer/releases/schedules.md b/developer/releases/schedules.md index 50c6824cdb..6f656cf161 100644 --- a/developer/releases/schedules.md +++ b/developer/releases/schedules.md @@ -1,14 +1,13 @@ --- +lang: en layout: doc -title: Release Schedules permalink: /doc/releases/schedules/ +ref: 15 +title: Release schedules --- -Release Schedules -================= - - * [Qubes R3.0 release schedule](/doc/releases/3.0/schedule/) - * [Qubes R3.1 release schedule](/doc/releases/3.1/schedule/) - * [Qubes R3.2 release schedule](/doc/releases/3.2/schedule/) - * [Qubes R4.0 release schedule](/doc/releases/4.0/schedule/) - +* [Qubes R3.0 release schedule](/doc/releases/3.0/schedule/) +* [Qubes R3.1 release schedule](/doc/releases/3.1/schedule/) +* [Qubes R3.2 release schedule](/doc/releases/3.2/schedule/) +* [Qubes R4.0 release schedule](/doc/releases/4.0/schedule/) +* [Qubes R4.1 release schedule](/doc/releases/4.1/schedule/) diff --git a/developer/releases/todo.md b/developer/releases/todo.md index e9a9539989..23d370edb2 100644 --- a/developer/releases/todo.md +++ b/developer/releases/todo.md @@ -1,18 +1,18 @@ --- +lang: en layout: doc -title: Release Checklist permalink: /doc/releases/todo/ redirect_from: - /en/doc/releases/todo/ +ref: 14 +title: Release checklist --- -Release Checklist -================= - *the checklist is probably unfinished* On -rc1 ------- + * write schedule * create package repositories (linux-yum, linux-deb) * update repository definition (core-agent-linux, installer-qubes-os/qubes-release) @@ -24,6 +24,7 @@ On -rc1 On subsequent -rc ----------------- + * push packages to `current` * update release notes * build ISO and push to mirrors @@ -31,6 +32,7 @@ On subsequent -rc On final release ---------------- + * push packages to `current` * finish release notes * update InstallationInstructions diff --git a/developer/releases/version-scheme.md b/developer/releases/version-scheme.md new file mode 100644 index 0000000000..de2616511b --- /dev/null +++ b/developer/releases/version-scheme.md @@ -0,0 +1,178 @@ +--- +lang: en +layout: doc +permalink: /doc/version-scheme/ +redirect_from: +- /en/doc/version-scheme/ +- /doc/VersionScheme/ +- /wiki/VersionScheme/ +ref: 151 +title: Version scheme +--- + +The Qubes OS Project uses the [semantic versioning](https://semver.org/) +standard. Version numbers are written as `..`. When +`` is zero, it is often omitted as a matter of convention, e.g., `4.1` is +short for `4.1.0`. + +In general, patch releases are for backward-compatible bug fixes, minor +releases are for backward-compatible enhancements and new features, and major +release are for any backward-incompatible changes. This means that, in general, +one should *not* try to introduce features or enhancements in patch releases or +any backward-incompatible changes in patch or minor releases. (Templates are a +notable exception, as upstream OSes almost always have their own release +schedules.) Bug fixes are allowed in all releases, and backward-compatible +changes are allowed in all major and minor releases. + +Following standard practice, **version** refers to any build that has been +assigned a version name or number, e.g., `3.2-rc2`, `4.0.4`, `4.1-beta1`. By +contrast, **release** refers to any version that is intended for consumption by +the general userbase. For example, `4.0.4` was both a **version** and a +**release**, since it was stable and intended for general public use, while +`4.1-beta1` was a **version** but *not* a **release**, since it was not stable +and was intended only for [testing](/doc/testing/). All releases are +versions, but not all versions are releases. + +The letter **R**, as in `R4.1`, stands for **release**. The abbreviation **RC**, +as in `3.2-rc2`, stands for **release candidate**. + +## Qubes distributions and products + +We intend to make it easy to make a remix of Qubes, targeting another +hypervisor or isolation provider. We may also create commercial products +intended for specific circumstances. There is one distinguished distribution +called **Qubes OS**. All source code for it is available for download under a +[free and open-source license](/doc/license/) and is openly developed on +[GitHub](https://github.com/QubesOS) and our [mailing +lists](https://www.qubes-os.org/support/). The rest of this document discusses +Qubes OS. Another remix may have its own version series. + +## Release versioning + +Qubes OS as a whole is released from time to time. When preparing a new +release, we decide on the `.` numbers (e.g., `3.0`). We then +publish the first release candidate, `3.0-rc1`. When we feel that enough +progress has been made, we'll release `3.0-rc2` and so on. All these versions +(not yet releases) are considered unstable and not for production use. You are +welcome to [help us test](/doc/testing/) these versions. + +When enough progress has been made, we announce the first stable release, e.g. +`3.0.0`. This not only a version but an actual release. It is considered stable +and we commit to supporting it according to our [support +schedule](/doc/supported-releases/). Core components are branched at this +moment and bug fixes are backported from the master branch. Please see [help, +support, mailing lists, and forum](/support/) for places to ask questions about +stable releases. No major features and interface incompatibilities are to be +included in this release. We release bug fixes as patch releases (`3.0.1`, +`3.0.2`, and so on), while backward-compatible enhancements and new features +are introduced in the next minor release (e.g., `3.1`). Any +backward-incompatible changes are introduced in the next major release (e.g., +`4.0`). + +Issues in our [issue tracker](/doc/issue-tracking/) are sorted by release +[milestones](/doc/issue-tracking/#milestones). + +## Release schedule + +There is no specific schedule for releases other that more general roadmap. +When time comes, Supreme Committee declares feature freeze and tags `-rc1` and +releases ISO image. From this time on, no new features are accepted. Also a +strict time schedule kicks in. + +Each release candidate period is as follows. For the first two weeks we accept +and assign bug reports to be fixed before next release candidate. For the next +two weeks we generally focus on fixing assigned bug reports, so issues +discovered during this time may be postponed until later RC. Finally after that +there is one week of current-testing freeze, during which time no new packages +are released, in hope that they will be installed by wider user base and +tested. + +The next RC is released five weeks after the former. All packets are published +in `current` repository and the cycle starts over. There should be no less than +1 and no more than 3 release candidates before final release. + +| Stage | Duration | +| ------------------------ | --------- | +| initial testing | two weeks | +| bug fixing | two weeks | +| `current-testing` freeze | one week | + +Starting with second cycle (that is, after `-rc1`) two weeks into the cycle +(after primary bug-reporting period) the Supreme Committee decides whether +there should be another RC. If, based on remaining issues, the Committee +decides to release final, then the Committee agrees upon the release date, +which should be no later than a week after. + +[![Release cycle](/attachment/doc/release-cycle.svg)](/attachment/doc/release-cycle.svg) + +## Bug priorities + +When deciding whether the current release candidate is the final one, the +Committee takes bug [priorities](/doc/issue-tracking/#priority) into +consideration. The meaning of them is as follows: + +- `blocker` --- when any such bug is present in the current release candidate, + it can't be considered final release. Bugs with this priority must be fixed + before the next release candidate, even if that means delaying its release + (which should be considered only last resort option). + +- `critical` --- when any such bug is present in the current release candidate, + it can't be considered final release. But such bugs are not qualified to + delay next release candidate release. + +- `major` --- existence of such bugs do not strictly prevent the current + release candidate be considered final (but of course we should try hard to + not have them there). Fixing bugs of this priority can be delayed and + qualified as updates to the final stable release. + +- `default` and `minor` --- existence of such bugs do not prevent the current + release candidate be considered final. Fixing such bugs can be delayed to the + next Qubes OS release. Eventually such fixes might be backported as an update + to the stable release(s). (`default` should really be assigned a more + specific priority, but in practice there are too many issues and not enough + time, so `default` ends up staying on many issues.) + +All above is about bugs, no features should be assigned to the current release +after first `-rc`. Supreme Committee is free to adjust priorities +appropriately. + +## Component version + +Qubes release is defined as specific versions of components, which are +developed more or less separately. Their versions are composed of major and +minor version of target Qubes OS release followed by third component which is +just incremented. There is no apparent indication that given version is stable +or not. + +There are some non-essential components like `qubes-apps-*` that are shared +between releases. Their versions indicate oldest qubes-release that is +supported. We try hard to support multiple releases by one branch to ease code +maintenance. + +Different Qubes releases remixes may comprise of different components and +version are not guaranteed to be monotonic between releases. We may decide that +for newer release some component should be downgraded. There is no guarantee +that arbitrary combination of different versions of random components will +yield usable (or even install-able) compilation. + +## Git tags and branches + +We mark each component version in the repository by tag containing +`v`. Likewise, each Qubes OS release is marked by `R` tag. + +At the release of some release we create branches named like `release2`. Only +bug fixes and compatible improvements are backported to these branches. These +branches should compile. All new development is done in `master` branch. This +branch is totally unsupported and may not even compile depending on maintainer +of repository. + +All version and release tags should be made and signed by someone from ITL +staff. Public keys are included in `qubes-builder` and available at +. + +## Check installed version + +If you want to know which version you are running, for example to report an +issue, you can either check in the Qubes Manager menu under `About > Qubes OS` +or in the file `/etc/qubes-release` in dom0. For the latter you can use a +command like `cat /etc/qubes-release` in a dom0 terminal. diff --git a/developer/services/admin-api-table.md b/developer/services/admin-api-table.md new file mode 100644 index 0000000000..ad60fef91b --- /dev/null +++ b/developer/services/admin-api-table.md @@ -0,0 +1,11 @@ +--- +lang: en +layout: fullscreen +permalink: /doc/admin-api/table/ +ref: 249 +title: Admin API table +--- + +This page displays the fullscreen table from [Admin API](/doc/admin-api/). + +{% include admin-api-table.md %} diff --git a/developer/services/admin-api.md b/developer/services/admin-api.md index 6580f06236..0367828226 100644 --- a/developer/services/admin-api.md +++ b/developer/services/admin-api.md @@ -1,15 +1,19 @@ --- -layout: doc-full -title: Admin API +lang: en +layout: doc permalink: /doc/admin-api/ redirect_from: +- /doc/qubes-admin-api/ - /doc/mgmt/ - /doc/mgmt1/ - /doc/mgmt-architecture/ - /doc/admin-api-architecture/ +ref: 36 +title: Admin API --- -# Qubes OS Admin API +_You may also be interested in the article +[Introducing the Qubes Admin API](/news/2017/06/27/qubes-admin-api/)._ ## Goals @@ -36,7 +40,7 @@ TBD ## Components -![Admin API Architecture][admin-api-architecture] +![Admin API Architecture](/attachment/doc/admin-api-architecture.svg) A central entity in the Qubes Admin API system is a `qubesd` daemon, which holds information about all domains in the system and mediates all actions (like @@ -49,122 +53,38 @@ The `qubesd` daemon may be accessed from other domains through a set of qrexec API calls called the "Admin API". This API is the intended management interface supported by the Qubes OS. The API is stable. When called, the RPC handler performs basic validation and forwards the request to the -`qubesd` via UNIX domain socket. The socket API is private and unstable. It is -documented [FIXME currently it isn't -- woju 20161221] in the developer's -documentation of the source code. +`qubesd` via UNIX domain socket. The socket API is private, unstable, and not +yet documented. ## The calls The API should be implemented as a set of qrexec calls. This is to make it easy to set the policy using current mechanism. -| call | dest | argument | inside | return | note | -| ------------------------------------- | --------- | --------- | ----------------------------------------- | --------------------------------------------------------- | ---- | -| `admin.vmclass.List` | `dom0` | - | - | `\n` | -| `admin.vm.List` | `dom0|` | - | - | ` class= state=\n` | -| `admin.vm.Create.` | `dom0` | template | `name= label=