Skip to content

Commit 5e1d0ea

Browse files
authored
[feat] database migration documentation (#121)
* db migration instructions * finishes documentation
1 parent 0797896 commit 5e1d0ea

File tree

3 files changed

+55
-0
lines changed

3 files changed

+55
-0
lines changed

docs/admin-manual/db-migrations.md

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
# DB Migration
2+
3+
This document describes the migration of databases, that are used by {{extra.project}}.
4+
5+
## Migrate from SQLite to PostgreSQL
6+
7+
This scenario assumes, that you have a working installation, which is using a [SQLite database](/3.5/settings/database/?h=db.sqlite3#database__url) setup. For migration, you'll need a PostgreSQL database.
8+
9+
!!! Note
10+
11+
The automatically generated table setup needs to be retained, as SQLite uses different types (e.g. string for IDs, instead of uuid).
12+
13+
### Prepare PostgreSQL
14+
15+
1. Start PostgreSQL 16.1
16+
2. Configure {{extra.project}} to use PostgreSQL instead of SQLite. Change the [`DATABASE__URL` environment variable](/3.5/settings/database/#database__url).
17+
3. (Re)start Papermerge. The startup process will create initiate the database, create all tables with the correct data types for the columns (e.g. uuid instead of text for IDs) and add a default user (which we'll remove again in the following steps).
18+
4. Stop the PostgreSQL database and {{extra.project}}.
19+
5. The automatically generated admin users needs to be delete again, as it has a random ID and would break the previously used user id. All users will be imported again with the correct IDs.
20+
- `DELETE FROM users`
21+
22+
### Import data
23+
24+
!!! Note
25+
26+
Backup your Papermerge instance, incl. the database first!
27+
28+
Migrating data from SQLite to PostgreSQL fortunately is an easy task using e.g. [pgloader](https://pgloader.readthedocs.io/en/latest/index.html). The migration is pretty much straigh forward. We need to point out, that the data types differ for some columns in the two databases. They are compatible, but different (text vs. uuid). By default, pgloader would delete tables and recreate them with the data types that it finds in SQLite. We want to prevent that.
29+
30+
Start by creating a configuration file `db.load` for pgloader, which defines source and destination databases, as well as migration options.
31+
32+
```
33+
LOAD database
34+
FROM sqlite:///<sourcefolder>/db.sqlite3
35+
INTO postgresql://papermerge@localhost:5432/pmgdb
36+
37+
WITH include no drop, create no tables, create no indexes, reset sequences, data only
38+
39+
SET work_mem to '16MB', maintenance_work_mem to '512 MB';
40+
```
41+
42+
1. Install pgloader
43+
- `sudo apt update`
44+
- `sudo apt install pgloader`
45+
2. Import the data
46+
- `pgloader db.load`
47+
3. Start PostgreSQL
48+
4. Start {{extra.project}}
49+
50+
Your {{extra.project}} instance is now using another database, nothing else changed.

docs/admin-manual/version-matrix.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,10 @@
22

33
The individual {{extra.project}} services, can be executed in Docker containers. To retain version compatibility, use only the listed image tags for {{extra.project}} versions.
44

5+
!!! Note
6+
7+
Papermerge 3.6 will not support SQLite databases anymore. Please [migrate to PostgreSQL](/admin-manual/db-migrations/#migrate-from-sqlite-to-postgresql).
8+
59
## {{extra.project}} Versions
610

711
The listed versions have been tested. Other combinations might work as well, but can cause side effects, that are not supported by {{extra.project}}.

mkdocs.yml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,7 @@ nav:
3434
- "admin-manual/overview.md"
3535
- "admin-manual/version-matrix.md"
3636
- "admin-manual/upgrade-instructions.md"
37+
- "admin-manual/db-migrations.md"
3738
- Developer Manual:
3839
- "developer-manual/overview.md"
3940
- "developer-manual/architecture.md"

0 commit comments

Comments
 (0)