Backup development

[flurfis]

Summary

A backup and restore feature was added. The feature covers schema information and data entries. A full backup of all data is created.

Changes

The backup and restore implementation is mainly isolated from the rest of the code, so apart from the new features, no significant changes to the existing architecture were made.

---

Features

• Ability to create and re-insert backups via the UI (all data will be collected, except the default entities)
• The backup collects all schema information via the snapshot from the catalog, and saves them in an object
  (views, materialized views, indexes and storage configurations are not included)
• The backup collects data entries from the whole database via queries. The data is transformed into JSON format and written to backup files.
• The schema information and the data entries are inserted into the database via queries.

---

Tests

Apart from a test that evaluates how the backup and restore functionality performs with different batch sizes and different data volumes, no tests were implemented.

---

ToDo

• Datatype: Arrays and default arrays are not yet correctly implemented for all data models.
• Videos, audios and images are not implemented/tested for any data model.
• Views, materialized views, indexes and storage configurations for all data models should be added (is expansion of existing architecture).
• Graph model: the insertion of the nodes and edges should be reworked, because if a graph was already backed up once, it cannot be inserted a second time. The graph insertion uses "_id" labels to identify the elements. If a graph already contains a "_id" property, the insertion cannot happen.
• The UI should be expanded that the user can choose what elements to backup/insert.
• The schema information should also be saved with the entry data. Right now it is saved together with the catalog, and cannot be copied and moved together with the entry data.
• Create a designated serializer/deserializer for the backup and restore functionality.
• Test the correctness of the process.
• Only one backup can exist at once. Implement that concurrent backup versions are possible.
• Implement a checksum system that validates the backup data.
• Check if element already exists and if it does, ask the user how to proceed (i.e. overwrite the element in the database or rename the either the element from the backup or the element in the database).
• No access restrictions or owner information is collected or inserted.