feat: implement apps and app store communication (#454)

* chore: create app store repository

* chore: create app store service

* chore: implement app store description

* chore: implement app store factory functionality

* chore: fix variable name

* chore: delete app representation

* chore: add attributes from resource, representation and artifact to app
 entity

chore: add app desc and factory functionality

* chore: add serialVersionUID to AppStore.java

* chore: add Fields to AppEndpointDesc and create factory method

* chore: create app endpoint repository

* docs: add license header to AppEndpointRepository

* chore: add AppView and Assembler, add App ResourceController

* chore: add AppEndpointService and View

* chore: add App Endpoint linker

* chore: add AppStoreView, Assembler and Controller

* chore: start implementing templates

* chore: fixed bug and add javadoc

* chore: refactor code

* chore: add additional props for app endpoint and representation

* chore: fix checkstyle error

* chore: refactor view and view assembler classes

* chore: add local data attribute to app entity

* Revert "chore: add local data attribute to app entity"

This reverts commit 58e3d82

* chore: add app impl to store data

* chore: change access level from artifact impl and local data to public

* chore: started implementing AppRequestMessageController (wip)

* chore: add sending of DescriptionRequestMessage to AppRequestMessageController

* chore: fix checkstyle and license

* fix: check instanceof before cast to AppImpl, add simple AppFactoryTest

* chore: add license header

* test: add tests for updating apps

* test: add test for AppEndpoint

* chore: add license header

* fix: nullsafe MappingUtils.fromIdsApp, create Test for fromIdsApp

* chore: checkstyle in MappingUtils

* test: add AppViewAssemblerTest

* test: add Test for APP Endpoint

* test: add AppStoreFactoryTest

* chore: add default fields to AppStoreView

* test: add AppStoreViewAssemblerTest

* chore: update javadoc for AppStoreView

* refactor: create AppStoreController for app store entities

* chore: remove unused imports

* test: update root_nothing_returnApiEntryPoint

* chore: started implementing app store registry service and controller

* fix: fix compiler errors, rename AppStoreController to AppStoresController to avoid bean name clash

* chore: remove redundant AppStoreControllers.java

* chore: don't use deprecated jwt parser method, rename route to /appstores

* style: remove unused imports

* chore: prepare AppRequestMessageController for message exchange with AppStore

* test: add AppStoreControllersTest

* chore: add service method for pulling an image and refactored code

* chore: add service methods for containers like create, start, stop and delete

* chore: marked some api's as not allowed, renamed tags

* chore: updated workflow and implementation for AppRequestMessageController

* chore: single operations on container are summarized under one api, removed unnecessary controller app store registry

* chore: deleted unnecessary methods on container and container configuration class

* chore: hide api endpoints for apps and for relation between app and app endpoints

* chore: started implementing app request message

* feat: add route for related AppStores (TODO: pageable query method in AppStoreRepository)

* chore: add pageable query method

* chore: refactor AppRequestMessageController, download AppResource and Artifact before creating an App (avoid write/read/write to DB inside one call)

* test: add AppRequestMessageController test

* chore: parse metadata from response jsonobject

* chore: fix MappingUtils checkstyle

* chore: add license to AppRequestMessageControllerTest

* chore: refactored and renamed controller classes

* chore: updated app request message controller, refactored controller classes

* chore: add new data type attribute for data class

* chore: app template is persisted in a JSON file

* chore: fix path of generating templates

* chore: add service method to update portainer settings

* chore: remove Portainer Settings JSON logic since API exist to use single template directly to deploy app

* chore: add roadmap to deploy app using portainer

* chore: fix typo

* chore: add missing link between portainer volume and template

* chore: change app props naming

* refactor: change structure of config and requestService

* refactor: set local data instead app data

* feat: call portainer to create new registry depending on AppStore template content

* refactor: PortainerRequestService and AppService

* feat: add Portainer API call to pull image

* refactor: change type of registry at API call

* chore: add createVolume to PortainerRequestService

* chore: add createContainer method to PortainerRequestService

* chore: createVolume return nginx created volume name

* chore: createContainer returns id, call startcontainer

* chore: use volumes in createContainer

* style: fix most checkstyle warnings

* chore: start building json payload in createContainer

* chore: fill some fields of json payload in createContainer

* chore: fill json payload in createContainer

* docs: document building of payload in createContainer

* style: fix checkstyle for createContainer

* test: test PortainerRequestService in ConnectorApplicationIT, for checking against portainer

* style: fix checkstyle for pullImage

* chore: add fromImage to query parameters

* chore: fix queryParameter and request body to include the registry
name of the image

* chore: fix registry name

* fix: fix templatename in createVolumes

* fix: create container correctly, start container in test

* style: fix checkstyle for PortainerRequestService

* feat: automatically use free port for installed app

* style: fix checkstyle

* chore: delete unnecessary service class

* chore: add util class to outsource code from service class

* chore: fix encoding in url

* chore: fix NPE

* chore: refactored logic of requesting apps from app store

* test: updated dummy app store tempalte

* chore: refactor testing app template

* chore: add comments explaining IDS-Message- and UI-Flow for developing

* feat: add methods to create network, and to link container to a network

* feat: add method to check if registry already exists

* chore: adjusted TODOs for AppStore integration

* chore: undo change at controller for app requesting

* style: fix checkstyle

* chore: deleted code line restored

* chore: The start logic action in the AppController revised

* chore: fix NPE

* chore: refactored logic of stop and delete an app container

* feat: add methods to get list of registries, containers, images, volumes and networks from portainer

* feat: add methods to delete volumes, images and networks, and method to disconnect a container from a network

* chore: createRegistry, return true if registry was created or already exists, false if response was not successful

* chore: set correct bootstrapID for app, find apps by bootstrapId

* chore: set correct bootstrapID for app, find apps by bootstrapId

* test: fix failing AppTemplate test

* revert: revert changes in config.json

* test: fix failing post /api/ids/app test

* test: fix illegal state of saveData_artifactPresent_persistData

* chore: use remoteID instead of bootstrapID to identify Apps

* chore: encode payload correctly, initialize Apps as AppImpl to allow downcasts

* fix: add null checking for object mappers

* chore: fill AppStore template auth correctly

* chore: fix container creation from appstore image

* chore: add TOTO to PortainerRequestService

* chore: remove app-deploy logic from /api/ids/app

* test: reset AppRequestMessageControllerTest

* refactor: App actions in AppControllers

* refactor: merge AppImpl into App

* fix: GET /api/apps throwing exception as soon as an app is available

* feat: remove registry if existing at App deploy (credentials are one time usage)

* chore: refactor checkstyle errors

* chore: refactor javadoc

* feat: delete registry after image pull (credentials are useless due to one-time-usage)

* fix: pullImage in PortainerRequestService (WIP)

* chore: add debug TODO

* fix: add Header X-Registry-Auth for portainer pull image

* chore: add debugging TODO

* fix: dont create network if existing

* chore: refactor checkstyle error

* chore: added debugging TODO

* chore: refactor code

* chore: add app output endpoints to portainer ports

* feat: wait until image fully pulled from registry

* chore: fix bug while creating volumes

* chore: fix checkstyle

* refactor: add waitForImagePull as method

* chore: endpoint id is determined by portainer

* chore: container id is deleted from dsc when app is deleted

* test: reset ConnectorApplicationIT

* chore: create container takes into account all types of port specifications

* chore: add method to update the ownership of a container

* chore: app will be updated if it already exists in the database

* chore: unused volumes are deleted and add ToDOs

* fix: fix app mapping, storage, and view

* fix: remove path encoding for image address

* feat: prevent multiple starting an app (action start -> action start for same app)

* fix: template encoding

* feat: prevent deleting a running app (has to be stopped first)

* chore: remove done TODO

* chore: add response handling

* fix: /api/apps/{id}/describe now requires app-id instead of container-id

* Merge remote-tracking branch 'origin/455-app-store' into 455-app-store

* refactor: remove parameters from TemplateBuilder constructor

* fix: add additionals to AppView

* chore: add apps uuid to volume names to create unique volume names per running app

* chore: fix some spotbugs warnings

* chore: fix some spotbugs warnings

* chore: fix most spotbugs warnings in PortainerRequestService

* test: fix EndpointServiceProxy and AppFactoryTests

* test: fix AppRequestControllerTest

* fix: catch unchecked exception in SubscriberNotificationService

* test: don't mock AppRequestController in AppRequestControllerTest, mock AppStore Downloaders instead

AppRequestController returns 201 created on success, not 200 OK

* chore: fix some spotbugs checks

* test: create test for AppController

* fix: api /describe

* chore: fix checkstyle errors

* test: create mocked okhttp response for AppControllerTest

* test: add container description test in AppControllerTest

* test: create empty PortainerRequestServiceTest

* test: add test to PortainerRequestServiceTest

* test: split test of PortainerRequestService into two components, one for JsonArray, one for JsonObject responses

* style: fix checkstyle (unused imports)

* fix: make ArtifactImpl setters package-private

* chore: link downloaded app to app store + refactor error handling in app communication process

* refactor: change log level of appStore-app-linking failures

* fix: link apps to app stores

* test: fix AppRequestControllerTest

* docs: add subsections for ecosystem section

* test: fix failing tests

* fix: log level in EntityPersistenceService

* fix: updateInternal hasUpdated in AppFactory

* chore: duplicate license header removed from GuiUtils

* refactor: media type and default API endpoint as constants in PortainerRequestService

* test: filled fields for testing the mapping of an app resource

* chore: create response util methods for app controller

* chore: improve error handling in containerManagement method

* chore: fix checkstyle

* chore: relation between app and app store can be set and read

* fix: FactoryUtils updatePolicyList return empty instead of default (fixes tests)

* chore: add handling for remaining possible bad status codes

* test: add AppAppStoreLinkerTest

* test: fix AppAppStoreLinkerTest warning

* fix: AppStoreCommunication checkInput optional empty

* test: temp disable build_ResourceTemplateOnlyDesc_returnOnlyResource due to InvalidDataAccessResourceUsageException with Ubuntu Pipeline VMs

* refactor: javadoc in AppStoreService

* refactor: javadoc in AppStoreService

* docs: complete docs for AppStoreFactory and PortainerRequestService

* docs: complete docs for AppFactory, AppRequestController and AppStoreCommunication

* refactor: deployApp remove PortainerNotConfigured exception since not thrown

* chore: add describe to action type

* chore: add describe functionality to the actions API

* chore: add describe action to test method

* docs: update openapi.yaml

* docs: add (app)store guide

* release: increase version from v6.1.3 to v6.2.0

Co-authored-by: omar <[email protected]>
Co-authored-by: Tim Berthold <[email protected]>
Co-authored-by: Gökhan Kahriman <[email protected]>
Co-authored-by: Tim Berthold <[email protected]>
Co-authored-by: Brian-Frederik Jahnke <[email protected]>
Co-authored-by: Pampus, Julia <[email protected]>
Co-authored-by: Julia Pampus <[email protected]>

CHANGELOG.md

@@ -1,6 +1,15 @@
 # Changelog
 All notable changes to this project will be documented in this file.
 
+## [6.2.0] - 2021-09-01
+
+### Added
+- Add app, app store, and app endpoint entities to the data model.
+  - Provide REST endpoints for managing entities and its relations.
+  - Add REST endpoint for managing image/container deployment with Portainer.
+- Add `POST api/ids/app` endpoint for downloading an IDS app's metadata and data from the IDS
+  AppStore.
+
 ## [6.1.3] - 2021-08-27
 
 ### Changed

charts/dataspace-connector/Chart.yaml

@@ -31,13 +31,13 @@ type: application
 # This is the chart version. This version number should be incremented each time you make changes
 # to the chart and its templates, including the app version.
 # Versions are expected to follow Semantic Versioning (https://semver.org/)
-version: 0.1.9
+version: 0.1.10
 
 # This is the version number of the application being deployed. This version number should be
 # incremented each time you make changes to the application. Versions are not expected to
 # follow Semantic Versioning. They should reflect the version the application is using.
 # It is recommended to use it with quotes.
-appVersion: "6.1.3"
+appVersion: "6.2.0"
 
 dependencies:
   - name: postgresql