diff --git a/README.md b/README.md index 72acf07b..d58043eb 100644 --- a/README.md +++ b/README.md @@ -6,83 +6,49 @@ The `com.sap.cds:cds-feature-attachments` dependency is a [CAP Java plugin](https://cap.cloud.sap/docs/java/building-plugins) that provides out-of-the box attachments storage and handling by using an aspect Attachments. +It supports the [AWS, Azure, and Google object stores](storage-targets/cds-feature-attachments-oss) and can connect to a [malware scanner](#malware-scanner). + ## Table of Contents -* [Additional Information](#additional-information) -* [Support, Feedback, Contributing](#support-feedback-contributing) -* [Minimum Version](#minimum-version) -* [Artifactory](#artifactory) -* [Storage Targets](#storage-targets) -* [Usage](#usage) - * [CDS Models](#cds-models) - * [Model Texts](#model-texts) - * [Status Texts](#status-texts) - * [UI](#ui) - * [Outbox](#outbox) - * [Malware Scanner](#malware-scanner) - * [Error Messages](#error-messages) - * [Restore Endpoint](#restore-endpoint) - * [Motivation](#motivation) - * [HTTP Endpoint](#http-endpoint) - * [Security](#security) - - -## Additional Information - -- [Process Description](./doc/Processes.md) - -- [Changelog](./doc/CHANGELOG.md) - -- [Contributing](./doc/CONTRIBUTING.md) - -- [License](./LICENSE) - -- [Implementation Details](./doc/Design.md) - -## Support, Feedback, Contributing - -This project is open to feature requests/suggestions, bug reports etc. -via [GitHub issues](https://github.com/cap-java/cds-feature-attachments/issues). -Contribution and feedback are encouraged and always welcome. For more information about how to contribute, the project -structure, as well as additional contribution information, -see our [Contribution Guidelines](./doc/CONTRIBUTING.md). - -## Minimum Version - -The following version are the minimum versions for the usage of the plugin: -| Component | Minimum Version | -|-----------|-----------------| -| CAP Java | 3.10.3 | -| UI5 | 1.136.0 | - -## Maven Central - -The feature is released to Maven Central at: -https://central.sonatype.com/artifact/com.sap.cds/cds-feature-attachments - -## Artifactory - -Snapshots are deployed to SAP's Artifactory in DMZ: -https://common.repositories.cloud.sap/artifactory/cap-java/com/sap/cds/cds-feature-attachments/ - -If you want to test snapshot versions of this plugin, you need to configure the Artifactory in your `${HOME}/.m2/settings.xml`. -See [here](https://maven.apache.org/settings.html#Repositories) for further details. +* [Quick Start](#quick-start) +* [Usage](#usage) + * [MVN Setup](#mvn-setup) + * [Changes in the CDS Models and for the UI](#changes-in-the-cds-models-and-for-the-UI) + * [Storage Targets](#storage-targets) + * [Malware Scanner](#malware-scanner) + * [Outbox](#outbox) + * [Restore Endpoint](#restore-endpoint) + * [Motivation](#motivation) + * [HTTP Endpoint](#http-endpoint) + * [Security](#security) +* [Releases: Maven Central and Artifactory](#releases-maven-central-and-artifactory) +* [Minimum UI and CAP Java Version](#minimum-ui5-and-cap-java-version) +* [Architecture Overview](#architecture-overview) + * [Design](#design) + * [Multitenancy](#multitenancy) + * [Object Stores](#object-stores) + * [Model Texts](#model-texts) +* [Monitoring & Logging](#monitoring--logging) +* [Support, Feedback, Contributing](#support-feedback-contributing) +* [References & Links](#references--links) -## Storage Targets +## Quick Start -By default, the plugin operates without a dedicated storage target, storing attachments directly in the [underlying database](cds-feature-attachments/src/main/resources/cds/com.sap.cds/cds-feature-attachments/attachments.cds#L17). +For a quick setup with in-memory storage: +- Add the `cds-feature-attachments` Maven dependency to the `srv/pom.xml` and configure the `cds-maven-plugin` with the `resolve` goal as described in [MVN Setup](#mvn-setup). +- Extend the CDS model with the `Attachments` aspect and annotate the service for UI integration as explained in [Changes in the CDS Models and for the UI](#changes-in-the-cds-models-and-for-the-UI). -Other available storage targets: -- [Object Store](storage-targets/cds-feature-attachments-oss) -- [local file system as a storage backend](storage-targets/cds-feature-attachments-fs) (only for testing scenarios) +The [incidents app](https://github.com/cap-java/incidents-app/) provides a demonstration of how to use this plugin. -When using a dedicated storage target, the attachment is not stored in the underlying database; instead, it is saved on the specified storage target, and only a reference to the file is kept in the database, as defined in the [CDS model](cds-feature-attachments/src/main/resources/cds/com.sap.cds/cds-feature-attachments/attachments.cds#L20). +For object store integration, see [Amazon, Azure, and Google Object Stores](storage-targets/cds-feature-attachments-oss). ## Usage -The usage of CAP Java plugins is described in the [CAP Java Documentation](https://cap.cloud.sap/docs/java/building-plugins#reference-the-new-cds-model-in-an-existing-cap-java-project). Following this documentation this plugin needs to be referenced in the `srv/pom.xml` of a CAP Java project: +### MVN Setup + +As described in the [CAP Java Documentation](https://cap.cloud.sap/docs/java/building-plugins#reference-the-new-cds-model-in-an-existing-cap-java-project), the attachments plugin needs to be referenced in the `srv/pom.xml` of the consuming CAP Java application: ```xml @@ -91,10 +57,8 @@ The usage of CAP Java plugins is described in the [CAP Java Documentation](https ${latest-version} ``` - -The latest version can be found in the [changelog](./doc/CHANGELOG.md) or in the [Maven Central Repository](https://central.sonatype.com/artifact/com.sap.cds/cds-feature-attachments/versions). - -To be able to also use the CDS models defined in this plugin the `cds-maven-plugin` needs to be used with the `resolve` goal to make the CDS models available in the project: +Additionally, the `cds-maven-plugin` must be configured with the `resolve` goal to ensure CDS models from dependencies are available. +For this, add the following to the `srv/pom.xml` before the entry `build` as well: ```xml @@ -113,240 +77,139 @@ To be able to also use the CDS models defined in this plugin the `cds-maven-plug ``` After that, the aspect `Attachments` can be used in the application's CDS model. -### CDS Models +### Changes in the CDS Models and for the UI -Depending on the location in the application's CDS model where the aspect `Attachments` shall be used, different approaches need to be implemented. -If the aspect `Attachments` shall be used on an entity provided in the `db` module, the corresponding entity needs to be extended from a CDS file in the `srv` module. Therefore the entity from the `db` folder needs to be imported with an `using` statement. Then, this entity can be extended with a new field that is a `Composition of many Attachments`. -The following example shows how to extend the `db` entity `Books` in a CDS file in the `srv` module: +To use the aspect `Attachments` on an existing entity, the corresponding entity needs to be extended in a CDS file in the `srv` module. +The following example shows how to extend the entity `Incidents` in the `srv` module with an additional `attachments.cds` file, it also directly adds the respective UI Facet. +To use this file with the [incidents app](https://github.com/cap-java/incidents-app/), check out the source code, copy the [file from the xmpls folder](https://github.com/cap-java/incidents-app/blob/main/xmpls/attachments.cds) to the srv folder and run the app as explained in the [incidents app README](https://github.com/cap-java/incidents-app/blob/main/README.md). -```cds -using {my.bookshop as my} from '../db/index'; -using {sap.attachments.Attachments} from 'com.sap.cds/cds-feature-attachments'; - -// Extends the Books entity with the Attachments composition -extend my.Books with { - covers : Composition of many Attachments; -}; -``` + ```cds + using { sap.capire.incidents as my } from '../db/schema'; + using { sap.attachments.Attachments } from 'com.sap.cds/cds-feature-attachments'; -To use the aspect `Attachments` in the `srv` module, the following code needs to be added to the existing entity definition: + extend my.Incidents with { + attachments: Composition of many Attachments; + } -```cds -using {sap.attachments.Attachments} from `com.sap.cds/cds-feature-attachments`; - -entity Items : cuid { - ... - attachments : Composition of many Attachments; - ... -} -``` - -The aspect `Attachments` shall be used directly for the composition. -It is very important to use the correct from clause for the `using` statement. Only if `com.sap.cds/cds-feature-attachments` is used and not concrete files of the feature are specified in the from-statement also the annotations and other definitions are found and used. - -#### Model Texts - -In the model several fields are annotated with the `@title` annotation. Starting with version 1.0.6 of the `cds-feature-attachments`, default texts are provided in [35 languages](https://github.com/cap-java/cds-feature-attachments/tree/main/cds-feature-attachments/src/main/resources/cds/com.sap.cds/cds-feature-attachments/_i18n). If these defaults are not sufficient for an application, they can be overwritten by applications with custom texts or translations. - -The following table gives an overview of the fields and the i18n codes: - -| Field Name | i18n Code | -|------------|-----------------------| -| `content` | `attachment_content` | -| `mimeType` | `attachment_mimeType` | -| `fileName` | `attachment_fileName` | -| `status` | `attachment_status` | -| `note` | `attachment_note` | - -In addition to the field names also header information (`@UI.HeaderInfo`) are annotated: - -| Header Info | i18n Code | -|------------------|---------------| -| `TypeName` | `attachment` | -| `TypeNamePlural` | `attachments` | + using { ProcessorService as service } from '../app/services'; + annotate service.Incidents with @( + UI.Facets: [ + ..., + { + $Type : 'UI.ReferenceFacet', + ID : 'AttachmentsFacet', + Label : '{i18n>attachments}', + Target : 'attachments/@UI.LineItem' + } + ] + ); + ``` -#### Status Texts +The UI Facet can also be added directly after other UI Facets in a `cds` file in the `app` folder. -For the status of the attachment only the code value is stored at the moment. -The [status codes](./cds-feature-attachments/src/main/resources/cds/com.sap.cds/cds-feature-attachments/attachments.cds) -are: +### Storage Targets -- `Unscanned` -- `Scanning` -- `Clean` -- `Infected` -- `Failed` +By default, the plugin operates without a dedicated storage target, storing attachments directly in the [underlying database](cds-feature-attachments/src/main/resources/cds/com.sap.cds/cds-feature-attachments/attachments.cds#L17). -If a text for the status needs to be displayed on the UI the model needs to be enhanced with the texts. -For this a new Statuses entity needs to be created like the following example: +Other available storage targets: +- [Amazon, Azure, and Google Object Stores](storage-targets/cds-feature-attachments-oss) +- [local file system as a storage backend](storage-targets/cds-feature-attachments-fs) (only for testing scenarios) -```cds -entity Statuses @cds.autoexpose @readonly { - key code : StatusCode; - text : localized String(255); -} -``` +When using a dedicated storage target, the attachment is not stored in the underlying database; instead, it is saved on the specified storage target and only a reference to the file is kept in the database, as defined in the [CDS model](cds-feature-attachments/src/main/resources/cds/com.sap.cds/cds-feature-attachments/attachments.cds#L20). -For this entity csv files can be included in the project structure with texts and translations, to show the translated -texts on the UI. +### Malware Scanner -With this a text can be added in example above like: +This plugin checks for a binding to +the [SAP Malware Scanning Service](https://help.sap.com/docs/malware-scanning-servce), which needs to have the label `malware-scanner`. The entry in the [mta-file](https://cap.cloud.sap/docs/guides/deployment/to-cf#add-mta-yaml) may look like: -```cds -extend Attachments with { - statusText : Association to Statuses on statusText.code = $self.status; -} ``` - -With this an annotation can be added to the attachments entity to have the status text displayed in the UI: - -```cds -status @( - Common.Text: { - $value: ![statusText.text], - ![@UI.TextArrangement]: #TextOnly - }, - ValueList: {entity:'Statuses'}, - sap.value.list: 'fixed-values' -); +_schema-version: '0.1' +ID: consuming-app +version: 1.0.0 +description: "App consuming the attachments plugin with a malware scanner" +parameters: + ... +modules: + - name: consuming-app-srv +# ------------------------------------------------------------ + type: java + path: srv + parameters: + ... + properties: + ... + build-parameters: + ... + requires: + - name: consuming-app-hdi-container + - name: consuming-app-uaa + - name: cf-logging + - name: malware-scanner +... +resources: + ... + - name: malware-scanner + type: org.cloudfoundry.managed-service + parameters: + service: malware-scanner + service-plan: clamav ``` -### UI +The malware scanner is used in the `AttachmentService` to scan +attachments. -To enhance the UI with the attachments the following annotations are used for the `UI.Facets` annotations -in your app: +If there is no malware scanner available, the attachments are automatically marked as `Clean`. -```cds - { - $Type : 'UI.ReferenceFacet', - ID : 'AttachmentsFacet', - Label : '{i18n>attachments}', - Target : 'attachments/@UI.LineItem' - } -``` - -A complete `UI.Facets` annotation could look like: +Scan status codes: +- `Clean`: Only attachments with the status `Clean` are accessible. +- `Scanning`: Immediately after upload, the attachment is marked as `Scanning`. Depending on processing speed, it may already appear as `Clean` when the page is reloaded. +- `Unscanned`: Attachment is still unscanned. +- `Failed`: Scanning failed. +- `Infected`: The attachment is infected. -```cds -annotate service.Incidents with @( - UI.Facets : [ - { - $Type : 'UI.CollectionFacet', - Label : '{i18n>Overview}', - ID : 'Overview', - Facets : [ - { - $Type : 'UI.ReferenceFacet', - Label : '{i18n>GeneralInformation}', - ID : 'i18nGeneralInformation', - Target : '@UI.FieldGroup#i18nGeneralInformation', - }, - { - $Type : 'UI.ReferenceFacet', - Label : '{i18n>Details}', - ID : 'i18nDetails', - Target : '@UI.FieldGroup#i18nDetails', - } - ] - }, - { - $Type : 'UI.ReferenceFacet', - Label : 'Conversations', - ID : 'Conversations', - Target : 'conversations/@UI.LineItem#Conversations', - }, - { - $Type : 'UI.ReferenceFacet', - ID : 'AttachmentsFacet', - Label : '{i18n>attachments}', - Target : 'attachments/@UI.LineItem' - } - ] -); -``` ### Outbox In this plugin the [persistent outbox](https://cap.cloud.sap/docs/java/outbox#persistent) is used to mark attachments as deleted. -The enablement of the outbox is also included in the cds models of this plugin. +When using this plugin, the persistent outbox is enabled by default. In the capire documentation of the [persistent outbox](https://cap.cloud.sap/docs/java/outbox#persistent) is it described how to overwrite the default outbox configuration. If the default shall be used, nothing needs to be done. -### Malware Scanner - -This plugin checks for a binding to -the [SAP Malware Scanning Service](https://help.sap.com/docs/malware-scanning-servce). -The concrete check if for a binding to a service with label `malware-scanner`. - -The malware scanner is used in the default implementation of the technical service `AttachmentService` to scan -attachments. -If the default implementation of this service is overwritten, e.g. by using the plugin enhancement of the -[SAP Document Management Service](https://help.sap.com/docs/document-management-service), then this overwriting plugin -is responsible for the malware scan and the plugin documentation needs to be checked for how the malware scan is done. - -If the default implementation is used and the malware scanner is not available the attachments are marked as clean -by setting the status of the attachment to: - -- `Clean` - -Only attachments with the status `Clean` are accessible. -Attachments with all other status codes are not accessible. - -If the malware scanner is available but during the request to the scanner an error occurs the status of the attachment -is set to: - -- `Failed` - -### Error Messages - -If attachments are uploaded but not scanned by a malware scanner (if a scanner is available) or are marked as infected -the direct access of the -attachment is not possible. -In case users try to access the content of the attachment the following errors messages are displayed: - -| Error Message | Error Message i18n Code | -|-------------------------------------------------------|-------------------------| -| Attachment is not clean | `not_clean` | -| Attachment is not scanned, try again in a few minutes | `not_scanned` | - -By adding the error message i18n code to the `i18n.properties` file the error message can be overwritten translated. -More information can be found in the capire documentation -for [i18n](https://cap.cloud.sap/docs/guides/i18n#where-to-place-text-bundles). ### Restore Endpoint The attachment service has an event `RESTORE_ATTACHMENTS`. -This event can be called with a timestamp to restore external stored attachments. +This event can be called with a timestamp to restore externally stored attachments. #### Motivation Documents which are marked as deleted can be restored. -The use case behind this feature is: - -If backups of databases are restored the attachments stored in external storages also needs to be restored. -To have a possibility to restore attachments which are marked as deleted a restore endpoint is available. +The use cases behind this feature are: +- Restoring attachments after a database backup is restored: +When restoring a database backup, any attachments stored in external storage (object stores, etc.) also need to be restored to maintain data consistency. +- Restoring attachments that were marked as deleted: +The restore endpoint provides a way to recover attachments that were previously marked as deleted, making it possible to undo deletions if needed. In the default implementation of the technical service `AttachmentService` this is not needed as the attachments are stored directly in the database and are restored with the database. -If the default implementation is overwritten, e.g. by using the plugin enhancement of the -[SAP Document Management Service](https://help.sap.com/docs/document-management-service), then this overwriting plugin +If the default implementation is replaced by overwriting the [respective handler](cds-feature-attachments/src/main/java/com/sap/cds/feature/attachments/service/handler/DefaultAttachmentsServiceHandler.java#L87), for example by the +[SAP Document Management Service](https://help.sap.com/docs/document-management-service), then the overwriting plugin needs to handle the restore of attachments. In such cases the restore endpoint can be used to restore attachments. -How long attachments are marked as deleted before they get deleted dependents on the configuration -of the used storage. +How long attachments are marked as deleted before they are actually deleted depends on the configuration of the used storage. #### HTTP Endpoint There is no predefined endpoint for the restore action. -To call the action of the service from outside the application a service could be defined like the following example: +To call the action of the service from outside the application a service could be defined as in the following example: ```cds service RestoreAttachments { @@ -354,6 +217,7 @@ service RestoreAttachments { } ``` +See [Security](#security) for how to secure this endpoint. The action `restoreAttachments` could get in a timestamp from which the attachments need to be restored. The action could be called with a POST request to the endpoint: @@ -394,7 +258,7 @@ In the Spring Boot context the `AttachmentService` can be autowired in the handl #### Security -To secure the endpoint security annotations can be used e.g. like the following example: +To secure the endpoint, security annotations can be used. For example: ```cds using {sap.attachments.Attachments} from `com.sap.cds/cds-feature-attachments`; @@ -409,7 +273,81 @@ annotate RestoreAttachments with @(requires: 'internal-user'); ``` Here the `RestoreAttachments` service is annotated with the `requires` annotation to secure the service. -Also, other annotations can be used to secure the service. +Various other annotations can be used to secure the service. More information about the CAP Java security concept can be found in the [CAP Java Documentation](https://cap.cloud.sap/docs/java/security). + +## Releases: Maven Central and Artifactory + +- The plugin is released to Maven Central at: https://central.sonatype.com/artifact/com.sap.cds/cds-feature-attachments (public access). +- See the [changelog](./doc/CHANGELOG.md) for the latest changes. + +- To test snapshot versions of this plugin, the artifactory in `${HOME}/.m2/settings.xml` needs to be configured. See [the maven settings](https://maven.apache.org/settings.html#Repositories) for further details. + +## Minimum UI5 and CAP Java Version + +| Component | Minimum Version | +|-----------|-----------------| +| CAP Java | 3.10.3 | +| UI5 | 1.136.0 | + + +## Architecture Overview +### Design +- [Design Details](./doc/Design.md) +- [Process of Creating, Reading and Deleting an Attachment](./doc/Processes.md) + +### Multitenancy + +- When using SAP HANA as the storage target, multitenancy support depends on the consuming application. In most cases, multitenancy is achieved by using a dedicated schema for each tenant, providing strong data isolation at the database level. +- When using an [object store](storage-targets/cds-feature-attachments-oss) as the storage target, true multitenancy is not yet implemented (as of version 1.2.1). In this case, all blobs are stored in a single bucket, and tenant data is not separated. + +### Object Stores + +See [Object Stores](storage-targets/cds-feature-attachments-oss). + +### Model Texts + +In the model, several fields are annotated with the `@title` annotation. Default texts are provided in [35 languages](https://github.com/cap-java/cds-feature-attachments/tree/main/cds-feature-attachments/src/main/resources/cds/com.sap.cds/cds-feature-attachments/_i18n). If these defaults are not sufficient for an application, they can be overwritten by applications with custom texts or translations. + +The following table gives an overview of the fields and the i18n codes: + +| Field Name | i18n Code | +|------------|-----------------------| +| `content` | `attachment_content` | +| `mimeType` | `attachment_mimeType` | +| `fileName` | `attachment_fileName` | +| `status` | `attachment_status` | +| `note` | `attachment_note` | + +In addition to the field names, header information (`@UI.HeaderInfo`) are also annotated: + +| Header Info | i18n Code | +|------------------|---------------| +| `TypeName` | `attachment` | +| `TypeNamePlural` | `attachments` | + + +## Monitoring & Logging + +To configure logging for the attachments plugin, add the following line to the `/srv/src/main/resources/application.yaml` of the consuming application: +``` +logging: + level: + ... + '[com.sap.cds.feature.attachments]': DEBUG +... +``` + +## Support, Feedback, Contributing + +This project is open to feature requests/suggestions, bug reports etc. +via [GitHub issues](https://github.com/cap-java/cds-feature-attachments/issues). + +Contribution and feedback are encouraged and always welcome. For more information about how to contribute, the project +structure, as well as additional contribution information, +see our [Contribution Guidelines](./doc/CONTRIBUTING.md). + +## References & Links +- [License](./LICENSE) diff --git a/storage-targets/cds-feature-attachments-oss/README.md b/storage-targets/cds-feature-attachments-oss/README.md index a7f23137..09420e42 100644 --- a/storage-targets/cds-feature-attachments-oss/README.md +++ b/storage-targets/cds-feature-attachments-oss/README.md @@ -14,13 +14,47 @@ To do this, replace the `cds-feature-attachments` dependency in your `pom.xml` w ``` -A valid Object Store service binding is required for this — for example, one provisioned through SAP BTP. +A valid Object Store service binding is required for this — for example, one provisioned through SAP BTP. The corresponding entry in the [mta-file](https://cap.cloud.sap/docs/guides/deployment/to-cf#add-mta-yaml) possibly looks like: -#### Tests +``` +_schema-version: '0.1' +ID: consuming-app +version: 1.0.0 +description: "App consuming the attachments plugin with an object store" +parameters: + ... +modules: + - name: consuming-app-srv +# ------------------------------------------------------------ + type: java + path: srv + parameters: + ... + properties: + ... + build-parameters: + ... + requires: + - name: consuming-app-hdi-container + - name: consuming-app-uaa + - name: cf-logging + - name: object-store-service +... +resources: + ... + - name: object-store-service + type: org.cloudfoundry.managed-service + parameters: + service: objectstore + service-plan: standard +``` + + +### Tests The unit tests in this module do not need a binding to the respective object stores, run them with `mvn clean install`. -The integration tests need a binding to a real object store, run them with `mvn clean install -Pintegration-tests-oss`. +The integration tests need a binding to a real object store. Run them with `mvn clean install -Pintegration-tests-oss`. To set the binding, provide the following environment variables: - AWS_S3_BUCKET - AWS_S3_REGION