CMR-10140-1: Adding subscribing to the local and AWS topics and publishing messages.

[eereiter]

Overview

What is the feature/fix?

Adding the capability to subscribe/unsubscribe to topics either AWS or local
Adding the capability to publish notification messages when it meets subscription criteria

What is the Solution?

Added the capability to subscribe/unsubscribe an AWS queue to either an AWS or local topic.
Added the capability to metadata_db to send notification messages if the concepts meet the subscription criteria.

What areas of the application does this impact?

Ingest Subscriptions and Metadata_DB.

Checklist

• I have updated/added unit and int tests that prove my fix is effective or that my feature works
• New and existing unit and int tests pass locally and remotely
• clj-kondo has been run locally and all errors corrected
• I have removed unnecessary/dead code and imports in files I have changed
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have cleaned up integration tests by doing one or more of the following:
  • migrated any are2 tests to are3 in files I have changed
  • de-duped, consolidated, removed dead int tests
  • transformed applicable int tests into unit tests
  • refactored to reduce number of system state resets by updating fixtures (use-fixtures :each (ingest/reset-fixture {})) to be :once instead of :each

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 48.06202% with 134 lines in your changes missing coverage. Please review.

Project coverage is 58.21%. Comparing base (170ed1b) to head (c3f52dc).

Files with missing lines	Patch %	Lines
...ueue-lib/src/cmr/message_queue/topic/aws_topic.clj	8.33%	33 Missing ⚠️
...ue-lib/src/cmr/message_queue/topic/local_topic.clj	29.72%	26 Missing ⚠️
...p/src/cmr/metadata_db/services/concept_service.clj	0.00%	21 Missing ⚠️
...src/cmr/metadata_db/services/sub_notifications.clj	12.50%	14 Missing ⚠️
.../cmr/metadata_db/data/oracle/sub_notifications.clj	9.09%	10 Missing ⚠️
...a-db-app/src/cmr/metadata_db/api/subscriptions.clj	0.00%	6 Missing ⚠️
...p/src/cmr/ingest/services/subscriptions_helper.clj	16.66%	5 Missing ⚠️
...rations/091_update_cmr_sub_notifications_table.clj	37.50%	5 Missing ⚠️
...app/src/cmr/metadata_db/services/subscriptions.clj	94.73%	0 Missing and 5 partials ⚠️
...ueue-lib/src/cmr/message_queue/queue/aws_queue.clj	25.00%	3 Missing ⚠️
... and 3 more

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #2185      +/-   ##
==========================================
- Coverage   58.23%   58.21%   -0.03%     
==========================================
  Files        1054     1055       +1     
  Lines       70803    70980     +177     
  Branches     2021     2022       +1     
==========================================
+ Hits        41234    41321      +87     
- Misses      27688    27771      +83     
- Partials     1881     1888       +7     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[TylerHeald1]

Does an SQS client need to be made for each subscription? Is it like a different interface to the SQS service for each subscription?

[eereiter]

I never figured out where the underlying http connection (assuming its http) lived in the AWS libraries. I thought that it would live in the client, so I wanted to limit those and keep them open for quicker communication, but if that is not the case, then we don't need it in the subscription. It is possible that a queue could live in a different place from other queues, but when the subscription is set up all of the clients are set to use the local queue manager.