Skip to content

Conversation

@iyt-trifork
Copy link
Contributor

@iyt-trifork iyt-trifork commented Aug 20, 2024

XA Transaction Issue with HAPI FHIR and PostgreSQL

We are experiencing issues in our HAPI FHIR application when using XA transactions, similar to those reported in issue #3412.

Problem

The PostgreSQL JDBC driver's PGXAConnection does not support suspend/resume operations, as detailed in the official documentation. This limitation causes exceptions when the PROPAGATION_REQUIRES_NEW transaction propagation behavior is used in XA environments, since it requires suspending the current transaction and resuming it later.

Proposed Solution

The solution focuses on simplifying the TagDefinition handling by addressing the primary concern of managing tag creation in a concurrent environment. Previously, complex retry and nested transaction logic was necessary due to a uniqueness constraint on the hfj_tag_def table, which prevented duplicate tag definitions. However, this constraint was relaxed, and duplicate tag definitions are now allowed, making the nested transaction and retry logic obsolete.

@iyt-trifork iyt-trifork changed the title Refactor getOrCreateTag method to use PROPAGATION_REQUIRED for XA transaction compatibility Refactor BaseHapiFhirDao.getOrCreateTag method to use PROPAGATION_REQUIRED for XA transaction compatibility Aug 20, 2024
@iyt-trifork iyt-trifork force-pushed the fix-xa-suspend-errors branch 5 times, most recently from 203354b to bc9df1e Compare August 20, 2024 13:23
Copy link
Contributor

@michaelabuckley michaelabuckley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can't create tags inline during the transaction. This will cause conflicts at commit times during mass ingestion when the create tags.

@iyt-trifork iyt-trifork force-pushed the fix-xa-suspend-errors branch from bc9df1e to 7534eb8 Compare August 21, 2024 14:13
@iyt-trifork iyt-trifork force-pushed the fix-xa-suspend-errors branch from 7534eb8 to 1154a9f Compare August 22, 2024 08:41
@iyt-trifork iyt-trifork changed the title Refactor BaseHapiFhirDao.getOrCreateTag method to use PROPAGATION_REQUIRED for XA transaction compatibility Refactor BaseHapiFhirDao.getOrCreateTag method to run in a separate thread for XA transaction compatibility Aug 22, 2024
@iyt-trifork iyt-trifork force-pushed the fix-xa-suspend-errors branch 3 times, most recently from c04f886 to fb0577f Compare August 29, 2024 07:46
@codecov
Copy link

codecov bot commented Aug 29, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 83.58%. Comparing base (406db33) to head (b18c22b).
Report is 87 commits behind head on master.

Additional details and impacted files
@@             Coverage Diff              @@
##             master    #6224      +/-   ##
============================================
+ Coverage     83.54%   83.58%   +0.04%     
- Complexity    27432    27560     +128     
============================================
  Files          1707     1716       +9     
  Lines        106185   106634     +449     
  Branches      13397    13430      +33     
============================================
+ Hits          88710    89135     +425     
+ Misses        11750    11740      -10     
- Partials       5725     5759      +34     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@jkiddo
Copy link
Contributor

jkiddo commented Sep 3, 2024

FYI @michaelabuckley all tests go green now

Copy link
Contributor

@michaelabuckley michaelabuckley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work. I have requested a couple more changes.

Copy link
Contributor

@michaelabuckley michaelabuckley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm sorry. I've led you down an obsolete path. I forgot that we relaxed the constraint on the hfj_tag_def table. It used to be unique. But that broke last year, and we now allow duplicate definitions in hfj_tag_def. So all this complexity with the nested transaction is no longer needed.

I think your original path will work. Just commit the tag definitions as part the main transaction, and add them to the shared cache once submitted.

See #4813

Again, I apologize for asking for this complexity.

…XA transaction compatibility

    The getOrCreateTag method previously used a propagation behavior that caused issues with
    XA transactions when using the PostgreSQL JDBC driver. The PGXAConnection does not support
    transaction suspend/resume, which made it incompatible with the existing propagation strategy
    'PROPAGATION_REQUIRES_NEW'.

    This refactor changes the getOrCreateTag logic to perform a lookup/write in a new transaction
    as before, but running in a separate thread, such that the main transaction is not suspended.
    The result is retrieved through a future.

    This change aims to improve compatibility and prevent transaction-related issues when using HAPI-FHIR with
    XA transactions and PostgreSQL.

    Closes hapifhir#3412
@iyt-trifork iyt-trifork force-pushed the fix-xa-suspend-errors branch 2 times, most recently from 341b059 to 966509c Compare September 13, 2024 13:09
- Simplified tag creation by removing unnecessary transaction complexity, since we allow duplicate tags in hfj_tag_def from hapifhir#4813
- Removed redundant retry logic based on updated DB constraints
jamesagnew added a commit that referenced this pull request Oct 30, 2024
Copy link
Collaborator

@jamesagnew jamesagnew left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a really nice clean solution in the end. Thanks for all the work on it!

@jamesagnew jamesagnew merged commit e43c140 into hapifhir:master Oct 30, 2024
jamesagnew added a commit that referenced this pull request Oct 30, 2024
jamesagnew added a commit that referenced this pull request Oct 30, 2024
@iyt-trifork iyt-trifork deleted the fix-xa-suspend-errors branch November 18, 2024 09:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

BaseHapiFhirDao.getOrCreateTag (hapi-fhir 5.7.0) causes exception: org.hibernate.exception.GenericJDBCException: could not prepare statement

4 participants