Optimize formio data access #5118
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
viktorvanwijk
added
owner: den-haag
needs-backport
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #5118 +/- ##
=======================================
Coverage 96.73% 96.73%
=======================================
Files 775 775
Lines 26662 26664 +2
Branches 3468 3469 +1
=======================================
+ Hits 25792 25794 +2
Misses 608 608
Partials 262 262 ☔ View full report in Codecov by Sentry. |
In FormioData, glom is used to perform a deep set/get on the data, which is required for keys with a '.' as they indicate a nested data structure. Unfortunately, glom is slow, as already previously shown by the patch for GH-4744. For forms with lots of form variables (300+), creating a FormioData instance while using `glom.assign` in __setitem__ is an expensive task. Especially for forms which also contain lots of logic rules (50+), this can hurt the performance of a check-logic call, as multiple FormioData instances are created when evaluating each logic rule. This patch is a bandaid fix where we bypass `glom.assign` when setting an item in FormioData.__setitem__. If the key does not contain a '.', which means it is a top-level key, we set the value on self.data directly. Otherwise, we use `glom.assign` for a deep assignment of the value. It is labeled as a bandaid fix, because the underlying problem of creating lots of FormioData instances is still present. This is not addressed here as it must be possible to backport this patch easily. Investigating and refactoring the complete data structures will take more time.
viktorvanwijk
changed the title
viktorvanwijk
force-pushed
the
hotfix/optimize-performance-formio-data-access
branch
from
8d10746 to
47bf93a
Compare
vaszig
approved these changes
Feb 26, 2025
viktorvanwijk
deleted the
hotfix/optimize-performance-formio-data-access
branch
viktorvanwijk
added a commit
that referenced
this pull request
Feb 26, 2025
In FormioData, glom is used to perform a deep set/get on the data, which is required for keys with a '.' as they indicate a nested data structure. Unfortunately, glom is slow, as already previously shown by the patch for GH-4744. For forms with lots of form variables (300+), creating a FormioData instance while using `glom.assign` in __setitem__ is an expensive task. Especially for forms which also contain lots of logic rules (50+), this can hurt the performance of a check-logic call, as multiple FormioData instances are created when evaluating each logic rule. This patch is a bandaid fix where we bypass `glom.assign` when setting an item in FormioData.__setitem__. If the key does not contain a '.', which means it is a top-level key, we set the value on self.data directly. Otherwise, we use `glom.assign` for a deep assignment of the value. It is labeled as a bandaid fix, because the underlying problem of creating lots of FormioData instances is still present. This is not addressed here as it must be possible to backport this patch easily. Investigating and refactoring the complete data structures will take more time. Backport-Of: #5118 (cherry picked from commit 47bf93a)
viktorvanwijk
added a commit
that referenced
this pull request
Feb 26, 2025
In FormioData, glom is used to perform a deep set/get on the data, which is required for keys with a '.' as they indicate a nested data structure. Unfortunately, glom is slow, as already previously shown by the patch for GH-4744. For forms with lots of form variables (300+), creating a FormioData instance while using `glom.assign` in __setitem__ is an expensive task. Especially for forms which also contain lots of logic rules (50+), this can hurt the performance of a check-logic call, as multiple FormioData instances are created when evaluating each logic rule. This patch is a bandaid fix where we bypass `glom.assign` when setting an item in FormioData.__setitem__. If the key does not contain a '.', which means it is a top-level key, we set the value on self.data directly. Otherwise, we use `glom.assign` for a deep assignment of the value. It is labeled as a bandaid fix, because the underlying problem of creating lots of FormioData instances is still present. This is not addressed here as it must be possible to backport this patch easily. Investigating and refactoring the complete data structures will take more time. Backport-Of: #5118 (cherry picked from commit 47bf93a)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No related issue
Changes
Bypass glom in
FormioData.__setitem__for top-level keys (more info is in the commit message).Checklist
Check off the items that are completed or not relevant.
Impact on features
Release management
I have updated the translations assets (you do NOT need to provide translations)
./bin/makemessages_js.sh./bin/compilemessages_js.shDockerfile/scripts
./binfolderCommit hygiene