-
Notifications
You must be signed in to change notification settings - Fork 28
Added append only, upsert only modes for polaris-synchronizer, and put modification aware behind opt-in flag. #11
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
travis-bowen
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These changes make it a lot easier to configure with basic scenarios!
...ynchronizer/api/src/main/java/org/apache/polaris/tools/sync/polaris/PolarisSynchronizer.java
Outdated
Show resolved
Hide resolved
|
|
||
| @CommandLine.Option( | ||
| names = {"--strategy"}, | ||
| defaultValue = "CREATE_ONLY", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is it possible to use the enum values here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Unfortunately, we can't use the enum value. I could declare a separate set of constants for them that are assigned strings with the same value, if you think that's worth adding for maintainability? Something like
enum Strategy {
CREATE_ONLY,
...
class Names {
public static final String CREATE_ONLY = "CREATE_ONLY";
...
}
}There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why doesn't e.g. CREATE_ONLY.name() work?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We can only use a String constant in an annotation.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh, makes sense. Bummer
…hind configurable flag
d1acb62 to
5a8d936
Compare
eric-maynard
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, nice!
Currently the polaris-synchronizer tool performs a full replication onto the target Polaris instance. This includes overwriting entities with the same name on the source and target, as well as removing entities from the target that do not exist on the source.
This change introduces a new option,
--strategyto specify a level of modification the tool should perform on the target. There are three supported strategies:CREATE_ONLY: only create entities on the target instance, no overwrites, no removesCREATE_AND_OVERWRITE: create new entities on the target instance, overwrite entities with the same name across the source and target, no removesREPLICATE: full replication, including reflecting removals from the source instance onto the targetThe lowest level of modification,
CREATE_ONLYis the default.This PR also adds a flag
--delta-onlyto make modification aware synchronization opt-in. If this flag is not specified,