Implement configurable insert vs update setting #74
Comments
Merged
pnadolny13
added a commit
that referenced
this issue
Jul 7, 2023
- bumps sdk - removes all code that was migrated to the SDK - fixes the testing hackiness to use the default testing framework from the docs now - had to remove the class fixture for resource/connection in the tests otherwise `ScopeMismatch: You tried to access the function scoped fixture runner with a class scoped request object` was raised Pending: - the updates to assert key properties are present in the records are causing 3 failed tests because the column names are being conformed in key properties but not in the record so it cant find them and is erroring. UPDATE: I fixed in here and created meltano/sdk#1819. SECOND UPDATE: I opened #74 and overrode the `_singer_validate_message` logic to do nothing for now. Once #74 is implemented we can run the validation if update mode is chosen.
|
Theres a pending PR meltano/sdk#1893 to add this to the SDK, then we'd need bump the SDK version and probably implement some pieces of it. |
|
Also very interested in this. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Related to meltano/sdk#1783
We should allow users to configure whether they want every record inserted in append only mode or if we should update records that exist. If someone wants to update then they need to have key properties otherwise the target should raise and exception. It feels like the current default in the community is to update and require key properties, so sticking with the status quo seems like a fine path forward.
This target implements logic to use copy or merge statements depending on whether key properties are present so we can do both, although meltano/sdk#1819 is requiring key properties so it would never use our copy logic.
The text was updated successfully, but these errors were encountered: