Docs: Clarify merge-on-read modes in docs for spark#4223
Docs: Clarify merge-on-read modes in docs for spark#4223ajantha-bhat wants to merge 1 commit intoapache:masterfrom
Conversation
ajantha-bhat
changed the title
docs/versioned/spark/spark-writes.md
Outdated
| !!! Note | ||
| By default Spark uses copy-on-write merge mode. | ||
| With spark-3.2 and onwards, iceberg supports merge-on-read mode. | ||
| To use merge-on-read merge mode, need to set the table property `write.merge.mode` to "merge-on-read". |
There was a problem hiding this comment.
Do we need to add that format-version needs to be v2?
Also nit: I think 'need to' is redundant, removing it seems more consistent with the rest of docs
|
|
||
| Only one record in the source data can update any given row of the target table, or else an error will be thrown. | ||
|
|
||
| !!! Note |
There was a problem hiding this comment.
Does the three !!! renders anything? (I dont see it when viewing the file as md). Should we just use a normal sub-header here?
There was a problem hiding this comment.
It causes this to render in a text box. I would prefer having a full section on the copy-on-write vs merge-on-read distinction and then refer to it from the individual command sections.
docs/versioned/spark/spark-writes.md
Outdated
|
|
||
| !!! Note | ||
| By default Spark uses copy-on-write merge mode. | ||
| With spark-3.2 and onwards, iceberg supports merge-on-read mode. |
There was a problem hiding this comment.
Not sure what everyone thinks, but seems there's not much context. Should we explain what it means (that it writes v2 delete files)?
|
@samredai FYI |
samredai
left a comment
samredai
left a comment
There was a problem hiding this comment.
LGTM, but I had one grammar nit
docs/versioned/spark/spark-writes.md
Outdated
| !!! Note | ||
| By default Spark uses copy-on-write merge mode. | ||
| With spark-3.2 and onwards, iceberg supports merge-on-read mode. | ||
| To use merge-on-read merge mode, need to set the table property `write.merge.mode` to "merge-on-read". |
There was a problem hiding this comment.
grammar nit: "need to set the table property" can just say "set the table property". Same comment for the other note boxes.
|
Nvm, it's already added but just not deployed yet! |
|
@samredai , @rdblue , @szehon-ho : This is commonly asked question in the Iceberg slack workspace. Today also someone asking this. |
+1 |
liuml07
left a comment
liuml07
left a comment
There was a problem hiding this comment.
Looks clear and simple. Thanks
docs/spark/spark-writes.md
Outdated
|
|
||
| !!! Note | ||
| By default Spark uses copy-on-write update mode. | ||
| With spark-3.2 and onwards, iceberg supports merge-on-read mode. |
There was a problem hiding this comment.
s/iceberg/Iceberg/
s/merge-on-read mode/merge-on-read update mode/
Same other places.
|
closing as stale. Might take another shot at it in the future based on the need. |
5 participants
Currently iceberg-spark documentation doesn't talk about merge-on-read mode. Hence the PR.