Skip to content

Add JAX-RS tests#1566

Merged
nastra merged 1 commit intoprojectnessie:mainfrom
nastra:jaxrs
Jul 12, 2021
Merged

Add JAX-RS tests#1566
nastra merged 1 commit intoprojectnessie:mainfrom
nastra:jaxrs

Conversation

@nastra
Copy link
Contributor

@nastra nastra commented Jul 8, 2021

Add JAX-RS tests for Nessie server implementation. Tests use Jersey and
Weld as a alternative container environment.


This change is Reviewable

@codecov
Copy link

codecov bot commented Jul 9, 2021

Codecov Report

Merging #1566 (4fb41df) into main (00bcd3b) will increase coverage by 20.11%.
The diff coverage is 63.93%.

❗ Current head 4fb41df differs from pull request most recent head 50d2f6e. Consider uploading reports for the commit 50d2f6e to get more accurate results
Impacted file tree graph

@@              Coverage Diff              @@
##               main    #1566       +/-   ##
=============================================
+ Coverage     56.13%   76.24%   +20.11%     
- Complexity        0     2084     +2084     
=============================================
  Files           150      281      +131     
  Lines          4949    12644     +7695     
  Branches        420     1006      +586     
=============================================
+ Hits           2778     9641     +6863     
- Misses         2029     2528      +499     
- Partials        142      475      +333     
Flag Coverage Δ
java 75.40% <63.93%> (+19.45%) ⬆️
python 86.40% <ø> (+28.37%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...in/java/org/projectnessie/client/auth/AwsAuth.java 0.00% <0.00%> (ø)
...e/client/rest/NessieBackendThrottledException.java 0.00% <0.00%> (ø)
.../java/org/projectnessie/hms/BaseTraceRawStore.java 0.00% <0.00%> (ø)
.../projectnessie/hms/annotation/MethodSignature.java 0.00% <0.00%> (ø)
...va/org/projectnessie/hms/annotation/NoopQuiet.java 0.00% <0.00%> (ø)
.../org/apache/hadoop/hive/metastore/api/Catalog.java 0.00% <ø> (ø)
.../org/projectnessie/hms/Hive2PartitionFilterer.java 81.25% <ø> (ø)
...er/extensions/NessieSparkSqlExtensionsParser.scala 0.00% <ø> (ø)
...ser/extensions/NessieSqlExtensionsAstBuilder.scala 0.00% <ø> (ø)
...atalyst/plans/logical/AssignReferenceCommand.scala 0.00% <ø> (ø)
... and 451 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9b0ff68...50d2f6e. Read the comment docs.

@nastra nastra requested review from rymurr and snazy July 9, 2021 07:45
@nastra nastra force-pushed the jaxrs branch 3 times, most recently from 3b9b544 to f73cbcf Compare July 9, 2021 09:07
Copy link
Contributor

@rymurr rymurr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few minor comments. Overall i think it looks good.

@nastra nastra force-pushed the jaxrs branch 2 times, most recently from 4fb41df to 3ab28aa Compare July 9, 2021 16:13
@nastra nastra force-pushed the jaxrs branch 2 times, most recently from d6b5682 to 9b34beb Compare July 12, 2021 07:22
Co-authored-by: Laurent Goujon <laurent@dremio.com>

Add JAX-RS tests for Nessie server implementation. Tests use Jersey and
Weld as a alternative container environment.
Copy link
Member

@snazy snazy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

Just make sure to run ./mvnw clean install -pl servers/quarkus-server -Pnative locally.

@nastra nastra merged commit 36d426d into projectnessie:main Jul 12, 2021
@nastra nastra deleted the jaxrs branch July 12, 2021 08:54
snazy added a commit to snazy/iceberg that referenced this pull request Jul 15, 2021
More changes in this PR in following commits.

Replace Gradle plugin with new JUnit extension.
See [Add JAX-RS tests and add JUnit/Jupyter extension](projectnessie/nessie#1566)
snazy added a commit to snazy/iceberg that referenced this pull request Jul 15, 2021
More changes in this PR in following commits.

Replace Gradle plugin with new JUnit extension.
See [Add JAX-RS tests and add JUnit/Jupyter extension](projectnessie/nessie#1566)
rymurr pushed a commit to apache/iceberg that referenced this pull request Jul 15, 2021
* Bump Nessie to 0.8.2 + replace Gradle plugin with new JUnit extension

More changes in this PR in following commits.

Replace Gradle plugin with new JUnit extension.
See [Add JAX-RS tests and add JUnit/Jupyter extension](projectnessie/nessie#1566)

* Changes required by Nessie-API changes

Apply changes to Iceberg required by API changes in Nessie:
* [Re-introduce wrapper classes for query params of CommitLog/Entries](projectnessie/nessie#1595)
* [Server-side commit range filtering](projectnessie/nessie#1596)
* [Add hashOnRef query param to support time travel on a named ref](projectnessie/nessie#1589)
* [Only accept NamedRefs in REST API](projectnessie/nessie#1583)

* Bugfix: must send the Contents.id of the existing table

Nessie's `Contents.id` is a random ID generated when the `Contents.Key` is first used (think:
CREATE TABLE) and must not be changed. This change addresses a bug in the Iceberg-Nesie code
that caused a new id for every change.

* Throw `CommitStateUnknownException` for `renameTable` as well

Follow-up of #2515

* Fix race-condition & save one roundtrip to Nessie during "commit"

When commiting a change, the Nessie-API now returns the hash of the commit for the change.
This returned hash should then be used as the "expected hash" for the next commit.

The previous approach was to commit the change to Nessie and then do another request to
retrieve the new hash of HEAD.

This old approach is prone to a race condition, namely when another commit happens after
"this" commit but before retrieving the "new HEAD", so "this" instance would wrongly
ignore the other commit's changes during conflict checks.

See [Let VersionStore.create()+commit() return the current hash](projectnessie/nessie#1089)
minchowang pushed a commit to minchowang/iceberg that referenced this pull request Aug 2, 2021
* Bump Nessie to 0.8.2 + replace Gradle plugin with new JUnit extension

More changes in this PR in following commits.

Replace Gradle plugin with new JUnit extension.
See [Add JAX-RS tests and add JUnit/Jupyter extension](projectnessie/nessie#1566)

* Changes required by Nessie-API changes

Apply changes to Iceberg required by API changes in Nessie:
* [Re-introduce wrapper classes for query params of CommitLog/Entries](projectnessie/nessie#1595)
* [Server-side commit range filtering](projectnessie/nessie#1596)
* [Add hashOnRef query param to support time travel on a named ref](projectnessie/nessie#1589)
* [Only accept NamedRefs in REST API](projectnessie/nessie#1583)

* Bugfix: must send the Contents.id of the existing table

Nessie's `Contents.id` is a random ID generated when the `Contents.Key` is first used (think:
CREATE TABLE) and must not be changed. This change addresses a bug in the Iceberg-Nesie code
that caused a new id for every change.

* Throw `CommitStateUnknownException` for `renameTable` as well

Follow-up of apache#2515

* Fix race-condition & save one roundtrip to Nessie during "commit"

When commiting a change, the Nessie-API now returns the hash of the commit for the change.
This returned hash should then be used as the "expected hash" for the next commit.

The previous approach was to commit the change to Nessie and then do another request to
retrieve the new hash of HEAD.

This old approach is prone to a race condition, namely when another commit happens after
"this" commit but before retrieving the "new HEAD", so "this" instance would wrongly
ignore the other commit's changes during conflict checks.

See [Let VersionStore.create()+commit() return the current hash](projectnessie/nessie#1089)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants