Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle Delta Lake vacuum file system error #22248

Merged
merged 1 commit into from
Oct 3, 2024

Conversation

nevillelyh
Copy link
Member

@nevillelyh nevillelyh commented Jun 3, 2024

Description

Error when deleting S3 files should be catagorized as file system error for accounting purposes.

Stack trace:

{
  "type": "io.trino.spi.TrinoException",
  "message": "Failure when vacuuming <table> with retention 7d: java.io.IOException: Exception while batch deleting paths",
  "cause": {
    "type": "java.lang.RuntimeException",
    "message": "Failure when vacuuming <table> with retention 7d: java.io.IOException: Exception while batch deleting paths",
    "cause": {
      "type": "java.io.IOException",
      "message": "Exception while batch deleting paths",
      "cause": {
        "type": "com.amazonaws.services.s3.model.MultiObjectDeleteException",
        "message": "One or more objects could not be deleted (Service: null; Status Code: 200; Error Code: null; Request ID: <request_id>; S3 Extended Request ID: <request_id> ; Proxy: null)",
        "suppressed": [],
        "stack": [
          "com.amazonaws.services.s3.AmazonS3Client.deleteObjects(AmazonS3Client.java:2388)",
          "io.trino.hdfs.s3.TrinoS3FileSystem.deletePaths(TrinoS3FileSystem.java:819)",
          "io.trino.hdfs.s3.TrinoS3FileSystem.deleteFiles(TrinoS3FileSystem.java:799)",
          "io.trino.filesystem.hdfs.HdfsFileSystem.lambda$deleteFiles$2(HdfsFileSystem.java:139)",
          "io.trino.hdfs.authentication.NoHdfsAuthentication.doAs(NoHdfsAuthentication.java:25)",
          "io.trino.hdfs.HdfsEnvironment.doAs(HdfsEnvironment.java:134)",
          "io.trino.filesystem.hdfs.HdfsFileSystem.deleteFiles(HdfsFileSystem.java:137)",
          "io.trino.filesystem.manager.SwitchingFileSystem.deleteFiles(SwitchingFileSystem.java:88)",
          "io.trino.filesystem.tracing.TracingFileSystem.lambda$deleteFiles$1(TracingFileSystem.java:79)",
          "io.trino.filesystem.tracing.Tracing.lambda$withTracing$1(Tracing.java:38)",
          "io.trino.filesystem.tracing.Tracing.withTracing(Tracing.java:47)",
          "io.trino.filesystem.tracing.Tracing.withTracing(Tracing.java:37)",
          "io.trino.filesystem.tracing.TracingFileSystem.deleteFiles(TracingFileSystem.java:79)",
          "io.trino.plugin.deltalake.procedure.VacuumProcedure.doVacuum(VacuumProcedure.java:296)",
          "io.trino.plugin.deltalake.procedure.VacuumProcedure.vacuum(VacuumProcedure.java:135)",
          "io.trino.plugin.objectstore.MethodHandles$Translate.invoke(MethodHandles.java:83)",
          "java.base/java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:733)",
          "java.base/java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:758)",
          "io.trino.execution.CallTask.execute(CallTask.java:210)",
          "io.trino.execution.CallTask.execute(CallTask.java:70)",
...

Additional context and related issues

Release notes

(x) This is not user-visible or is docs only, and no release notes are required.
( ) Release notes are required. Please propose a release note for me.
( ) Release notes are required, with the following suggested text:

# Section
* Fix some things. ({issue}`issuenumber`)

@cla-bot cla-bot bot added the cla-signed label Jun 3, 2024
@github-actions github-actions bot added the delta-lake Delta Lake connector label Jun 3, 2024
@findepi findepi requested a review from ebyhr June 3, 2024 20:45
Copy link

github-actions bot commented Jul 4, 2024

This pull request has gone a while without any activity. Tagging the Trino developer relations team: @bitsondatadev @colebow @mosabua

@github-actions github-actions bot added the stale label Jul 4, 2024
@mosabua
Copy link
Member

mosabua commented Jul 4, 2024

fyi @findinpath

@findinpath
Copy link
Contributor

@ebyhr is there anything that we should do extra here?
I see you did in the meantime 7a863b3 to expose more information about the error.

Is storage an internal part of Trino (legalizing INTERNAL_ERROR) or rather an external part?

cc @electrum @mayankvadariya @pajaks

@github-actions github-actions bot removed the stale label Jul 5, 2024
@github-actions github-actions bot added the stale label Jul 26, 2024
@github-actions github-actions bot closed this Aug 19, 2024
@electrum electrum reopened this Aug 19, 2024
@@ -137,6 +138,9 @@ public void vacuum(
catch (TrinoException e) {
throw e;
}
catch (IOException e) {
throw new TrinoException(DELTA_LAKE_FILESYSTEM_ERROR, format("Failure when vacuuming %s.%s with retention %s: %s", schema, table, retention, e), e);
}
catch (Exception e) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be changed to catch (RuntimeException) as there are no other checked exceptions to catch.

@github-actions github-actions bot removed the stale label Aug 20, 2024
@mosabua mosabua added the stale-ignore Use this label on PRs that should be ignored by the stale bot so they are not flagged or closed. label Aug 26, 2024
@ebyhr ebyhr force-pushed the neville/delta-vacuum-error branch from 95722d0 to c166c83 Compare October 3, 2024 03:05
@trinodb trinodb deleted a comment from github-actions bot Oct 3, 2024
@trinodb trinodb deleted a comment from github-actions bot Oct 3, 2024
@ebyhr ebyhr merged commit 990416a into trinodb:master Oct 3, 2024
22 checks passed
@github-actions github-actions bot added this to the 460 milestone Oct 3, 2024
@mosabua mosabua mentioned this pull request Oct 3, 2024
1 task
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla-signed delta-lake Delta Lake connector stale-ignore Use this label on PRs that should be ignored by the stale bot so they are not flagged or closed.
Development

Successfully merging this pull request may close these issues.

5 participants