Skip to content

Conversation

@ivanjevtic-db
Copy link
Contributor

@ivanjevtic-db ivanjevtic-db commented Sep 12, 2024

What changes were proposed in this pull request?

Removed error condition _LEGACY_ERROR_TEMP_2011, removed dataTypeUnexpectedError, typeUnsupportedError, and replaced them with SparkException.internalError.

Why are the changes needed?

It is impossible to trigger the error from user space.
Here I changed dataTypeUnexpectedError to internalError, since typeSoFar argument will always be either:

  • NullType if this is the first row or
  • Some type which is returned by the inferField function(which is a valid type).

Here I changed typeUnsupportedError to internalError, since:

  • in this function call, the exception will be caught and
  • in this function call, a valid desiredType will always be passed.

Here and here I changed dataTypeUnexpectedError to internalError, since there is a type signature here which prevents child.dataType from being unexpected type.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Build passing.

Was this patch authored or co-authored using generative AI tooling?

No.

@github-actions github-actions bot added the SQL label Sep 12, 2024
@ivanjevtic-db ivanjevtic-db marked this pull request as draft September 12, 2024 10:13
@ivanjevtic-db ivanjevtic-db marked this pull request as ready for review September 12, 2024 10:13
@ivanjevtic-db ivanjevtic-db changed the title [SPARK-42846] Remove error class _LEGACY_ERROR_TEMP_2011 [SPARK-42846][SQL] Remove error class _LEGACY_ERROR_TEMP_2011 Sep 12, 2024
Copy link
Member

@MaxGekk MaxGekk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you update PR's title and description, and replace error class by error condition. See #48027

@ivanjevtic-db ivanjevtic-db changed the title [SPARK-42846][SQL] Remove error class _LEGACY_ERROR_TEMP_2011 [SPARK-42846][SQL] Remove error condition _LEGACY_ERROR_TEMP_2011 Sep 12, 2024
@MaxGekk
Copy link
Member

MaxGekk commented Sep 12, 2024

@ivanjevtic-db Do you have an account at OSS JIRA? If so, please, leave some comment in https://issues.apache.org/jira/browse/SPARK-42846 otherwise submit request:
Screenshot 2024-09-12 at 14 39 14

@ivanjevtic-db
Copy link
Contributor Author

I requested, will add comment when I get access.

@MaxGekk
Copy link
Member

MaxGekk commented Sep 12, 2024

+1, LGTM. Merging to master.
Thank you, @ivanjevtic-db.

@MaxGekk MaxGekk closed this in bc54eac Sep 12, 2024
@MaxGekk
Copy link
Member

MaxGekk commented Sep 12, 2024

@ivanjevtic-db Congratulations with your first contribution to Apache Spark!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants