Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: try drop old udf - DON'T MERGE #17264

Closed
wants to merge 1 commit into from

Conversation

TCeason
Copy link
Collaborator

@TCeason TCeason commented Jan 13, 2025

I hereby agree to the terms of the CLA available at: https://docs.databend.com/dev/policies/cla/

Summary

In earlier version, udf serialize as json. In the current version, we can not drop/list these udfs.

show user functions;
2001=>InvalidReply: source:(PbDecodeError: failed to decode Protobuf message: buffer underflow; when:(decode value of __fd_udfs/tn3ftqihs/plusp)) while list UDFs

drop function IF EXISTS plusp;
2001=>InvalidReply: source:(PbDecodeError: failed to decode Protobuf message: buffer underflow; when:(decode value of __fd_udfs/tn3ftqihs/plusp))

So if drop udf return err, we directly drop the kv.

Tests

  • Unit Test
  • Logic Test
  • Benchmark Test
  • No Test - Explain why

Type of change

  • Bug Fix (non-breaking change which fixes an issue)
  • New Feature (non-breaking change which adds functionality)
  • Breaking Change (fix or feature that could cause existing functionality not to work as expected)
  • Documentation Update
  • Refactoring
  • Performance Improvement
  • Other (please describe):

This change is Reviewable

@github-actions github-actions bot added the pr-refactor this PR changes the code base without new features or bugfix label Jan 13, 2025
@TCeason TCeason marked this pull request as draft January 13, 2025 09:16
@TCeason TCeason marked this pull request as ready for review January 13, 2025 10:09
Copy link
Member

@drmingdrmer drmingdrmer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 2 of 2 files at r1, all commit messages.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on @TCeason)


src/query/management/src/udf/udf_mgr.rs line 168 at r1 (raw file):

        } else {
            self.try_drop_old_udf(udf_name).await?;
            Ok(None)

To this method, Ok(None) means nothing is removed.

If it's meant to remove an obsolete udf, the caller should call try_drop_old_udf() directly, instead of delegate to try_drop_old_udf() in this method.

Code quote:

            self.try_drop_old_udf(udf_name).await?;
            Ok(None)

@TCeason TCeason added this pull request to the merge queue Jan 13, 2025
@TCeason
Copy link
Collaborator Author

TCeason commented Jan 13, 2025

Reviewed 2 of 2 files at r1, all commit messages.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on @TCeason)

src/query/management/src/udf/udf_mgr.rs line 168 at r1 (raw file):

        } else {
            self.try_drop_old_udf(udf_name).await?;
            Ok(None)

To this method, Ok(None) means nothing is removed.

If it's meant to remove an obsolete udf, the caller should call try_drop_old_udf() directly, instead of delegate to try_drop_old_udf() in this method.

Code quote:

            self.try_drop_old_udf(udf_name).await?;
            Ok(None)

Get so after we drop the old udf we should remove this logic

@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Jan 13, 2025
@TCeason TCeason added this pull request to the merge queue Jan 13, 2025
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Jan 13, 2025
@TCeason TCeason added this pull request to the merge queue Jan 14, 2025
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Jan 14, 2025
} else {
self.try_drop_old_udf(udf_name).await?;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need a log here to know which old udf is dropped.

@TCeason TCeason marked this pull request as draft January 14, 2025 00:40
@TCeason TCeason added the ci-cloud Build docker image for cloud test label Jan 14, 2025
@TCeason TCeason marked this pull request as ready for review January 14, 2025 02:01
@TCeason TCeason added ci-cloud Build docker image for cloud test and removed ci-cloud Build docker image for cloud test labels Jan 14, 2025
Copy link
Contributor

Docker Image for PR

  • tag: pr-17264-4b65245-1736822418

note: this image tag is only available for internal use,
please check the internal doc for more details.

@TCeason TCeason added ci-cloud Build docker image for cloud test and removed ci-cloud Build docker image for cloud test labels Jan 14, 2025
Copy link
Contributor

Docker Image for PR

  • tag: pr-17264-722e28e-1736846581

note: this image tag is only available for internal use,
please check the internal doc for more details.

In eariler version, udf is serialize as json. In current version we can not drop/list these udfs.

show user functions;
2001=>InvalidReply: source:(PbDecodeError: failed to decode Protobuf message: buffer underflow; when:(decode value of __fd_udfs/tn3ftqihs/plusp))
while list UDFs

drop function IF EXISTS plusp;
2001=>InvalidReply: source:(PbDecodeError: failed to decode Protobuf message: buffer underflow; when:(decode value of __fd_udfs/tn3ftqihs/plusp))

So if drop udf return err, we directly drop the kv.
@TCeason TCeason removed the ci-cloud Build docker image for cloud test label Jan 14, 2025
@TCeason TCeason added the ci-cloud Build docker image for cloud test label Jan 14, 2025
@BohuTANG BohuTANG changed the title refactor: try drop old udf refactor: try drop old udf - DON'T MERGE Jan 14, 2025
Copy link
Contributor

Docker Image for PR

  • tag: pr-17264-4a9ec00-1736851907

note: this image tag is only available for internal use,
please check the internal doc for more details.

@TCeason
Copy link
Collaborator Author

TCeason commented Jan 14, 2025

Already normal. Close pr.

@TCeason TCeason closed this Jan 14, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ci-cloud Build docker image for cloud test pr-refactor this PR changes the code base without new features or bugfix
Projects
None yet
Development

Successfully merging this pull request may close these issues.

bug: failed to user functions
4 participants