Skip to content

Allow for specialized loops only up to 6 parameters#20412

Closed
a-kramarz wants to merge 3 commits intotrinodb:masterfrom
a-kramarz:fix-specialized-loops-generation
Closed

Allow for specialized loops only up to 6 parameters#20412
a-kramarz wants to merge 3 commits intotrinodb:masterfrom
a-kramarz:fix-specialized-loops-generation

Conversation

@a-kramarz
Copy link
Copy Markdown

@a-kramarz a-kramarz commented Jan 18, 2024

Description

#19385 (precisely 046b09d) introduced specialized loops generation for aggregations. The issue with that is the size of the generated method grows exponentially with the number of the operator parameters. It turns out that the org.ow2.asm library used internally by Airlift's bytecode lib has a hardcoded limit for the generated bytecode size (65535 bytes). An aggregation function with 7 parameters results in generated code of 88550 bytes in size and a MethodTooLargeException thrown during instantiation. Apparently there's no built-in aggregation with more than 6 parameters that would allow to catch this issue 🙂

Release notes

( ) This is not user-visible or is docs only, and no release notes are required.
(x) Release notes are required. Please propose a release note for me.
( ) Release notes are required, with the following suggested text:

# Section
* Fix some things. ({issue}`issuenumber`)

@cla-bot
Copy link
Copy Markdown

cla-bot bot commented Jan 18, 2024

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to cla@trino.io. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

@wendigo wendigo requested a review from martint January 18, 2024 16:27
@martint martint requested a review from dain January 18, 2024 20:33
@a-kramarz a-kramarz force-pushed the fix-specialized-loops-generation branch from 960256e to ac42eb8 Compare January 24, 2024 18:04
@cla-bot
Copy link
Copy Markdown

cla-bot bot commented Jan 24, 2024

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to cla@trino.io. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

@a-kramarz
Copy link
Copy Markdown
Author

I've added a test case reproducing the issue, kindly asking for a review @dain 🙂

@cla-bot
Copy link
Copy Markdown

cla-bot bot commented Jan 25, 2024

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to cla@trino.io. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

@github-actions
Copy link
Copy Markdown

This pull request has gone a while without any activity. Tagging the Trino developer relations team: @bitsondatadev @colebow @mosabua

@github-actions github-actions bot added the stale label Feb 15, 2024
@github-actions
Copy link
Copy Markdown

github-actions bot commented Mar 8, 2024

Closing this pull request, as it has been stale for six weeks. Feel free to re-open at any time.

@github-actions github-actions bot closed this Mar 8, 2024
@mosabua
Copy link
Copy Markdown
Member

mosabua commented Mar 19, 2024

Thank you for your pull request @a-kramarz

Unfortunately this artificial restriction based on implementation details such as libraries and bytecode generation limits is not the direction we should be going as a project. Our plan instead is to remove the old code and get rid of restrictions. In this case the real solution is to change the generated code to support more arguments. Currently there not much incentive to do that because we only have aggs with 4 arguments.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Development

Successfully merging this pull request may close these issues.

2 participants