[Backport 3.5] Fix the regression of terms agg optimization - #20718

✅ This commit did not trigger a Mend scan. The conclusion status of this Security Check is 'Successful' because there was no commit in this branch that introduced new vulnerabilities with a score higher than the configured threshold. Review the Mend Security check associated with that commit.
Scan token: b693783e5088407e93af17d89e567d84