Skip to content

Conversation

@dai-chen
Copy link
Collaborator

@dai-chen dai-chen commented Oct 8, 2025

Description

This PR adds per_second() support to the timechart command by implementing Option 3 (Eval Transformation). As a short-term solution, it extends the Timechart AST to recognize the per_second function and rewrites into the equivalent math formula shown in the examples below. See issue #4350 for background and alternatives.

Examples

-- Original
source=events
| timechart span=5m per_second(packets)

-- Rewritten
source=logs
| timechart span=5m sum(packets) as `per_second(packets)`
| eval `per_second(packets)` = `per_second(packets)` / 300   -- 5 minutes = 300 seconds

For spans whose length varies with the calendar (month/quarter/year), the number of seconds depends on actual start/end timestamps (e.g., September has 30 days; October has 31 days; February in a leap year has 29 days). To ensure correctness, the rewrite computes the exact bucket length dynamically:

-- Example: span=2mon
source=logs
| timechart span=2mon sum(packets) as "per_second(packets)"
| eval `per_second(packets)` = `per_second(packets)` /
    timestampdiff(
      SECOND,
      @timestamp,                                  -- span start
      timestampadd(MONTH, 2, @timestamp)           -- span end   (start + 2 months)
    )

Related Issues

Resolves partially #4350.

Check List

  • New functionality includes testing.
  • New functionality has been documented.
  • New functionality has javadoc added.
  • New functionality has a user manual doc added.
  • New PPL command checklist all confirmed.
  • API changes companion pull request created.
  • Commits are signed per the DCO using --signoff or -s.
  • Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Chen Dai <[email protected]>
Signed-off-by: Chen Dai <[email protected]>
@dai-chen dai-chen self-assigned this Oct 8, 2025
@dai-chen dai-chen added enhancement New feature or request PPL Piped processing language backport 2.19-dev labels Oct 8, 2025
Copy link
Collaborator

@RyanL1997 RyanL1997 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @dai-chen , thanks for the change.


return eval(
timechart(AstDSL.alias(perFunc.aggName, sum(perFunc.aggArg))),
let(perFunc.aggName).multiply(perFunc.seconds).dividedBy(spanSeconds));
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens if the span evaluates to 0 seconds (e.g., with millisecond spans or edge cases)? Should there be validation or error handling for division by zero?

Copy link
Collaborator Author

@dai-chen dai-chen Oct 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. Let me check if timechart has the validation or not. Thanks!

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I quick tested timechart command and found negative span value is not allowed in our grammar. But zero-span validation is missing. Let me create an issue and raise separate PR to fix.

opensearchsql> source=test_data_2023 | timechart span=0m per_second(packets);
TransportError(500, 'SearchPhaseExecutionException', {'error': 
{'reason':'Error occurred in OpenSearch engine: all shards failed', 'details': 'Shard[0]:
  java.lang.IllegalArgumentException: Zero or negative time interval not supported\n\n
    For more details, please send request for Json format to see the raw response from OpenSearch engine.',
      'type': 'SearchPhaseExecutionException'}, 'status': 400})

opensearchsql> source=test_data_2023 | timechart span=-1m per_second(packets);
{'reason': 'Invalid Query', 'details': "[-] is not a valid term at this part of the query: '...23
  | timechart span=-' <-- HERE. extraneous input '-' expecting {SPANLENGTH, INTEGER_LITERAL}",
    'type': 'SyntaxCheckException'}

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Created issue: #4527. Thanks!

ykmr1224
ykmr1224 previously approved these changes Oct 10, 2025
Signed-off-by: Chen Dai <[email protected]>
@dai-chen dai-chen merged commit 4d416db into opensearch-project:main Oct 13, 2025
33 checks passed
opensearch-trigger-bot bot pushed a commit that referenced this pull request Oct 13, 2025
Add per_second() support to the timechart command by implementing Option 3 (Eval Transformation).

---------

Signed-off-by: Chen Dai <[email protected]>
(cherry picked from commit 4d416db)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
dai-chen pushed a commit that referenced this pull request Oct 15, 2025
)

Add per_second() support to the timechart command by implementing Option 3 (Eval Transformation).

---------


(cherry picked from commit 4d416db)

Signed-off-by: Chen Dai <[email protected]>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Signed-off-by: Lantao Jin <[email protected]>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Lantao Jin <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport 2.19-dev enhancement New feature or request PPL Piped processing language

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants