Add TIMEDIFF and DATEDIFF functions. (#131)

[Yury-Fridlyand]

Signed-off-by: Yury-Fridlyand [email protected]

Description

I referred to MySQL docs and tried to reproduce MySQL v.8.0.30 behavior as a reference.

See team review and discussion on Bit-Quill#131

New function: TIMEDIFF.

Changes

Signature

(TIME, TIME) -> TIME

Future changes (TODOs):

1. Accept more/different types if and after [Discussion] Do we need to support more than 24 hours in TIME? #852
2. Accept strings

Test queries:

-- OpenSearch
SELECT TIMEDIFF('23:59:59', '13:00:00');
-- MySQL
SELECT TIMEDIFF('2007-12-31 23:59:59', '13:00:00');
SELECT TIMEDIFF('2007-12-31 23:59:59', '2004-01-01 13:00:00');
SELECT TIMEDIFF('2000-01-01 00:00:00', now());
SELECT TIMEDIFF(DATE('2008-12-12'), DATE('2008-11-15'));

New function: DATEDIFF.

Changes

Calculates the difference of date part of given values.

Fully compliant with MySQL.

Signature

(DATE/DATETIME/TIMESTAMP/TIME, DATE/DATETIME/TIMESTAMP/TIME) -> LONG

Future changes (TODOs):

Accept strings if it gives performance gain

Test queries:

SELECT DATEDIFF(TIMESTAMP('2000-01-02 00:00:00'), TIMESTAMP('2000-01-01 23:59:59'));
SELECT DATEDIFF(TIME('23:59:59'), TIMESTAMP('2004-01-01 00:00:00'));
SELECT DATEDIFF(TIME('23:59:59'), TIME('00:00:00'));
SELECT DATEDIFF(TIMESTAMP('2000-01-02 00:00:00'), DATETIME('2000-01-01 23:59:59'));

Test data

I found that first 6 rows from date0, time0, time1, datetime0 are good for testing - these columns have different data types in MySQL. In OpenSearch SQL all [date][time] columns have timestamp type, so I use CAST for clear testing.

data

mysql> show fields from Calcs where field IN ('date0', 'time0', 'time1', 'datetime0');
+-----------+-----------+------+-----+---------+-------+
| Field     | Type      | Null | Key | Default | Extra |
+-----------+-----------+------+-----+---------+-------+
| date0     | date      | YES  |     | NULL    |       |
| time0     | datetime  | YES  |     | NULL    |       |
| time1     | time      | YES  |     | NULL    |       |
| datetime0 | timestamp | YES  |     | NULL    |       |
+-----------+-----------+------+-----+---------+-------+
4 rows in set (0.00 sec)
mysql> select date0, time0, time1, datetime0 from calcs;
+------------+---------------------+----------+---------------------+
| date0      | time0               | time1    | datetime0           |
+------------+---------------------+----------+---------------------+
| 2004-04-15 | 1899-12-30 21:07:32 | 19:36:22 | 2004-07-09 10:17:35 |
| 1972-07-04 | 1900-01-01 13:48:48 | 02:05:25 | 2004-07-26 12:30:34 |
| 1975-11-12 | 1900-01-01 18:21:08 | 09:33:31 | 2004-08-02 07:59:23 |
| 2004-06-04 | 1900-01-01 18:51:48 | 22:50:16 | 2004-07-05 13:14:20 |
| 2004-06-19 | 1900-01-01 15:01:19 | NULL     | 2004-07-28 23:30:22 |
| NULL       | 1900-01-01 08:59:39 | 19:57:33 | 2004-07-22 00:30:23 |
| NULL       | 1900-01-01 07:37:48 | NULL     | 2004-07-28 06:54:50 |
| NULL       | 1900-01-01 19:45:54 | 19:48:23 | 2004-07-12 17:30:16 |
| NULL       | 1900-01-01 09:00:59 | 22:20:14 | 2004-07-04 22:49:28 |
| NULL       | 1900-01-01 20:36:00 | NULL     | 2004-07-23 21:13:37 |
| NULL       | 1900-01-01 01:31:32 | 00:05:57 | 2004-07-14 08:16:44 |
| NULL       | 1899-12-30 22:15:40 | 04:40:49 | 2004-07-25 15:22:26 |
| NULL       | 1900-01-01 13:53:46 | 04:48:07 | 2004-07-17 14:01:56 |
| NULL       | 1900-01-01 04:57:51 | NULL     | 2004-07-19 22:21:31 |
| NULL       | 1899-12-30 22:42:43 | 18:58:41 | 2004-07-31 11:57:52 |
| NULL       | 1899-12-30 22:24:08 | NULL     | 2004-07-14 07:43:00 |
| NULL       | 1900-01-01 11:58:29 | 12:33:57 | 2004-07-28 12:34:28 |
+------------+---------------------+----------+---------------------+
17 rows in set (0.00 sec)

Check List

• New functionality includes testing.
  • All tests pass, including unit test, integration test and doctest
• New functionality has been documented.
  • New functionality has javadoc added
  • New functionality has user manual doc added
• Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 62.76%. Comparing base (1b58f7d) to head (9ed9529).
⚠️ Report is 832 commits behind head on main.

❌ Your project status has failed because the head coverage (62.76%) is below the target coverage (99.00%). You can increase the head coverage or adjust the target coverage.

❗ There is a different number of reports uploaded between BASE (1b58f7d) and HEAD (9ed9529). Click for more details.

HEAD has 2 uploads less than BASE

Flag	BASE (1b58f7d)	HEAD (9ed9529)
sql-engine	2	0

Additional details and impacted files

@@              Coverage Diff              @@
##               main    #1195       +/-   ##
=============================================
- Coverage     98.32%   62.76%   -35.56%     
=============================================
  Files           344       10      -334     
  Lines          8762      658     -8104     
  Branches        554      119      -435     
=============================================
- Hits           8615      413     -8202     
- Misses          142      192       +50     
- Partials          5       53       +48     

Flag	Coverage Δ
query-workbench	62.76% <ø> (?)
sql-engine	?

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[VenkatManuru]

Hi

I don't see these functions available in latest version https://opensearch.org/docs/latest/search-plugins/sql/functions/#date-and-time. Can you please confirm if these are available? If not, by when these will be made available?

[Yury-Fridlyand]

@VenkatManuru,
It was released in 2.5.0. Updates to documentation on the website are delivered slowly, it is not yet updated, unfortunately.
Please, refer to docs in the SQL plugin repo, it is always up-to-date.