Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix string literal length for compressed_data_info #7187

Merged
merged 1 commit into from
Aug 7, 2024

Conversation

mkindahl
Copy link
Contributor

@mkindahl mkindahl commented Aug 7, 2024

The function compressed_data_info returns a record containing name
but copies the data from string literal. Since heap_form_tuple expects a
name value as source as well, it will copy too much data from the
source, leading to ASAN complaining about copying outside the allocated
range of data.

The problem is fixed by NameData for each
source string, ensuring that each of the string literals has sufficient
data for being copied as names.

@mkindahl mkindahl self-assigned this Aug 7, 2024
@mkindahl mkindahl added this to the TimescaleDB 2.17.0 milestone Aug 7, 2024
@svenklemm
Copy link
Member

cstring seems like an unusual return type choice, why not make it text?

Copy link

codecov bot commented Aug 7, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 81.81%. Comparing base (59f50f2) to head (b6b51e6).
Report is 288 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #7187      +/-   ##
==========================================
+ Coverage   80.06%   81.81%   +1.74%     
==========================================
  Files         190      203      +13     
  Lines       37181    38048     +867     
  Branches     9450     9858     +408     
==========================================
+ Hits        29770    31128    +1358     
+ Misses       2997     2960      -37     
+ Partials     4414     3960     -454     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@mkindahl
Copy link
Contributor Author

mkindahl commented Aug 7, 2024

cstring seems like an unusual return type choice, why not make it text?

It can be made text, but internally it is a string literal and I see no advantage to make it a text type. Since the convention seems to be for most other build-in PostgreSQL functions that return string literals to use cstring return type, it seems like the most natural choice.

For example:

mats=# select proname, proargtypes::regtype[],prorettype::regtype from pg_proc where prorettype = 'cstring'::regtype limit 10;
    proname    |    proargtypes     | prorettype 
---------------+--------------------+------------
 boolout       | [0:0]={boolean}    | cstring
 byteaout      | [0:0]={bytea}      | cstring
 charout       | [0:0]={"\"char\""} | cstring
 nameout       | [0:0]={name}       | cstring
 int2out       | [0:0]={smallint}   | cstring
 int2vectorout | [0:0]={int2vector} | cstring
 int4out       | [0:0]={integer}    | cstring
 regprocout    | [0:0]={regproc}    | cstring
 textout       | [0:0]={text}       | cstring
 tidout        | [0:0]={tid}        | cstring
(10 rows)

If you feel strongly about it, I can use a text type.

@mkindahl mkindahl force-pushed the fix-compressed-data-info branch 3 times, most recently from ccf2e23 to e5a4739 Compare August 7, 2024 07:30
@erimatnor
Copy link
Contributor

cstring seems like an unusual return type choice, why not make it text?

It can be made text, but internally it is a string literal and I see no advantage to make it a text type. Since the convention seems to be for most other build-in PostgreSQL functions that return string literals to use cstring return type, it seems like the most natural choice.

For example:

mats=# select proname, proargtypes::regtype[],prorettype::regtype from pg_proc where prorettype = 'cstring'::regtype limit 10;
    proname    |    proargtypes     | prorettype 
---------------+--------------------+------------
 boolout       | [0:0]={boolean}    | cstring
 byteaout      | [0:0]={bytea}      | cstring
 charout       | [0:0]={"\"char\""} | cstring
 nameout       | [0:0]={name}       | cstring
 int2out       | [0:0]={smallint}   | cstring
 int2vectorout | [0:0]={int2vector} | cstring
 int4out       | [0:0]={integer}    | cstring
 regprocout    | [0:0]={regproc}    | cstring
 textout       | [0:0]={text}       | cstring
 tidout        | [0:0]={tid}        | cstring
(10 rows)

If you feel strongly about it, I can use a text type.

Or, we could just make sure we return a name type since that is the definition from the start.

@svenklemm
Copy link
Member

cstring is part of the interface of out functions but outside of that it is not used, for those text is used. eg pg_catalog.version which returns a static string returns text

sven@test[434551]=# SELECT oid::regprocedure from pg_proc where prorettype = 'cstring'::regtype and proname not like '%out' and proname not like 'cstring_%';
 oid 
-----
(0 rows)

@erimatnor
Copy link
Contributor

cstring is part of the interface of out functions but outside of that it is not used, for those text is used. eg pg_catalog.version which returns a static string returns text

sven@test[434551]=# SELECT oid::regprocedure from pg_proc where prorettype = 'cstring'::regtype and proname not like '%out' and proname not like 'cstring_%';
 oid 
-----
(0 rows)

FWIW, this is kind of like an out function for compressed_data.

Copy link
Contributor

@erimatnor erimatnor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, don't care too much about the return type. This is just an internal function used mostly for debugging so cstring is fine I guess. Still, I would probably just have fixed the code to construct a NAME object instead of changing the function definition.

sql/updates/latest-dev.sql Outdated Show resolved Hide resolved
@mkindahl mkindahl changed the title Fix return type of compressed_data_info Fix string literal length for compressed_data_info Aug 7, 2024
@mkindahl
Copy link
Contributor Author

mkindahl commented Aug 7, 2024

cstring is part of the interface of out functions but outside of that it is not used, for those text is used. eg pg_catalog.version which returns a static string returns text

sven@test[434551]=# SELECT oid::regprocedure from pg_proc where prorettype = 'cstring'::regtype and proname not like '%out' and proname not like 'cstring_%';
 oid 
-----
(0 rows)

Switched to using name and just ensuring that the source buffer was big enough.

@mkindahl mkindahl force-pushed the fix-compressed-data-info branch 2 times, most recently from 57fe2b7 to 53829e5 Compare August 7, 2024 09:35
@mkindahl mkindahl force-pushed the fix-compressed-data-info branch 2 times, most recently from 6a49afa to c1aab64 Compare August 7, 2024 12:03
The function `compressed_data_info` returns a record containing `name`
but copies the data from string literal. Since `heap_form_tuple`
expects a `name` value as source as well, it will copy too much data
from the source, leading to ASAN complaining about copying outside the
allocated range of data.

The problem is fixed by using `NameData` for each source string,
ensuring that each of the string literals has sufficient data for being
copied as names.
@mkindahl mkindahl merged commit 6ed3e2b into timescale:main Aug 7, 2024
37 of 38 checks passed
@mkindahl mkindahl deleted the fix-compressed-data-info branch August 7, 2024 14:22
pallavisontakke added a commit to pallavisontakke/timescaledb that referenced this pull request Sep 20, 2024
This release contains performance improvements and bug fixes since
the 2.16.1 release. We recommend that you upgrade at the next
available opportunity.

**Features**
* timescale#6882: Allow DELETE on the compressed chunks without decompression.
* timescale#7033 Use MERGE statement on CAgg Refresh
* timescale#7126: Add functions to show the compression information.
* timescale#7147: Vectorize partial aggregation for `sum
* timescale#7204: Track additional extensions in telemetry.
* timescale#7207: Refactor the `decompress_batches_scan` functions for easier maintenance.
* timescale#7209: Add a function to drop the `osm` chunk.

**Bugfixes**
* timescale#7187: Fix the string literal length for the `compressed_data_info` function.
* timescale#7191: Fix creating default indexes on chunks when migrating the data.
* timescale#7195: Fix the `segment by` and `order by` checks when dropping a column from a compressed hypertable.
* timescale#7201: Use the generic extension description when building `apt` and `rpm` loader packages.
* timescale#7227: Add an index to the `compression_chunk_size` catalog table.
* timescale#7229: Fix the foreign key constraints where the index and the constraint column order are different.
* timescale#7230: Do not propagate the foreign key constraints to the `osm` chunk.
* timescale#7234: Release the cache after accessing the cache entry.
* timescale#7258 Force English in the pg_config command executed by cmake to avoid unexpected building errors
* timescale#7270 Fix memory leak in compressed DML batch filtering

**Thanks**
* @MiguelTubio for reporting and fixing a Windows build error
* @posuch for reporting the misleading extension description in the generic loader packages.
@pallavisontakke pallavisontakke mentioned this pull request Sep 20, 2024
pallavisontakke added a commit to pallavisontakke/timescaledb that referenced this pull request Sep 25, 2024
This release contains performance improvements and bug fixes since
the 2.16.1 release. We recommend that you upgrade at the next
available opportunity.

**Features**
* timescale#6882: Allow DELETE on the compressed chunks without decompression.
* timescale#7033 Use MERGE statement on CAgg Refresh
* timescale#7126: Add functions to show the compression information.
* timescale#7147: Vectorize partial aggregation for `sum
* timescale#7204: Track additional extensions in telemetry.
* timescale#7207: Refactor the `decompress_batches_scan` functions for easier maintenance.
* timescale#7209: Add a function to drop the `osm` chunk.

**Bugfixes**
* timescale#7187: Fix the string literal length for the `compressed_data_info` function.
* timescale#7191: Fix creating default indexes on chunks when migrating the data.
* timescale#7195: Fix the `segment by` and `order by` checks when dropping a column from a compressed hypertable.
* timescale#7201: Use the generic extension description when building `apt` and `rpm` loader packages.
* timescale#7227: Add an index to the `compression_chunk_size` catalog table.
* timescale#7229: Fix the foreign key constraints where the index and the constraint column order are different.
* timescale#7230: Do not propagate the foreign key constraints to the `osm` chunk.
* timescale#7234: Release the cache after accessing the cache entry.
* timescale#7258 Force English in the pg_config command executed by cmake to avoid unexpected building errors
* timescale#7270 Fix memory leak in compressed DML batch filtering

**Thanks**
* @MiguelTubio for reporting and fixing a Windows build error
* @posuch for reporting the misleading extension description in the generic loader packages.
pallavisontakke added a commit to pallavisontakke/timescaledb that referenced this pull request Sep 26, 2024
This release contains performance improvements and bug fixes since
the 2.16.1 release. We recommend that you upgrade at the next
available opportunity.

**Features**
* timescale#6882: Allow DELETE on the compressed chunks without decompression.
* timescale#7033 Use MERGE statement on CAgg Refresh
* timescale#7126: Add functions to show the compression information.
* timescale#7147: Vectorize partial aggregation for `sum
* timescale#7204: Track additional extensions in telemetry.
* timescale#7207: Refactor the `decompress_batches_scan` functions for easier maintenance.
* timescale#7209: Add a function to drop the `osm` chunk.

**Bugfixes**
* timescale#7187: Fix the string literal length for the `compressed_data_info` function.
* timescale#7191: Fix creating default indexes on chunks when migrating the data.
* timescale#7195: Fix the `segment by` and `order by` checks when dropping a column from a compressed hypertable.
* timescale#7201: Use the generic extension description when building `apt` and `rpm` loader packages.
* timescale#7227: Add an index to the `compression_chunk_size` catalog table.
* timescale#7229: Fix the foreign key constraints where the index and the constraint column order are different.
* timescale#7230: Do not propagate the foreign key constraints to the `osm` chunk.
* timescale#7234: Release the cache after accessing the cache entry.
* timescale#7258 Force English in the pg_config command executed by cmake to avoid unexpected building errors
* timescale#7270 Fix memory leak in compressed DML batch filtering

**Thanks**
* @MiguelTubio for reporting and fixing a Windows build error
* @posuch for reporting the misleading extension description in the generic loader packages.
pallavisontakke added a commit to pallavisontakke/timescaledb that referenced this pull request Sep 30, 2024
This release contains performance improvements and bug fixes since
the 2.16.1 release. We recommend that you upgrade at the next
available opportunity.

**Features**
* timescale#6882: Allow DELETE on the compressed chunks without decompression.
* timescale#7033 Use MERGE statement on CAgg Refresh
* timescale#7126: Add functions to show the compression information.
* timescale#7147: Vectorize partial aggregation for `sum
* timescale#7204: Track additional extensions in telemetry.
* timescale#7207: Refactor the `decompress_batches_scan` functions for easier maintenance.
* timescale#7209: Add a function to drop the `osm` chunk.

**Bugfixes**
* timescale#7187: Fix the string literal length for the `compressed_data_info` function.
* timescale#7191: Fix creating default indexes on chunks when migrating the data.
* timescale#7195: Fix the `segment by` and `order by` checks when dropping a column from a compressed hypertable.
* timescale#7201: Use the generic extension description when building `apt` and `rpm` loader packages.
* timescale#7227: Add an index to the `compression_chunk_size` catalog table.
* timescale#7229: Fix the foreign key constraints where the index and the constraint column order are different.
* timescale#7230: Do not propagate the foreign key constraints to the `osm` chunk.
* timescale#7234: Release the cache after accessing the cache entry.
* timescale#7258 Force English in the pg_config command executed by cmake to avoid unexpected building errors
* timescale#7270 Fix memory leak in compressed DML batch filtering

**Thanks**
* @MiguelTubio for reporting and fixing a Windows build error
* @posuch for reporting the misleading extension description in the generic loader packages.
pallavisontakke added a commit to pallavisontakke/timescaledb that referenced this pull request Oct 8, 2024
This release contains performance improvements and bug fixes since
the 2.16.1 release. We recommend that you upgrade at the next
available opportunity.

**Features**
* timescale#6882: Allow DELETE on the compressed chunks without decompression.
* timescale#7033 Use MERGE statement on CAgg Refresh
* timescale#7126: Add functions to show the compression information.
* timescale#7147: Vectorize partial aggregation for `sum
* timescale#7200: Vectorize common aggregate functions like `min`, `max`, `sum`, `avg`, `stddev`, `variance` for compressed columns of arithmetic types, when there is grouping on segmentby columns or no grouping.
* timescale#7204: Track additional extensions in telemetry.
* timescale#7207: Refactor the `decompress_batches_scan` functions for easier maintenance.
* timescale#7209: Add a function to drop the `osm` chunk.
* timescale#7275: Add support for RETURNING clause for MERGE
* timescale#7295 Support ALTER TABLE SET ACCESS METHOD on hypertable

**Bugfixes**
* timescale#7187: Fix the string literal length for the `compressed_data_info` function.
* timescale#7191: Fix creating default indexes on chunks when migrating the data.
* timescale#7195: Fix the `segment by` and `order by` checks when dropping a column from a compressed hypertable.
* timescale#7201: Use the generic extension description when building `apt` and `rpm` loader packages.
* timescale#7227: Add an index to the `compression_chunk_size` catalog table.
* timescale#7229: Fix the foreign key constraints where the index and the constraint column order are different.
* timescale#7230: Do not propagate the foreign key constraints to the `osm` chunk.
* timescale#7234: Release the cache after accessing the cache entry.
* timescale#7258 Force English in the pg_config command executed by cmake to avoid unexpected building errors
* timescale#7270 Fix memory leak in compressed DML batch filtering
* timescale#7286: Fix index column check while searching for index
* timescale#7290 Add check for NULL offset for caggs built on top of caggs
* timescale#7301 Make foreign key behaviour for hypertables consistent
* timescale#7318: Fix chunk skipping range filtering
* timescale#7320 Set license specific extension comment in install script

**Thanks**
* @MiguelTubio for reporting and fixing a Windows build error
* @posuch for reporting the misleading extension description in the generic loader packages.
* @snyrkill for discovering and reporting the issue
@pallavisontakke pallavisontakke mentioned this pull request Oct 8, 2024
pallavisontakke added a commit that referenced this pull request Oct 8, 2024
This release adds support for PostgreSQL 17, significantly improves the
performance of continuous aggregate refreshes,
and contains performance improvements for analytical queries and delete
operations over compressed hypertables.
We recommend that you upgrade at the next available opportunity.

**Highlighted features in TimescaleDB v2.17.0**

* Full PostgreSQL 17 support for all existing features. TimescaleDB
v2.17 is available for PostgreSQL 14, 15, 16, and 17.

* Significant performance improvements for continuous aggregate
policies: continuous aggregate refresh is now using
`merge` instead of deleting old materialized data and re-inserting.

This update can decrease dramatically the amount of data that must be
written on the continuous aggregate in the
presence of a small number of changes, reduce the `i/o` cost of
refreshing a continuous aggregate, and generate fewer
  Write-Ahead Logs (`WAL`).
Overall, continuous aggregate policies will be more lightweight, use
less system resources, and complete faster.

* Increased performance for real-time analytical queries over compressed
hypertables:
we are excited to introduce additional Single Instruction, Multiple Data
(`SIMD`) vectorization optimization to our
engine by supporting vectorized execution for queries that group by
using the `segment_by` column(s) and
aggregate using the basic aggregate functions (`sum`, `count`, `avg`,
`min`, `max`).

Stay tuned for more to come in follow-up releases! Support for grouping
on additional columns, filtered aggregation,
  vectorized expressions, and `time_bucket` is coming soon.

* Improved performance of deletes on compressed hypertables when a large
amount of data is affected.

This improvement speeds up operations that delete whole segments by
skipping the decompression step.
It is enabled for all deletes that filter by the `segment_by` column(s).

**PostgreSQL 14 deprecation announcement**

We will continue supporting PostgreSQL 14 until April 2025. Closer to
that time, we will announce the specific
version of TimescaleDB in which PostgreSQL 14 support will not be
included going forward.

**Features**
* #6882: Allow delete of full segments on compressed chunks without
decompression.
* #7033: Use `merge` statement on continuous aggregates refresh.
* #7126: Add functions to show the compression information.
* #7147: Vectorize partial aggregation for `sum(int4)` with grouping on
`segment by` columns.
* #7204: Track additional extensions in telemetry.
* #7207: Refactor the `decompress_batches_scan` functions for easier
maintenance.
* #7209: Add a function to drop the `osm` chunk.
* #7275: Add support for the `returning` clause for `merge`.
* #7200: Vectorize common aggregate functions like `min`, `max`, `sum`,
`avg`, `stddev`, `variance` for compressed columns
of arithmetic types, when there is grouping on `segment by` columns or
no grouping.

**Bug fixes**
* #7187: Fix the string literal length for the `compressed_data_info`
function.
* #7191: Fix creating default indexes on chunks when migrating the data.
* #7195: Fix the `segment by` and `order by` checks when dropping a
column from a compressed hypertable.
* #7201: Use the generic extension description when building `apt` and
`rpm` loader packages.
* #7227: Add an index to the `compression_chunk_size` catalog table.
* #7229: Fix the foreign key constraints where the index and the
constraint column order are different.
* #7230: Do not propagate the foreign key constraints to the `osm`
chunk.
* #7234: Release the cache after accessing the cache entry.
* #7258: Force English in the `pg_config` command executed by `cmake` to
avoid the unexpected building errors.
* #7270: Fix the memory leak in compressed DML batch filtering.
* #7286: Fix the index column check while searching for the index.
* #7290: Add check for null offset for continuous aggregates built on
top of continuous aggregates.
* #7301: Make foreign key behavior for hypertables consistent.
* #7318: Fix chunk skipping range filtering.
* #7320: Set the license specific extension comment in the install
script.

**Thanks**
* @MiguelTubio for reporting and fixing the Windows build error.
* @posuch for reporting the misleading extension description in the
generic loader packages.
* @snyrkill for discovering and reporting the issue with continuous
aggregates built on top of continuous aggregates.
svenklemm added a commit that referenced this pull request Oct 8, 2024
This release adds support for PostgreSQL 17, significantly improves the
performance of continuous aggregate refreshes,
and contains performance improvements for analytical queries and delete
operations over compressed hypertables.
We recommend that you upgrade at the next available opportunity.

**Highlighted features in TimescaleDB v2.17.0**

* Full PostgreSQL 17 support for all existing features. TimescaleDB
v2.17 is available for PostgreSQL 14, 15, 16, and 17.

* Significant performance improvements for continuous aggregate
policies: continuous aggregate refresh is now using
`merge` instead of deleting old materialized data and re-inserting.

This update can decrease dramatically the amount of data that must be
written on the continuous aggregate in the
presence of a small number of changes, reduce the `i/o` cost of
refreshing a continuous aggregate, and generate fewer
  Write-Ahead Logs (`WAL`).
Overall, continuous aggregate policies will be more lightweight, use
less system resources, and complete faster.

* Increased performance for real-time analytical queries over compressed
hypertables:
we are excited to introduce additional Single Instruction, Multiple Data
(`SIMD`) vectorization optimization to our
engine by supporting vectorized execution for queries that group by
using the `segment_by` column(s) and
aggregate using the basic aggregate functions (`sum`, `count`, `avg`,
`min`, `max`).

Stay tuned for more to come in follow-up releases! Support for grouping
on additional columns, filtered aggregation,
  vectorized expressions, and `time_bucket` is coming soon.

* Improved performance of deletes on compressed hypertables when a large
amount of data is affected.

This improvement speeds up operations that delete whole segments by
skipping the decompression step.
It is enabled for all deletes that filter by the `segment_by` column(s).

**PostgreSQL 14 deprecation announcement**

We will continue supporting PostgreSQL 14 until April 2025. Closer to
that time, we will announce the specific
version of TimescaleDB in which PostgreSQL 14 support will not be
included going forward.

**Features**
* #6882: Allow delete of full segments on compressed chunks without
decompression.
* #7033: Use `merge` statement on continuous aggregates refresh.
* #7126: Add functions to show the compression information.
* #7147: Vectorize partial aggregation for `sum(int4)` with grouping on
`segment by` columns.
* #7204: Track additional extensions in telemetry.
* #7207: Refactor the `decompress_batches_scan` functions for easier
maintenance.
* #7209: Add a function to drop the `osm` chunk.
* #7275: Add support for the `returning` clause for `merge`.
* #7200: Vectorize common aggregate functions like `min`, `max`, `sum`,
`avg`, `stddev`, `variance` for compressed columns
of arithmetic types, when there is grouping on `segment by` columns or
no grouping.

**Bug fixes**
* #7187: Fix the string literal length for the `compressed_data_info`
function.
* #7191: Fix creating default indexes on chunks when migrating the data.
* #7195: Fix the `segment by` and `order by` checks when dropping a
column from a compressed hypertable.
* #7201: Use the generic extension description when building `apt` and
`rpm` loader packages.
* #7227: Add an index to the `compression_chunk_size` catalog table.
* #7229: Fix the foreign key constraints where the index and the
constraint column order are different.
* #7230: Do not propagate the foreign key constraints to the `osm`
chunk.
* #7234: Release the cache after accessing the cache entry.
* #7258: Force English in the `pg_config` command executed by `cmake` to
avoid the unexpected building errors.
* #7270: Fix the memory leak in compressed DML batch filtering.
* #7286: Fix the index column check while searching for the index.
* #7290: Add check for null offset for continuous aggregates built on
top of continuous aggregates.
* #7301: Make foreign key behavior for hypertables consistent.
* #7318: Fix chunk skipping range filtering.
* #7320: Set the license specific extension comment in the install
script.

**Thanks**
* @MiguelTubio for reporting and fixing the Windows build error.
* @posuch for reporting the misleading extension description in the generic loader packages.
* @snyrkill for discovering and reporting the issue with continuous
aggregates built on top of continuous aggregates.

---------

Signed-off-by: Pallavi Sontakke <[email protected]>
Signed-off-by: Yannis Roussos <[email protected]>
Signed-off-by: Sven Klemm <[email protected]>
Co-authored-by: Yannis Roussos <[email protected]>
Co-authored-by: atovpeko <[email protected]>
Co-authored-by: Sven Klemm <[email protected]>
kpan2034 pushed a commit to kpan2034/timescaledb that referenced this pull request Oct 11, 2024
This release adds support for PostgreSQL 17, significantly improves the
performance of continuous aggregate refreshes,
and contains performance improvements for analytical queries and delete
operations over compressed hypertables.
We recommend that you upgrade at the next available opportunity.

**Highlighted features in TimescaleDB v2.17.0**

* Full PostgreSQL 17 support for all existing features. TimescaleDB
v2.17 is available for PostgreSQL 14, 15, 16, and 17.

* Significant performance improvements for continuous aggregate
policies: continuous aggregate refresh is now using
`merge` instead of deleting old materialized data and re-inserting.

This update can decrease dramatically the amount of data that must be
written on the continuous aggregate in the
presence of a small number of changes, reduce the `i/o` cost of
refreshing a continuous aggregate, and generate fewer
  Write-Ahead Logs (`WAL`).
Overall, continuous aggregate policies will be more lightweight, use
less system resources, and complete faster.

* Increased performance for real-time analytical queries over compressed
hypertables:
we are excited to introduce additional Single Instruction, Multiple Data
(`SIMD`) vectorization optimization to our
engine by supporting vectorized execution for queries that group by
using the `segment_by` column(s) and
aggregate using the basic aggregate functions (`sum`, `count`, `avg`,
`min`, `max`).

Stay tuned for more to come in follow-up releases! Support for grouping
on additional columns, filtered aggregation,
  vectorized expressions, and `time_bucket` is coming soon.

* Improved performance of deletes on compressed hypertables when a large
amount of data is affected.

This improvement speeds up operations that delete whole segments by
skipping the decompression step.
It is enabled for all deletes that filter by the `segment_by` column(s).

**PostgreSQL 14 deprecation announcement**

We will continue supporting PostgreSQL 14 until April 2025. Closer to
that time, we will announce the specific
version of TimescaleDB in which PostgreSQL 14 support will not be
included going forward.

**Features**
* timescale#6882: Allow delete of full segments on compressed chunks without
decompression.
* timescale#7033: Use `merge` statement on continuous aggregates refresh.
* timescale#7126: Add functions to show the compression information.
* timescale#7147: Vectorize partial aggregation for `sum(int4)` with grouping on
`segment by` columns.
* timescale#7204: Track additional extensions in telemetry.
* timescale#7207: Refactor the `decompress_batches_scan` functions for easier
maintenance.
* timescale#7209: Add a function to drop the `osm` chunk.
* timescale#7275: Add support for the `returning` clause for `merge`.
* timescale#7200: Vectorize common aggregate functions like `min`, `max`, `sum`,
`avg`, `stddev`, `variance` for compressed columns
of arithmetic types, when there is grouping on `segment by` columns or
no grouping.

**Bug fixes**
* timescale#7187: Fix the string literal length for the `compressed_data_info`
function.
* timescale#7191: Fix creating default indexes on chunks when migrating the data.
* timescale#7195: Fix the `segment by` and `order by` checks when dropping a
column from a compressed hypertable.
* timescale#7201: Use the generic extension description when building `apt` and
`rpm` loader packages.
* timescale#7227: Add an index to the `compression_chunk_size` catalog table.
* timescale#7229: Fix the foreign key constraints where the index and the
constraint column order are different.
* timescale#7230: Do not propagate the foreign key constraints to the `osm`
chunk.
* timescale#7234: Release the cache after accessing the cache entry.
* timescale#7258: Force English in the `pg_config` command executed by `cmake` to
avoid the unexpected building errors.
* timescale#7270: Fix the memory leak in compressed DML batch filtering.
* timescale#7286: Fix the index column check while searching for the index.
* timescale#7290: Add check for null offset for continuous aggregates built on
top of continuous aggregates.
* timescale#7301: Make foreign key behavior for hypertables consistent.
* timescale#7318: Fix chunk skipping range filtering.
* timescale#7320: Set the license specific extension comment in the install
script.

**Thanks**
* @MiguelTubio for reporting and fixing the Windows build error.
* @posuch for reporting the misleading extension description in the generic loader packages.
* @snyrkill for discovering and reporting the issue with continuous
aggregates built on top of continuous aggregates.

---------

Signed-off-by: Pallavi Sontakke <[email protected]>
Signed-off-by: Yannis Roussos <[email protected]>
Signed-off-by: Sven Klemm <[email protected]>
Co-authored-by: Yannis Roussos <[email protected]>
Co-authored-by: atovpeko <[email protected]>
Co-authored-by: Sven Klemm <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants