5.5.2

Release 5.5.2

API Changes

New Features

1. Kernel: Add firebird SQL parser module and database type #33773

Enhancements

1. Kernel: Add arguments not null check when creating RouteUnit - #33382
2. Kernel: Add index columns not empty judgement for IndexColumnTokenGenerator - #33384
3. Kernel: Add binding to owner table - #33533
4. Kernel: Add binding to owner table - #33533
5. Kernel: Add WithAvailable interface and encrypt with, combine, insert select support checker - #34175
6. Metadata: Add load-table-metadata-batch-size props to concurrent load table metadata - #34009
7. DistSQL: Check inline expression when create sharding table rule with inline sharding algorithm - #33735
8. SQL Parser: Support parsing Doris BITXOR - #33258
9. SQL Parser: Support parsing Doris INSTR - #33289
10. SQL Parser: Support parsing Doris STRRIGHT - #33393
11. SQL Parser: Support parsing MySQL by adding non-reserved keywords in BaseRule.g4 file according to MySQL 8.4 doc - #33846
12. SQL Parser: Support parsing Doris EXTRACT_URL_PARAMETER - #33571
13. SQL Parser: Enhance create view, alter view, drop view sql parser - #34283
14. SQL Binder: Add sql bind logic for create table statement - #34074
15. SQL Binder: Support create index statement sql bind - #34112
16. SQL Parser: Support MySQL update with statement parse - #34126
17. SQL Binder: Remove TablesContext#findTableNames method and implement select order by, group by bind logic - #34123
18. SQL Binder: Support select with statement sql bind and add bind test case - #34141
19. SQL Binder: Support sql bind for select with current select projection reference - #34151
20. SQL Binder: Support alter table, drop table sql bind and add test case - #34154
21. SQL Binder: Support rename table statement sql bind and split segment bind to ddl and dml package - #34158
22. SQL Binder: Support copy statement sql bind and add bind test case - #34159
23. SQL Binder: Support truncate table sql bind and add test case - #34162
24. SQL Binder: Support create view, alter view, drop view sql bind logic - #34167
25. SQL Binder: Support load data and load xml sql bind and add test case - #34177
26. SQL Binder: Support optimize table sql bind and add test case - #34242
27. SQL Binder: Support show create table, show columns, show index statement bind - #34271
28. SQL Binder: Support deny user sql bind and add test case - #34279
29. SQL Binder: Support with segment bind check with UniqueCommonTableExpressionException - #34163
30. Storage: Support setting hive_conf_list, hive_var_list and sess_var_list for jdbcURL when connecting to HiveServer2 - #33749
31. Storage: Support connecting to HiveServer2 through database connection pools other than HikariCP - #33762
32. Storage: Partial support for connecting to embedded ClickHouse chDB - #33786
33. Transaction: Support savepoint/release savepoint TCL statements in jdbc adapter -#34173
34. Transaction: Bump the minimum Seata Client version for Seata AT integration to 2.2.0 - #33872
35. SQL Federation: Upgrade calcite version to 1.38.0 and update all license info in LICENSE file - #33279
36. JDBC: Add show database name for JDBC when execute SHOW COMPUTE NODES - #33437
37. JDBC: Support ZonedDateTime on ResultSet - #33660
38. Proxy: Add query parameters and check for MySQL kill processId - #33274
39. Proxy: Support table not exist exception for PostgreSQL proxy - #33885
40. Proxy Native: Change the Base Docker Image of ShardingSphere Proxy Native - #33263
41. Proxy Native: Support connecting to HiveServer2 with ZooKeeper Service Discovery enabled in GraalVM Native Image - #33768
42. Proxy Native: Support local transactions of ClickHouse under GraalVM Native Image - #33801
43. Proxy Native: Support Seata AT integration under Proxy Native in GraalVM Native Image - #33889
44. Sharding: Support GroupConcat function for aggregating multiple shards in MySQL, OpenGauss, Doris - #33808
45. Agent: Simplify the use of Agent's Docker Image - #33356
46. Mode: Support modifying Hikari-CP configurations via props in standalone mode #34185
47. Encrypt: Support insert statement rewrite use quote #34259
48. Infra: Support connecting to Firebird via jdbcUrl containing the absolute path to fdb - #34335

Bug Fixes

1. SQL Parser: Fixes LiteralExpressionSegment cast exception in SQL parser - #33332
2. SQL Parser: Fixes PostgreSQL and openGauss time extract function parse week and quarter error - #33564
3. SQL Parser: Fixes MySQL parse zone unreserved keyword error - #33720
4. SQL Parser: Fixes MySQL range parse error when use table owner - #33874
5. SQL Binder: Fixes table does not exist exception when use HintManager#setDatabaseName to transparent - #33370
6. SQL Binder: Use Multimap and CaseInsensitiveString to replace CaseInsensitiveMap for supporting MySQL multi table join with same table alias - #33303
7. SQL Binder: Fixes the combine statement cannot find the outer table when bind - #33357
8. SQL Binder: Fixes SQL performance issues caused by repeated subquery fetches - #33361
9. SQL Binder: Fixes the expression segment cannot find the outer table when binding - #34015
10. Storage: Fixes cannot connect to HiveServer2 using remote Hive Metastore Server - #33837
11. Proxy: Fixes BatchUpdateException when execute INSERT INTO ON DUPLICATE KEY UPDATE in proxy adapter - #33796
12. Proxy: Fixes "ALL PRIVILEGES ON DB.*" is not recognized during SELECT privilege verification for MySQL - #34037
13. Proxy: Fixes MySQL longblob wrong column type returned by proxy protocol - #34121
14. Proxy: Fixes MySQL proxy error if insert SQL contains more parameters not in insert values syntax - #34287
15. Sharding: Remove ShardingRouteAlgorithmException check logic temporarily to support different actual table name configuration - #33367
16. Sharding: Fixes SQL COUNT with GROUP BY to prevent incorrect row returns - #33380
17. Sharding: Fixes avg, sum, min, max function return empty data when no query result return - #33449
18. Encrypt: Fixes merge exception without encrypt rule in database - #33708
19. Encrypt: Use sql bind info in EncryptInsertPredicateColumnTokenGenerator to avoid wrong column table mapping - #34110
20. Mode: Fixes JDBCRepository improper ha...

5.5.1

API Changes

1. Authority: Mark privilege provider ALL_PERMITTED as deprecated and will be removed in future
2. DistSQL: Remove optional param usageCount from show storage units
3. Readwrite-splitting: Change dataSources to dataSourceGroups for YAML

New Features

1. Kernel: Add new doris, hive and presto sql parser module and database type

Enhancements

1. DistSQL: Check privilege when registering or altering storage unit
2. DistSQL: Check duplicate actual data nodes when creating or altering sharding table rule
3. DistSQL: Add like support for show storage units
4. DistSQL: Rollback if import database configuration failed
5. DistSQL: add table type to result set of show logical tables
6. SQL Parser: Support MySQL change replication with no server ids
7. SQL Parser: Support MySQL row and column aliases with ON DUPLICATE KEY UPDATE
8. SQL Parser: Support MySQL CALL sql with mysql prefix
9. SQL Parser: Support MySQL create loadable function returns int/dec
10. SQL Parser: Support parsing MySQL CREATE TABLESPACE/TABLE in Mysql-8
11. SQL Parser: Support parsing MySQL CREATE INDEX with ENGINE_ATTRIBUTE
12. SQL Parser: Support parsing MySQL LOAD DATA with @ variable
13. SQL Parser: Support parsing MySQL START REPLICA statement
14. SQL Parser: Support parsing MySQL json table function
15. SQL Parser: Add MySQL column not null SQL parsing
16. SQL Parser: Add MySQL charset SQL parsing
17. SQL Parser: Enhance mysql json function visit statement result
18. SQL Parser: Support PostgreSQL COLLATION and CONSTRAINT keywords
19. SQL Parser: Support PostgreSQL, openGauss function table and update from parse
20. SQL Parser: Support CTE with PostgreSQL and openGauss
21. SQL Parser: Optimize the timezone support of PostgreSQLDateValueParser.
22. SQL Parser: Support PostgreSQL limit and offset as null
23. SQL Parser: Enhance Oracle update statement as alias parse and add test case
24. SQL Parser: Support Oracle column length unit type SQL parsing
25. Metadata: Collect table type for PostgreSQL
26. Kernel: Support sql hint extract when sql contains dbeaver hint comment
27. Kernel: Add extract combine left select in extractFromSelectStatementWithoutProjection method
28. Kernel: Add assisted node to create and delete databases
29. Transaction: Use same transaction type in one transaction in JDBC adapter
30. Transaction: Support for switching transaction types
31. SQL Federation: Support sql federation bit_count function for mysql
32. SQL Federation: Improve atan and anan2 sql function for mysql
33. SQL Federation: Support federated query mysql bin function
34. SQL Federation: Support for federated query NOT operator
35. Proxy: Make the results of show tables in order
36. Proxy: Trigger metadata collection when creating and deleting database and table
37. Proxy: Optimize Agent to support collecting metrics data from multiple data sources when used with Driver
38. Proxy Native: Add more graalvm reachability metadata for caffeine cache
39. Sharding: Revise all local index for sharding table and add object uniqueness level spi to control index token generator
40. Sharding: Add NullsOrderType.LOW and NullsOrderType.HIGH to handle NULL order by in sharding feature
41. Sharding: Add inline sharding algorithms match actual data nodes check
42. Encrypt: Add unsupported check for combine statement with encrypt columns
43. Encrypt: Support select distinct(column) encrypt rewrite and refactor SubstitutableColumnNameToken build logic
44. Encrypt: Support like concat nested concat statement rewrite with encrypt feature
45. Pipeline: Add SHARDING_TOTAL_COUNT impl of JobExecutorServiceHandler to improve CPU core requirement
46. Pipeline: Support page query for inventory dumper and data consistency streaming query
47. Pipeline: Use case-insensitive identifiers to enhance the table metadata loader
48. Pipeline: Support primary key columns ordering for standard pipeline table metadata loader

Bug Fixes

1. DistSQL: Fix load single table error after creating logical data source
2. DistSQL: Fix set default single table storage unit to random failed
3. DistSQL: Fix set default single table storage unit to logical data source failed
4. DistSQL: Fix NPE when import metadata
5. SQL Parser: Support PostgreSQL do nothing conflict action
6. SQL Parser: Fix oracle nvl function and interval hour parse error
7. Metadata: Keep in-used storage node when unregister storage unit
8. Metadata: Fixed the problem of missing storage unit when registering storage unit
9. Kernel: Replace ThreadLocal with TransmittableThreadLocal in HintManager to support cross Thread usage
10. Kernel: Use ConnectionPropertiesParser spi to parse jdbcUrl to solve oracle url parse error
11. Kernel: Fix column count assert exception caused by postgresql system table judge
12. SQL Binder: Add TableAvailable interface for CloseStatementContext, MoveStatementContext, FetchStatementContext and fix sql rewrite test case
13. Transaction: Fix setSavepoint method invocation not cleaning up
14. Transaction: Fix savepoint manager not cleaned up in distributed transactions
15. Transaction: Fix transaction context not cleaned up when xa transaction is committed
16. Transaction: Fix the issue that cursor is not rewritten
17. SQL Federation: Fix federated query LocalDateTime conversion
18. SQL Federation: Fix push down sql execute error when sql contains chinese character with sql federation
19. SQL Federation: Fix federation query binary type data query
20. SQL Federation: Fix null result in federated query for a single projected column
21. SQL Federation: Fix sql federation unknown type exception caused by calcite wrong result type with bigint
22. Proxy: Support binary type bool value in PostgreSQL
23. Proxy: Support array type prepared param in PostgreSQL
24. Proxy: Support binary protocol value for text array in PostgreSQL
25. Proxy: Fix duplicate results when querying information_schema.SCHEMATA
26. Proxy: Fix incorrect results for querying information_schema.SCHEMATA
27. Proxy: Fix NPE when execute show table status
28. Proxy: Fix no database selected exception occurs when show tables from db
29. Proxy: Fix the error that the process does not exit after proxy startup fail
30. Proxy: Fix the error that the persisted system metadata was not cleared after the database was deleted in the PostgreSQL scenario
31. Proxy: Fix no database selected exception in the query SQL
32. JDBC: Resolve the issue where "zip file closed" in versions prior to SpringBoot 2.3.
33. JDBC: Fix the NPE when it does not contain a specified logic database in Driver.
34. Sharding: Fix Sharding column not tracked through aliases
35. Sharding: Fix alter view exception when config sharding rule and binding table rule
36. Sharding: Fix is need accumulate logic
37. Sharding: Fix the generated key column name case insensitivity error
38. Encrypt: Fix show create table wrong result with encrypt when data type contains float
39. Encrypt: Fix the issue where updating a non-encrypted table and using a subquery on an encrypted table.
40. Encrypt: Add insert select rewrite for encrypt
41. Pipeline: InventoryTaskSplitter compatible with BigInteger primary key
42. Pipeline: Fix default data source lost when create migration job

5.5.0

API Change

1. Proxy: Remove the outdated schemaName configuration of the Proxy
2. Kernel: Remove the sqlCommentParseEnabled configuration in SQL Parser Rule to reduce code complexity
3. SQL Parser: Consider removing sqlCommentParseEnabled config in SQLParser rule
4. Federation: Add allQueryUseSQLFederation config for sql federation

Bug Fix

1. Pipeline: Fix commit/rollback migration job doesn't drop related consistency check job when check job is not completed
2. Pipeline: Fix show consistency check status stop_time display
3. Proxy: Restore original databaseName in connectionSession after unicast
4. Proxy: Fix show tables can be executed without use database
5. Proxy: Fix the incorrect current database after unicast routing
6. Proxy: Fix the problem that show tables can be executed before use database
7. Readwrite-splitting: Fix check exception when using shadow data source
8. Governance: Fix thread blocking problem when create logic database for Etcd register center
9. Governance：Fix register storage units and create feature rules failure when use Standalone mode
10. MetaData: Fix database system schema is not built when create database
11. Metadata: Fix NPE of system schema builder rule
12. SQL Parser: Fix PostgreSQL NPE when parse columnRef
13. SQL Parser: Fix npe cause by parse MySQL select window statement
14. Sharding: Fix routing error when joining tables in uppercase
15. Sharding: Fix drop sharding table exception when table name is uppercase
16. Sharding: Fix generated key with upper case column name
17. Federation: Fix Object 'DUAL' not found exception when execute select 1 from dual with sql federation
18. Transaction: Fix xa auto commit in executeQuery
19. Proxy: Fix multi statements with specified database name

Enhancement

1. Pipeline: Improve CDC stability and performance
2. Pipeline: MemoryPipelineChannel supports configurable zero queue size for less memory consumption
3. Pipeline: Show dedicated error when mode type is not Cluster
4. Authority: Add isSuper option for user
5. Pipeline: Ignore error message in status DistSQL result on job cancelling
6. DistSQL: Use JSON format to output props in RQL
7. DistSQL: Optimize REFRESH DATABASE METADATA logic
8. DistSQL: SHOW COMPUTE NODES supports JDBC nodes
9. DistSQL: Optimize syntax of REFRESH DATABASE METADATA
10. DistSQL: Optimize the output of SHOW TABLE METADATA
11. Proxy: Add MySQL precompiled parameter verification to avoid turning on rewriteBatchedStatements=true when BenchmarkSQL connects to Proxy, causing an ArrayIndexOutOfBoundsException exception when the Proxy parameterCount exceeds 65535.
12. SQL Parser: Add EOF to throw exception when parse distsql rollback migration statement
13. SQL Parser: Support more plsql statement parse and add plsql parse assert logic
14. SQL Parser: Support parse index hint
15. SQL Parser: Support mysql intersect combine operation sql parse
16. SQL Parser: Supoort parse chinese white space for oracle
17. SQL Parser: Fix mysql TimeStampDiff function parse
18. SQL Parser: Fix sqlServer unqualified shorthand parsing
19. SQL Parser: Support sqlServer SEARCH unreserved word parsing
20. SQL Parser: Add sql server MEMBER unreserved word
21. Binder: Add ParameterMarkerSegmentBinder logic for Oracle MergeStatementBinder
22. Federation: Support mysql cross join statement for sql federation
23. Transaction: Add advice message in XATransactionCheckPrivilegeFailedException
24. Sharding: Validate duplicate sharding actual data nodes
25. Sharding: Support null condition value routing
26. Proxy: Fix mysql client multi statements option in protocol

New Feature

1. DistSQL: New syntax for query plugin (SPI) implementation
2. DistSQL: New syntax for managing SQL_TRANSLATOR rule

Refactor

1. JDBC: Merge shardingsphere-jdbc-core to shardingsphere-jdbc module
2. Proxy: Rename server.yaml to global.yaml, config- to database- and keep compatible with old config
3. Proxy: Rename config-xxx.yaml to database-xxx.yaml
4. Proxy: Optimize MySQLComStmtPrepareExecutor logic
5. Encrypt: Refactor encrypt interface and add encrypt algorithm metadata
6. Infra: Create new algorithm module in infra and move snowflake, uuid to this module
7. Infra: Modify unmarshal method to return a default object when yaml is empty
8. Infra: Optimize StandardParameterBuilder logic

5.4.1

New Features

1. Proxy: Supports dbcp and c3p0 connection pools
2. Metadata: Standalone mode adapts to metadata new structure
3. Governance: Governance supports register instance level data source

Enhancements

1. Mode: Improve Standalone mode JDBC type impl reset data on initialization
2. Scaling: Isolate ShardingSphereDataSource Standalone repository in pipeline
3. Scaling: Disable system-schema-metadata-enabled in pipeline
4. JDBC: Move jdbc core META-INF/services/java.sql.Driver from test to main
5. Scaling: Add algorithm columns in SHOW MIGRATION CHECK ALGORITHMS DistSQL result
6. Scaling: Add type_aliaes column in SHOW MIGRATION CHECK ALGORITHMS DistSQL result
7. Encrypt: Add duplicate name check for derived columns and logical columns
8. Encrypt: Deny DDL for cipher columns in Proxy
9. Encrypt: Add the default type for derived columns to varchar(4000)

Bug Fixes

1. Scaling: Fix get inventory position not correctly on breakpoint resuming when table names are similar
2. Scaling: Fix CDC importer not start on breakpoint resuming when first inventory task is finished
3. Single Table: Fix not switched active version when CREATE/DROP table modifies the Single rule configuration
4. JDBC: Fix JDBC memory leak with MySQL in the 5.4.0 version

Change Log

1. MILESTONE

5.4.0

API Changes

1. Metadata: Change sharding broadcast tables to global broadcast tables
2. JDBC: Remove exclamation mark (!) for global rules
3. DistSQL: Simplify keywords ASSISTED_QUERY, LIKE_QUERY in encrypt DistSQL
4. DistSQL: Optimize SQL_PARSER rule syntax
5. Encrypt: Adjust encryption yaml API to distinguish between encrypt, like, and assisted query configurations
6. Encrypt: Remove plain column and queryWithCipherColumn configuration in encrypt feature
7. Readwrite-splitting: Refactor read/write splitting api
8. Proxy: Remove property proxy-instance-type configuration
9. Proxy: Remove property proxy-backend-executor-suitable
10. Proxy: Remove property proxy-mysql-default-version
11. Scaling: Refactor commit rollback streaming to drop streaming
12. Sharding: Merge ShardingCacheRule into ShardingRule (Experimental)

New Features

1. DistSQL: New syntax to manage SQL_FEDERATION rule
2. Proxy: Support Unix Domain Socket

Enhancements

1. Scaling: CDC supports pure incremental mode
2. Scaling: CDC supports exporting data by transaction
3. Scaling: CDC supports MySQL and PostgreSQL
4. Scaling: CDC supports single table
5. Scaling: CDC supports all openGauss data types
6. Scaling: CDC supports replication reconnection
7. Scaling: Remove DataConsistencyCalculateAlgorithmChooser
8. Scaling: Improve performance of integer unique key table inventory data splitting
9. Scaling: Adjust process configuration default value to reduce resource consumption
10. Scaling: Auto refresh table metadata for migration
11. Scaling: Compatible with openGauss existing replication slot reuse when database not existing
12. Scaling: Show data consistency check status result should be empty when it's not completed
13. Scaling: Enable concurrent CRC32 match on source and target
14. Scaling: Pipeline job compatible with sharding rule audit strategy
15. Metadata: Refactor metadata persistence structure
16. Metadata: Optimize the process of loading single table metadata
17. Metadata: Support MySQL/PostgreSQL/openGauss system tables empty query
18. DistSQL: Add support for transactionalReadQueryStrategy for read/write splitting rule
19. DistSQL: Enhanced algorithm properties check
20. Transaction: Add privilege check
21. Transaction: Remove the TransactionTypeHolder and only create the current transaction manager
22. Parser: Support MySQL LOAD DATA and LOAD XML statement with single table or broadcast table
23. Parser: Improve the parsing support of high-priority SQL statements in the test results of the MySQL test program
24. Parser: Oracle dialect parser now supports Chinese comma
25. Encrypt: Support query of encrypt column in projection subquery when use encrypt feature
26. Kernel: Adds table existence metadata check for INSERT, DELETE, UPDATE and SELECT statements
27. JDBC: Implement batch execution for ShardingSphereStatement
28. Proxy: Frontend supports SSL/TLS
29. Proxy: Support Flush message for PostgreSQL/openGauss Proxy
30. Proxy: Support data type bit, bool for PostgreSQL Proxy

Bug Fixes

1. Scaling: Fix pipeline job failure status persistence and usage
2. Scaling: Fix CDC DELETE event Record.beforeList is null
3. Scaling: Fix openGauss mppdb decoding plugin single quote issue
4. Scaling: Fix execute engine not closed after job stopping
5. Scaling: Fix stop job before task starting
6. Metadata: Fix case sensitive issue when loading schema meta data with H2 database
7. Metadata: Fix "object not found" exception when config PostgreSQL/openGauss schema name as database name
8. DistSQL: Fix wrong result of check_table_metadata_enabled when execute SHOW DIST VARIABLE
9. Encrypt: Fix SQL rewrite exception when use PostgreSQL/openGauss encrypt like feature
10. Sharding: Support null sharding condition pass to sharding algorithm to allow user control null value route
11. Parser: Support BETWEEN AND expression parsing in MySQL Projection
12. Mask: Fix wrong mask result when config same value of from-x and to-y with KEEP_FROM_X_TO_Y
13. Infra: Fix ClassNotFoundException may occur when missing pgjdbc
14. Proxy: Fix MySQL packet out of order when client sending pipelining requests

Change Log

1. MILESTONE

5.3.2

API Changes

1. Proxy: Add property system-log-level, support dynamic change of log level by DistSQL.
2. DistSQL: Remove Hint-related DistSQL, users can use SQL Hint instead

New Features

1. Scaling: Support any type of column unique key table

Enhancements

1. Scaling: Use stream query for inventory dump and data consistency check
2. Scaling: Compatible with VARBINARY column type in MySQL binlog parsing
3. Scaling: Refactor AbstractSimplePipelineJob.execute to blocking
4. Scaling: Improve table records count calculation
5. Scaling: Support proxy sharding rule absent for migration job
6. Scaling: Add useServerPrepStmts=false for pipeline job on MySQL
7. Scaling: Improve datetime/time parsing for MySQL binlog
8. Scaling: Add global status for prepare stage
9. Scaling: Add netTimeoutForStreamingResults for pipeline job on MySQL
10. Authority: Support specifying password authentication method
11. Authority: Add md5 authentication support for openGauss protocol
12. Agent: Add more metrics for JDBC
13. Kernel: ShardingSphere Driver configuration supports Apollo
14. Kernel: Adjust SKIP_ENCRYPT_REWRITE SQL Hint to SKIP_SQL_REWRITE to support more scenarios
15. Kernel: Support openGauss EXPLAIN PERFORMANCE
16. Encrypt: Like supports concat function

Bug Fixes

1. Scaling: Fix unicode char and special char decoding for PostgreSQL incremental task
2. Scaling: Fix Migration not support PostgreSQL json type
3. DistSQL: CREATE SHARDING TABLE RULE supports NONE strategy
4. Kernel: Fix use system database error when data sources are empty
5. Kernel: Fix set worker-id does not take effect with Standalone mode
6. Kernel: Clear storage node information when delete readwrite-splitting and database discovery rules
7. Kernel: Fix the abnormal problem of Column index out of range in single table complex query
8. Kernel: Fix PostgreSQL like lower case failed.
9. Kernel: Fixed the exception of built-in metabase data collection when the front and back database types were inconsistent
10. Kernel: Fix the problem of routing error reporting under certain table names
11. Kernel: Fix MySQL create procedure parse error
12. Kernel: Fix union extract table name NPE
13. Kernel: Fix upper case table constraint not rewrite error
14. Kernel: Fix failed to parse PostgreSQL / openGauss SQL contains money type
15. Kernel: Fix PostgreSQL / openGauss positional parameter rule
16. Kernel: Fix PostgreSQL / openGauss failed to parse const with type cast
17. Kernel: Fix Chinese characters encode exception when execute select with sql federation engine
18. Kernel: Fix IndexOutOfBoundsException when execute set variable statement int jdbc adapter
19. Kernel: Fix index does not exist exception when execute drop index statement
20. Proxy: Properly handle number sign in MySQL Proxy binary protocol
21. Proxy: Fix PostgreSQL Proxy failed to handle bytea data type
22. Proxy: Fix PostgreSQL Proxy failed to parse time value with microseconds
23. Proxy: Fix PostgreSQL protocol codec for date type in binary format
24. Proxy: Fix possible CCE PostgreSQLInt2BinaryProtocolValue
25. Proxy: Fix possible error when client pass quoted charset to PostgreSQL/openGauss Proxy

Change Log

1. MILESTONE

5.3.1

New Features

1. Kernel: Add new data masking, dynamic data masking features, and built-in data masking algorithms
2. Scaling: Basic support of CDC feature
3. DistSQL: Add masking rule related DistSQL

Enhancements

1. Kernel: Cluster mode avoids secondary refresh of metadata
2. Kernel: SHOW COMPUTE NODES supports displaying the version number of each instance
3. Kernel: System database add cluster information table
4. Kernel: Standalone mode persistent metadata supports MySQL
5. Kernel: SQL HINT performance improvement
6. Kernel: Restore routing to the specified database with Hint
7. Encrypt: Supports underscore wildcards for Encrypt CharDigestLikeEncryptAlgorithm
8. Kernel: Support SQL federation SELECT NULLS LAST/FIRST statement
9. Kernel: Refactor encrypt integration test logic and add more test cases
10. Kernel: Add salt props for MD5MaskAlgorithm, MD5EncryptAlgorithm
11. Kernel: Refactor ShardingConditionEngine to support SPI configuration
12. DistSQL: Support defining column level QUERY_WITH_CIPHER_COLUMN when CREATE ENCRYPT RULE
13. DistSQL: Add algorithm type check for CREATE SHARDING TABLE RULE

Bug Fixes

1. Fix the problem of ZooKeeper cluster error reporting when ShardingSphere connects to Kubernetes
2. Kernel: Fix use Consul in cluster mode start up failure
3. DB Discovery: Close heartbeat job when drop discovery rule
4. Kernel: Fix wrong decide result when execute same sharding condition subquery with SQL federation
5. Kernel: Fix priority problem of UNION, INTERSECT, EXCEPT set operation in SQL Federation for PostgreSQL and openGuass dialect
6. Kernel: Fix create view index out of range exception when view contains set operator
7. Kernel: Add XA resource exceeds length check
8. Kernel: Fix transaction support for spring requires_new
9. Encrypt: Fix AESEncryptAlgorithm decrypt exception when config char type with PostgreSQL and openGauss
10. Encrypt: Fix abnormal expansion result for shorthand when encrypt subquery contains an alias
11. Kernel: Fix unsigned flag of column metadata was not serialized
12. Kernel: Fix PostgreSQL / openGauss select fetch parsing issue to support federation execution engine
13. Proxy: Fix packet sequence ID may be incorrect if error occurred in MySQL Proxy
14. Proxy: Fix error occur in Proxy when using PostgreSQL composite type
15. Proxy: Set proper column definition flag for MySQL COM_STMT_PREPARE
16. Proxy: When querying PG metadata through Proxy and the result set is empty, the labels are lost

Change Log

1. MILESTONE

5.3.0

API Changes

1. DistSQL: Refactor syntax API, please refer to the user manual
2. Proxy: Change the configuration style of global rule, remove the exclamation mark
3. Proxy: Allow zero-configuration startup, enable the default account root/root when there is no Authority configuration
4. Proxy: Remove the default logback.xml and use API initialization
5. JDBC: Remove the Spring configuration and use Driver + YAML configuration instead

Enhancements

1. DistSQL: New syntax REFRESH DATABASE METADATA, refresh logic database metadata
2. Kernel: Support DistSQL REFRESH DATABASE METADATA to load configuration from the governance center and rebuild MetaDataContext
3. Support postgresql/openGauss setting transaction isolation level
4. Scaling: Increase inventory task progress update frequency
5. Scaling: DATA_MATCH consistency check support breakpoint resume
6. Scaling: Support drop consistency check job via DistSQL
7. Scaling: Rename column from sharding_total_count to job_item_count in job list DistSQL response
8. Scaling: Add sharding column in incremental task SQL to avoid broadcast routing
9. Scaling: Sharding column could be updated when generating SQL
10. Scaling: Improve column value reader for DATA_MATCH consistency check
11. DistSQL: Encrypt DistSQL syntax optimization, support like query algorithm
12. DistSQL: Add properties value check when REGISTER STORAGE UNIT
13. DistSQL: Remove useless algorithms at the same time when DROP RULE
14. DistSQL: EXPORT DATABASE CONFIGURATION supports broadcast tables
15. DistSQL: REGISTER STORAGE UNIT supports heterogeneous data sources
16. Encrypt: Support Encrypt LIKE feature
17. Automatically start distributed transactions when executing DML statements across multiple shards
18. Kernel: Support client \d for PostgreSQL and openGauss
19. Kernel: Support select group by, order by statement when column contains null values
20. Kernel: Support parse RETURNING clause of PostgreSQL/openGauss Insert
21. Kernel: SQL HINT performance improvement
22. Kernel: Support mysql case when then statement parse
23. Kernel: Supporting data source level heterogeneous database gateway
24. (Experimental) Sharding: Add sharding cache plugin
25. Proxy: Support more PostgreSQL datetime formats
26. Proxy: Support MySQL COM_RESET_CONNECTION
27. Scaling: Improve MySQLBinlogEventType.valueOf to support unknown event type
28. Kernel: Support case when for federation

Bug Fix

1. Scaling: Fix barrier node created at job deletion
2. Scaling: Fix part of columns value might be ignored in DATA_MATCH consistency check
3. Scaling: Fix jdbc url parameters are not updated in consistency check
4. Scaling: Fix tables sharding algorithm type INLINE is case-sensitive
5. Scaling: Fix incremental task on MySQL require mysql system database permission
6. Proxy: Fix the NPE when executing select SQL without storage node
7. Proxy: Support DATABASE_PERMITTED permission verification in unicast scenarios
8. Kernel: Fix the wrong value of worker-id in show compute nodes
9. Kernel: Fix route error when the number of readable data sources and weight configurations of the Weight algorithm are not equal
10. Kernel: Fix multiple groups of readwrite-splitting refer to the same load balancer name, and the load balancer fails problem
11. Kernel: Fix can not disable and enable compute node problem
12. JDBC: Fix data source is closed in ShardingSphereDriver cluster mode when startup problem
13. Kernel: Fix wrong rewrite result when part of logical table name of the binding table is consistent with the actual table name, and some are inconsistent
14. Kernel: Fix startup exception when use SpringBoot without configuring rules
15. Encrypt: Fix null pointer exception when Encrypt value is null
16. Kernel: Fix oracle parsing does not support varchar2 specified type
17. Kernel: Fix serial flag judgment error within the transaction
18. Kernel: Fix cursor fetch error caused by wasNull change
19. Kernel: Fix alter transaction rule error when refresh metadata
20. Encrypt: Fix EncryptRule cast to TransparentRule exception that occurs when the call procedure statement is executed in the Encrypt scenario
21. Encrypt: Fix exception which caused by ExpressionProjection in shorthand projection
22. Proxy: Fix PostgreSQL Proxy int2 negative value decoding incorrect
23. Proxy: PostgreSQL/openGauss support describe insert returning clause
24. Proxy: Fix gsql 3.0 may be stuck when connecting Proxy
25. Proxy: Fix parameters are missed when checking SQL in Proxy backend
26. Proxy: Enable MySQL Proxy to encode large packets
27. Kernel: Fix oracle parse comment without whitespace error
28. DistSQL: Fix show create table for encrypt table

Refactor

1. Scaling: Reverse table name and column name when generating SQL if it's SQL keyword
2. Scaling: Improve increamental task failure handling
3. Kernel: Governance center node adjustment, unified hump to underscore

Change Log

1. MILESTONE

5.2.1

New Feature

1. Add ShardingSphere default system database to support global metadata management
2. Support asynchronous data consistency check
3. Added support for Consul governance center
4. Added support for Nacos governance center
5. Added support for the view function in the governance center

Enhancement

1. SQL Federation engine adds ADVANCED executor and adapts to openGauss database
2. Support ShardingSphere Proxy startup after read-write splitting read database is disabled
3. SQL HINT supports force sharding route
4. Show processlist supports showing Proxy connections (sleep, active)
5. Optimized ShardingSphere-JDBC data source configuration error message
6. ShardingSphere-JDBC supports SpringBoot 3.x version
7. Support load MySQL, PostgreSQL, openGauss and SQLServer view metadata
8. Update Snakeyaml to 1.33 and open YAML 3MB limit
9. Reuse cached connections as possible when unicast sharding
10. Support Parsing ALTER ROLE in Oracle
11. Add support of ALTER RESOURCE COST for Oracle
12. Support parsing Drop Materialized View in Oracle
13. Support parsing DROP LIBRARY in Oracle
14. Support parsing DROP JAVA in Oracle
15. Support parsing DROP PLUGGABLE DATABASE in Oracle
16. Support parsing DROP INDEX TYPE in Oracle
17. Support Parsing ALTER PACKAGE in openGauss
18. Support openGauss select offset, count statement parse and remove useless syntax in PostgreSQL grammar
19. Add max_size to openGauss syntax
20. Optimize alter view/drop view parse logic and fix alter view refresher bug
21. Add sql parser error detail to ParseCancellationException
22. Add support for parse OptOnConflict for postgres
23. Enhance support for ALTER TABLE and ALTER VIEW in PostgreSQL
24. Add a missing keyword in PostgreSQL Declare Statement
25. Add json function support to mysql parser
26. ShardingSphere-Proxy automatically adapts to cgroup memory limits in Docker environment
27. Show migration status DistSQL respond new added error_message column
28. Show migration status respond new added processed_records_count column
29. Support MySQL 8 caching_sha2_password authentication in incremental dump
30. Improve drop pipeline process configuration
31. Support unique key table migration
32. Support migrate table to new name table
33. Improve thread pool usage in pipeline task and runner
34. Support cancelable data consistency check
35. DistSQL: When creating or altering readwrite-splitting rule, check duplicate write or read resources
36. DistSQL: Add validity check for ALTER SHARDING BINDING TABLE RULES
37. Standalone mode H2 support persistent metadata
38. Fix openGauss cursor execution in xa transaction
39. Added transaction related exceptions

Bug Fix

1. Generate proper placeholder for PostgreSQL when rewriting
2. Fix opengauss update set parse exception
3. Fix parse exception when execute insert statement with negative value
4. Fix wrong connectDescriptorUrlPattern in OracleDataSourceMetaData
5. Fix insert SQL garbled after sharding rewrote in special rules
6. Fix exception when execute select * from information_schema.tables
7. Fix exception when execute alter view rename
8. Fix PostgreSQL check data source permission when using rolsuper
9. DistSQL: fix NPE for REFRESH TABLE METADATA when there is no resource
10. Fix Unmodified table metadata when modify rules
11. Fix database discovery
12. The MySQL.NORMAL_REPLICATION algorithm cannot find primary node"
13. Fixed using etcd to build a cluster event not aware
14. Fix NPE occurred when transaction management is not created
15. Fix routing exception that occurs when the sharding value of the InlineShardingAlgorithm algorithm exceeds Integer

API Changes

1. SQL HINT syntax format adjust to SQL-style format
2. DistSQL: Remove syntax COUNT DATABASE RULES
3. ShardingSphere mode remove overwrite configuration
4. Agent: Optimize configuration of agent.yaml

Change Log

1. MILESTONE

5.2.0

New Feature

1. Support SQL audit for sharding feature
2. Support MySQL show processlist and kill process list id feature
3. Scaling: Add dedicated DistSQL for data migration
4. Scaling: Basic support migrate data to heterogeneous database
5. DistSQL: New syntax CREATE MIGRATION PROCESS CONFIGURATION
6. DistSQL: New syntax ALTER MIGRATION PROCESS CONFIGURATION
7. DistSQL: New syntax SHOW MIGRATION PROCESS CONFIGURATION
8. DistSQL: New syntax ADD MIGRATION SOURCE RESOURCE
9. DistSQL: New syntax SHOW SQL_TRANSLATOR RULE
10. DistSQL: New syntax CREATE SHARDING AUDITOR
11. DistSQL: New syntax ALTER SHARDING AUDITOR
12. DistSQL: New syntax SHOW SHARDING AUDIT ALGORITHMS

Enhancement

1. Support column visible feature for MySQL, Oracle, SQLServer and H2
2. Support cartesian product configuration for read write splitting
3. Support spring namespace and spring boot usage for sql translator
4. Support JSR-310 Year and Month in IntervalShardingAlgorithm
5. Support broadcast table update/delete limit statement
6. Support create index on table(column) statement rewrite when config encrypt
7. Support openGauss cursor, fetch, move, close statement for sharding, readwrite-splitting
8. Support encrypt column rewrite when execute column is null in predicate
9. Support encrypt show create table return logic columns
10. Support create table with index statement rewrite when config encrypt
11. Support PostgreSQL create operator statement parse
12. Support PostgreSQL create materialized view statement parse
13. Support PostgreSQL nested comments parse
14. Support PostgreSQL alter subscription statement parse
15. Support PostgreSQL create group statement parse
16. Support PostgreSQL alter statictics statement parse
17. Support PostgreSQL create foreign table statement parse
18. Support PostgreSQL alter server statement parse
19. Support PostgreSQL create foreign data wrapper statement parse
20. Support PostgreSQL create event trigger statement parse
21. Support PostgreSQL security label statement parse
22. Support PostgreSQL reindex statement parse
23. Support PostgreSQL reassign owned statement and refresh materialized view statement parse
24. Support PostgreSQL prepare transaction statement parse
25. Support PostgreSQL create collation statement parse
26. Support PostgreSQL lock statement parse
27. Support PostgreSQL alter rule statement parse
28. Support PostgreSQL notify statement parse
29. Support PostgreSQL unlisten statement parse
30. Support Oracle alter function and alter hierarchy statement parse
31. Support Oracle alter pluggable database statement parse
32. Support Oracle alter materialized view log statement parse
33. Support Oracle alter diskgroup statement parse
34. Support Oracle alter operator statement parse
35. Support oracle alter cluster statement parse
36. Support oracle alter audit policy statement parse
37. Support Oracle alter index type statement parse
38. Support Oracle lock table statement parse
39. Support Oracle alter java statement parse
40. Support Oracle inline constraint statement parse
41. Support openGauss geometric operator statement parse
42. Optimize MySQL visible/invisible parse of create/alter table statements
43. Support scope of variable prefixed with @@ in MySQL SET statement parse
44. Support MySQL create procedure with create view parse
45. Support column segments parse in create index on table statement
46. Support openGauss cursor, fetch, move, close statement for sharding, readwrite-splitting
47. Support encrypt column rewrite when execute column is null in predicate
48. Support encrypt show create table return logic columns
49. Support create table with index statement rewrite when config encrypt
50. Support parsing ALTER LOCKDOWN PROFILE in Oracle
51. Support parsing ALTER MATERIALIZED VIEW in Oracle
52. Support parsing ALTER MATERIALIZED ZONEMAP in Oracle
53. Support parsing ALTER LIBRARY in Oracle
54. Support parsing ALTER INMEMORY JOIN GROUP in Oracle
55. Support parsing DROP OPERATOR in Oracle
56. Support parsing DROP RESTORE POINT in Oracle
57. Support parsing CREATE RESTORE POINT in Oracle
58. Support parsing DROP INMEMORY JOIN GROUP in Oracle
59. Support parsing create_bit_xor_table in MySQL
60. Support parsing MySQL DO statement
61. Support parsing DropServer in openGauss
62. Support parsing CREATE AGGREGATE In openGauss
63. Support parsing ALTER ROUTINE in PostgreSQL
64. Add PostgreSQL Create Cast Statement
65. Add PostgreSQL Create Aggregate Statement
66. Support fetch/move/close cursor statement in PostgreSQL
67. Support Parsing ALTER PUBLICATION in PostgreSQL
68. Add PostgreSQL Create Access Method Statement
69. Support Parsing ALTER POLICY in PostgreSQL
70. Support parsing ALTER OPERATOR in PostgreSQL
71. Add PostgreSQL Copy Statement
72. Add PostgreSQL Comment Statement
73. Support listen statement in postgreSQL
74. Support DECLARE cursor statement
75. Add default serverConfig in helm charts
76. Assemble openGauss JDBC Driver into Proxy distribution
77. ShardingSphere-Proxy listen on specified IP addresses
78. Support COM_STMT_SEND_LONG_DATA in MySQL Proxy
79. SELECT VERSION() support alias in MySQL Proxy
80. Fix openGauss Proxy could not be connected if no resource defined
81. Support using JRE defined in JAVA_HOME in ShardingSphere-Proxy's startup script
82. Avoid client blocked when OOM occurred in ShardingSphere-Proxy
83. Support using c3p0 in ShardingSphere-JDBC
84. Support SET NAMES with value quoted by double-quote
85. Connection.prepareStatement with columns arguments is available in ShardingSphere-JDBC
86. Scaling: Improve MySQL connect and reconnect
87. Scaling: Fix MySQL json column may cause leak at incremental task
88. Scaling: Add permission check for PostgreSQL data sources
89. Scaling: Incremental migration support for MySQL MGR mode
90. Scaling: Improve job progress persistence
91. Scaling: Start job DistSQL execute and return synchronously
92. Scaling: Inventory migration support table has primary key and unique key
93. Scaling: Close unerlying ElasticJob when stopping job
94. Scaling: Improve logical replication slot name generation for PostgreSQL and openGauss
95. Scaling: Make query DistSQL could be executed when no database selected
96. DistSQL: Add worker_id to the result set of SHOW INSTANCE LIST & SHOW INSTANCE INFO
97. DistSQL: Improve the result of EXPORT DATABASE CONFIG
98. DistSQL: Support more databases for FORMAT SQL
99. DistSQL: Optimize the execution logic of CREATE TRAFFIC RULE
100. DistSQL: Add paramter writeDataSourceQueryEnabled for RDL READWRITE_SPLITTING RULE.
101. DistSQL: Support assistEncryptor for Encrypt RDL
102. DistSQL: Add sharding algorithm type check when CREATE SHARDING TABLE RULE
103. Support database discovery to configure multiple groups of high availability under the same logic database
104. Support ShardingSphere-Proxy to start up under empty logic library
105. Support for isolating EventBus events by instance
106. Support the database to detect changes in the master node and restart the detection heartbeat task
107. Support ShardingSphere-Proxy to generate new worker-id when re-registering in cluster mode
108. Thrown exception when inserting expression value in shadow column on executing insert
109. Support distributed transactions across multiple logical databases
110. Support executing truncate in XA & PostgreSQL
111. Support alter local transaction rule with DistSQL
112. Support global transaction manager
113. Delete support for branch transaction on proxy

Bug Fix

1. Fix single table metadata refresh error caused by filtering DataSourceContainedRule
2. Fix parsing exception caused by the null value of MySQL blob type
3. Fix PostgreSQL/openGauss reset statement parse error
4. Fix wrong parameter rewrite when use sharding and encrypt
5. Fix the failed conversion of Month related classes on IntervalShardingAlgorithm
6. Fix NullPointerException when execute select union statement contains subquery
7. Fix wrong encrypt rewrite result due to incorrect order of metadata
8. Fix MySQL trim function parse error
9. Fix MySQL insert values with _binary parse error
10. Fix MySQL syntax error cannot be thrown to client
11. Avoid EventLoop blocked because of closing JDBC resources
12. Correct server status flags returned by MySQL Proxy
13. Fix a possible connection leak issue if Proxy client disconnected in transaction
14. Fixed a possible consistency issue with the statement being executed when the Proxy client is disconnected
15. Avoid pooled connection polluted by executing SET statements
16. Make SHOW TABLES FROM work in ShardingSphere-Proxy
17. Fix PostgreSQL DDL could not be executed by Extended Query
18. Fix SHOW VARIABLES could not be executed in PostgreSQL Proxy without resource
19. Fix FileNotFoundException when use ShardingSphere Driver with SpringBoot fatjar
20. Scaling: Fix the problem that the table contains both primary key and unique index at inventory migration
21. Scaling: Improve incremental migration, support the latest position in the middle of batch insert event
22. Scaling: Fix the error caused by null field value in openGauss incremental migration
23. DistSQL: Fix incorrect strategy name in result of SHOW SHARDING TABLE RULES
24. DistSQL: Fix current rule config is modified in advance when ALTER SHARDING TABLE RULE
25. DistSQL: Fix connection leak when ALTER RESOURCE
26. DistSQL: Fix CREATE TRAFFIC RULE failed when load balance algorithm is null
27. Fix that the monitoring heartbeat task was not stopped when the database was discovered and the logical library was deleted
28. Fix cluster mode ShardingSphere-JDBC load all logic database
29. Fix worker-id generated by SnowflakeKeyGenerateAlgorithm in cluster mode may exceed the maximum value
30. Fix DistSQL adding shadow algorithm exception without shadow data source
31. Fix cross-database data source confusion caused by same data source name in multiple logical databases
    1...