[lambda]array_map(append_trailing_char_if_absent) function error in external table

[lvchenyang-maker]

Steps to reproduce the behavior (Required)

1. CREATE TABLE '...'

create external catalog hive_catalog_2e3394f0_59b2_11ed_91ff_00163e0e550b PROPERTIES("type" = "hive","hive.metastore.uris" = "thrift://ip:port");

2. SELECT '....'

select col_int,array_map(x ->append_trailing_char_if_absent('', 'a'),col_array_string) from hive_catalog_d75112be_58f0_11ed_b73c_00163e0e550b.hive_extbl_test_arra

Expected behavior (Required)

['a']

Real behavior (Required)

['']

StarRocks version (Required)

• You can get the StarRocks version by executing SQL select current_version()

+----------------------+
| current_version()    |
+----------------------+
| MAIN-RELEASE c1b178c |
+----------------------+

[fzhedu]

Root cause:

SR cannot identify empty strings from hive, no matter with lambda function.

hive

hive> select * from s;
OK

NULL

abc
Time taken: 0.159 seconds, Fetched: 4 row(s)

hive> select count(*) from s where a = '';
OK
2
Time taken: 34.251 seconds, Fetched: 1 row(s)

SR

mysql> select count(*) from s;
+----------+
| count(*) |
+----------+
|        2 |
+----------+
1 row in set (0.12 sec)

mysql> select count(*) from s where a = '';
+----------+
| count(*) |
+----------+
|        0 |
+----------+
1 row in set (0.07 sec)

[zombee0]

the reason is that append_trailing_char_if_absent('', 'a') treat arg[0] as const column and add 'a' when the arg[0].length=0 while on the col is not constant, it check the length and skip the row that src_length=0

[fzhedu]

append_trailing_char_if_absent's behavior is different when the first arg is an empty string by from columns or not.

mysql> Create table t ( a int, name char(20), pv double) ENGINE=OLAP DUPLICATE KEY(a)  DISTRIBUTED BY HASH(a) BUCKETS 10 PROPERTIES("replication_num" = "1");
Query OK, 0 rows affected (0.01 sec)

mysql> insert into t values (1,'',3);
Query OK, 1 row affected (0.03 sec)
{'label':'insert_a4e0c3d6-670e-11ed-b25b-00163e1f2aee', 'status':'VISIBLE', 'txnId':'5640284'}

mysql> select append_trailing_char_if_absent(name,'a') from t;
+-------------------------------------------+
| append_trailing_char_if_absent(name, 'a') |
+-------------------------------------------+
|                                           |
+-------------------------------------------+
1 row in set (0.01 sec)

mysql> select append_trailing_char_if_absent(name,'a') from t;
+-------------------------------------------+
| append_trailing_char_if_absent(name, 'a') |
+-------------------------------------------+
|                                           |
+-------------------------------------------+
1 row in set (0.00 sec)

mysql> select append_trailing_char_if_absent('','a') from t;
+-----------------------------------------+
| append_trailing_char_if_absent('', 'a') |
+-----------------------------------------+
| a                                       |
+-----------------------------------------+
1 row in set (0.00 sec)