v1.14.7

What's Changed

• feat: support wait sink node close by @Yisaer in #3373
• chore: bump go_modules/golang.org/x/crypto-0.31.0 by @OlgasAcc in #3447

New Contributors

• @OlgasAcc made their first contribution in #3447

Full Changelog: v1.14.6...v1.14.7

v2.0.2

What's Changed

• fix: replace props when get (#3409)
• fix: async import data return response (#3408)
• refactor(neuron): adapt to multi tags
• fix(op): merger use merge time as et
• fix(op): send out batch when EOF
• fix(op): avoid rate limit panic

Full Changelog: v2.0.1...v2.0.2

v2.1.0-alpha.1

Features

• feat(enc): support aes gcm by @ngjaying in #3344
• feat(rule): support send out nil fields by @ngjaying in #3342
• Incremental window
  • feat: support inc agg window by @Yisaer in #3322
  • feat: support incremental window in having by @Yisaer in #3358
  • feat: support tumbling/sliding inc window by @Yisaer in #3361
  • feat: support inc window filter by @Yisaer in #3379
  • feat: support incremental hopping window by @Yisaer in #3386
• feat: support xml converter by @Yisaer in #3341
• feat(func): add reverse function by @Dream95 in #3357
• feat: support kafka sink compression by @Yisaer in #3374
• feat: support auth request headers by @Yisaer in #3376
• feat: validate confkey by @Yisaer in #3378
• feat: rewrite duration in conf by @Yisaer in #3387
• Support MQTT 5
  • feat(io): support mqtt5 by @ngjaying in #3384
  • feat(mqtt): v5 sink supports properties by @ngjaying in #3390
  • feat(trace): support mqtt 5 trace by @ngjaying in #3394
• feat: support merge tokens into body by @Yisaer in #3392
• feat: support fileDir notify source by @Yisaer in #3380
• feat: support sql lookup template sql by @Yisaer in #3398

What's Changed

• docs(rule): doc of triggered prop by @ngjaying in #3338
• chore: sendError defaults to false by @ngjaying in #3339
• ci: fix doc version by @ngjaying in #3340
• fix(func): object_construct always issue keys by @ngjaying in #3345
• fix: remove tryrun in ruletest by @Yisaer in #3348
• chore(test): change video by @ngjaying in #3354
• build: fix make lint error by @Dream95 in #3363
• fix: sql source support null value by @Yisaer in #3365
• fix(conn): conn status nil exception by @ngjaying in #3372
• fix(planner): clean subtopo when creation fail by @Yisaer in #3383
• fix(kafka): fix dynamic prop problem by @ngjaying in #3391
• fix(neuron): avoid flooding logs when disconnect by @ngjaying in #3393
• docs: add kafka sink doc by @Yisaer in #3396

Full Changelog: v2.0.0...v2.1.0-alpha.1

v2.0.1

What's Changed

• docs(rule): doc of triggered prop by @ngjaying in #3338
• chore: sendError defaults to false by @ngjaying in #3339
• ci: fix doc version by @ngjaying in #3340
• fix(func): object_construct always issue keys by @ngjaying in #3345
• fix: remove tryrun in ruletest by @Yisaer in #3348
• chore(test): change video by @ngjaying in #3354
• build: fix make lint error by @Dream95 in #3363
• fix: sql source support null value by @Yisaer in #3365
• fix(conn): conn status nil exception by @ngjaying in #3372
• feat: support kafka sink compression by @Yisaer in #3374
• feat: support auth request headers by @Yisaer in #3376
• feat: validate confkey by @Yisaer in #3378
• fix(planner): clean subtopo when creation fail by @Yisaer in #3383
• feat: rewrite duration in conf by @Yisaer in #3387
• fix(kafka): fix dynamic prop problem by @ngjaying in #3391
• feat: support merge tokens into body by @Yisaer in #3392
• fix(neuron): avoid flooding logs when disconnect by @ngjaying in #3393

Full Changelog: v2.0.0...v2.0.1

v1.14.6

What's Changed

• fix: fix sql converter by @Yisaer in #3368
• fix: larger scanner buffer by @Yisaer in #3369

Full Changelog: v1.14.5...v1.14.6

v2.0.0

IO(Source/Sink)

• Connection management https://ekuiper.org/docs/en/latest/guide/connections/overview.html

  • Connection pool framework (gradually adapt existing connections)
    • Connection become a physical instance instead of a rule logical configuration, it will run from the start and refered by multiple rules
    • Standalone connection status
    • Auto reconnect
  • Adapt mqtt, sql, nng(neuron), edgex, websocket, httppush
  • APIs to manage connection instances
  • Connection status metrics

• Source/sink operators split source split/sink split

  • Finer-grained runtime metrics can be supported to understand the status and latency of each sub-task.
  • Parallel computation of sub-tasks can be implemented, improving the overall efficiency of rule execution.

• Source down sampling support doc

• Formats

  • Delimited format support header
  • Add urlencoded format (usually use in HTTP form request, extract from REST sink)

• Multi-level decode (format and payloadFormat)

• File sink supports enctryption (AES now)

• REST sink supports full format like delimited(csv), also add validation for mapping of format and bodyType

SQL/Streaming runtime

• General ONNX function plugin, run ONNX model inference without coding doc
• Support stream EOF so that we can run a rule in batch mode
  • File/simulator source stops after read all and no loop/interval is set
  • MQTT stream allows to stop by sending EOF message
• Rule gracefully exit: wait until all operators close
• New function: object_size
• Support window function doc
• Datetime format, support multiple S in format layout

Operation

• Tracing support doc
  • REST API tracing
  • Query rule trace API
  • Tracing data from/to Neuron
  • Local trace
  • Tracing collector
• Rule state cleanup
  • View rule metrics after close
  • Avoid malform state
• New rule metrics: all metrics now
  • connection related metrics for source/sink
• CPU usage metric
• API updates
  • Import
    • Asnc API
    • Supports YAML content
  • Meta list return plugin type info
  • List streams API show format and type
  • Drop dependency validation
    • stream
    • plugin
• Store conf in KV by default
• Conf: use time.Duration for intervals

Extension

• Native plugin develop
  • Update extention spec, users need to rewrite plugins
  • Do not need to depend on eKuiper project any more
• Portable plugin
  • Support sink ack so that it can leverage sink cache doc
  • Start as long running instance
  • Plugin status API, reference count
  • Remove recv size limit (won’t fail when communication payload is bigger than 1MB)
• Support REST external service retry

Test & Build

• Upgrade to go 1.23
• Build: add docker alpine-python image
• UT coverage 71%
• Security
  • Fix integer truncate problem
  • SQL prepare to avoid injection
  • Escape html response

What's Changed

• chore: remove repetitive words by @InventiveCoder in #2718
• build: upgrade base image go version by @ngjaying in #2720
• feat: introduce rule metrics by @Yisaer in #2716
• build: upgrade go and dependant mods by @ngjaying in #2719
• feat: refact sql indexValue into muti indexValue by @Yisaer in #2717
• refactor(xsql): unify row and collection by @ngjaying in #2700
• fix: fix doc error by @Yisaer in #2724
• feat(module): modularize io by @ngjaying in #2721
• feat: add multi field index api by @Yisaer in #2729
• feat(module): modularize converter by @ngjaying in #2728
• docs: fix some errors by @L-607 in #2731
• feat: let IntegerLiteral Value as int64 by @Yisaer in #2727
• fix: fix describe rule exposes password by @Yisaer in #2732
• docs: add doc for multiple sql index field by @Yisaer in #2733
• docs: fix websocket doc path by @Yisaer in #2735
• feat(modularize): modularize function by @ngjaying in #2737
• chore(ci): fix tag list error by @Swilder-M in #2738
• fix: replace password when create rule by @Yisaer in #2736
• feat: support nested array schema by @Yisaer in #2734
• fix(planner): avoid create subtopo in error by @ngjaying in #2740
• fix: check rule exists before create by @Yisaer in #2741
• fix: redisPubSub ping by @retoool in #2739
• fix: doc problem for tdengine sink by @carlclone in #2743
• feat: add security to edgex via openziti by @dovholuknf in #2603
• refactor: split sink batch by @ngjaying in #2745
• fix: fix load stream when meet error by @Yisaer in #2751
• ci: delete v prefix in git tag by @Rory-Z in #2752
• fix: ignore nil value in array convert by @carlclone in #2748
• fix: fix template sql index value update strategy by @Yisaer in #2754
• feat: support modify sqlSource field index value by @Yisaer in #2730
• feat: support rule stop by error metrics by @Yisaer in #2755
• feat: package v2 by @ngjaying in #2766
• refactor: move api to standalone go mod by @ngjaying in #2770
• chore(temp): remove extensions by @ngjaying in #2777
• feat(v2): mvp features by @ngjaying in #2780
• feat: new source/sink v2 API by @ngjaying in #2783
• refactor(contract): keep interfaces only by @ngjaying in #2790
• chore: refer to correct contract mod by @ngjaying in #2792
• fix(test): source/sink test update by @ngjaying in #2797
• feat(api): add error ingest for source subscribe by @ngjaying in #2800
• refactor(sink): split compression op by @ngjaying in #2802
• feat(sink): split encryptor by @ngjaying in #2807
• feat(op): add dedup trigger op by @ngjaying in #2801
• refactor(converter): add context to converter API by @ngjaying in #2812
• feat(sink): add back file sink by @ngjaying in #2811
• feat: separate schema layer from decoder by @Yisaer in #2815
• refactor(sink): mqtt sink use connection pool by @ngjaying in #2814
• feat: remove useless flag in conveter by @Yisaer in #2823
• feat(sink): support file sink rolling hook by @ngjaying in #2819
• refactor: provide schema when instantiating by @ngjaying in #2821
• chore: update contract version by @ngjaying in #2826
• feat(function): object pick enhancement (#232) by @ngjaying in #2833
• refactor(model): extract raw tuple by @ngjaying in #2835
• feat(sink): add sink dynamic properties support by @ngjaying in #2837
• refactor(op): decode op schema more logs by @ngjaying in #2840
• chore: update codecov action by @ngjaying in #2843
• fix: detach for non shared connection by @ngjaying in #2845
• refactor(sink): split cache op by @ngjaying in #2852
• fix: revise interval config by @Yisaer in #2839
• feat(cache): resend alter queue support by @ngjaying in #2855
• fix: let config compatible by @Yisaer in #2861
• refactor(node): use time.Duration and time.Time by @ngjaying in #2864
• fix: support wait operators close by @Yisaer in #2862
• feat(io): add file source in lines mode by @ngjaying ...

v2.0.0-beta.4

What's Changed

• docs: add doc for yaml import/export by @Yisaer in #3278
• fix(docs): remove duplicate entry by @ngjaying in #3277
• feat: support trace strategy by @Yisaer in #3284
• feat: support trace projectset by @Yisaer in #3286
• chore: fix sql query by @ngjaying in #3282
• docs(cookbook): add over clause examples by @ngjaying in #3280
• fix: batch/window op support dynamic trace by @Yisaer in #3287
• test: add onnx ut by @ngjaying in #3281
• refactor(node): extract process start/end hook by @ngjaying in #3285
• refactor(trace): move trace into common func by @ngjaying in #3288
• feat: support cpu profile api by @Yisaer in #3279
• feat(trace): support trace source by @ngjaying in #3289
• refactor(trace): unify trace context init by @ngjaying in #3290
• feat(trace): cache op and tests by @ngjaying in #3292
• feat: write panic stack in log by @Yisaer in #3295
• feat: revise plan explain info by @Yisaer in #3294
• feat(trace): support more nodes by @ngjaying in #3296
• fix: replace props when import data by @Yisaer in #3297
• feat: support use bypass to rewrite sql by @Yisaer in #3299
• fix(trace): local trace wrong content by @ngjaying in #3300
• fix(mqtt): should set client id by @ngjaying in #3303
• fix(state): clean topo when stop by EOF by @ngjaying in #3304
• fix(portable): remove nng recv size limit by @Yisaer in #3306
• fix: fix import connection configuration by @Yisaer in #3308
• fix(conn): allow to exit wait by @ngjaying in #3307
• feat(portable): plugin error message for func by @ngjaying in #3305

Full Changelog: v2.0.0-beta.3...v2.0.0-beta.4

v1.14.5

What's Changed

• fix: sql source scan value by database type by @Yisaer in #3228
• fix: warning when config not in connection.yaml by @Yisaer in #3229
• chore: update base image go ver by @ngjaying in #3312
• chore: update go and docker image by @ngjaying in #3311
• fix(build): update docker build ver by @ngjaying in #3313

Full Changelog: v1.14.4...v1.14.5

v2.0.0-beta.3

Features

• build: Dockerfile-alpine-python and Makefile build by @vkaramanis in #3271
• feat(format): support urlencoded format by @ngjaying in #3276

What's Changed

• chore: downgrade go-ora by @ngjaying in #3261
• chore: enable x509negativeserial by @ngjaying in #3265
• fix: load connection props for stream by @Yisaer in #3266
• fix: replace props when update rule by @Yisaer in #3268
• fix(io): clean up io ping by @ngjaying in #3267
• fix: fix websocket wg by @Yisaer in #3272
• fix: fix conneciton drop by @Yisaer in #3264
• refactor: revise project operator field name by @Yisaer in #3274
• fix(conn): ping fixes by @ngjaying in #3273
• refactor(rest): convert to byte collector by @ngjaying in #3275

New Contributors

• @vkaramanis made their first contribution in #3271

Full Changelog: v2.0.0-beta.2...v2.0.0-beta.3

v2.0.0-beta.2

Features

• feat: General pytorch function by @578223592 in #3062
• Tracing updates
  • feat: change trace api by @Yisaer in #3206
  • feat: support trace rest api by @Yisaer in #3167
• feat(tool): restore plugin test server by @ngjaying in #3211
• feat: support update connection by @Yisaer in #3236
• feat(prom): support connection status by @ngjaying in #3250

What's Changed

• test: add test for sql/kafka by @Yisaer in #3203
• fix: fix sql source test by @Yisaer in #3208
• fix: fix sql sink test by @Yisaer in #3209
• ci: cache jmeter package by @Rory-Z in #3212
• docs: add data trace example doc by @Yisaer in #3204
• docs(io): source/sink split by @ngjaying in #3207
• refactor(zmq): add timeout for receive by @ngjaying in #3210
• fix: fix trace rest api propagation by @Yisaer in #3214
• chore(io): clean up io binder and logs by @ngjaying in #3215
• fix: fix source ctx tracer propagate by @Yisaer in #3217
• fix: fix trace propagator by source by @Yisaer in #3218
• fix: sql source scan value by database type by @Yisaer in #3219
• fix: fix connection pool get unexisted named conn by @Yisaer in #3222
• fix: restore sink ping by @Yisaer in #3220
• fix: warning when config not in connection.yaml by @Yisaer in #3221
• refactor(conn): save connection status in meta by @ngjaying in #3225
• fix: restore driver by @Yisaer in #3226
• fix: cache raw config not to load duplicated by @Yisaer in #3227
• fix(simulator): interval config problem by @ngjaying in #3230
• fix(conn): connection status for shared stream by @ngjaying in #3231
• fix: fix load cfgkey from storage panic by @Yisaer in #3232
• fix: topo no changed before stop during update by @Yisaer in #3233
• fix: support kafka password param by @Yisaer in #3234
• fix(metric): escape metric messages by @ngjaying in #3235
• fix: fix sort log file by @Yisaer in #3239
• fix(portable): install error cleanup by @ngjaying in #3237
• fix(file): rolling interval not obeyed problem by @ngjaying in #3238
• chore: update sqlite driver mod by @ngjaying in #3240
• fix(rest): error message escape by @ngjaying in #3241
• feat: update connection api and doc by @Yisaer in #3242
• fix: fix driver import by @Yisaer in #3253
• fix(conn): trigger status handler when init by @ngjaying in #3252
• fix: sql prepare and html response escape by @Yisaer in #3245
• fix: validate filepath in request by @Yisaer in #3248
• fix(trial): do not close control ws by @ngjaying in #3254
• fix: replace ruleJson's password/url by @Yisaer in #3251
• fix: fix omit option by @Yisaer in #3256
• fix(onnx): type check problems by @ngjaying in #3257
• fix(transform): fix trans datafield for list by @ngjaying in #3244
• fix: fix cast security issue by @Yisaer in #3243
• ci: CI pipeline for onnx by @578223592 in #3258
• docs: fix resource path by @ngjaying in #3259
• fix: fix tracer function by @Yisaer in #3260

Full Changelog: v2.0.0-beta.1...v2.0.0-beta.2