Skip to content

v1.4.1

Latest
Latest
Compare
Choose a tag to compare
@flarco flarco released this 01 Feb 21:30
v1.4.1
4914669
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Sling v1.4.1 (2025-02-01T21:46:18Z)

Major Changes

  1. Pipeline Implementation

    • Added new Pipeline type for executing sequential steps
    • Introduced pipeline configuration loading from YAML/JSON files
    • Added support for pipeline steps with various types (log, replication, command, etc.)

  2. State Management Refactoring

    • Renamed Hooks to State in runtime state management
    • Created RuntimeState interface
    • Added separate ReplicationState and PipelineState implementations

  3. Chunking Functionality

    • Added support for chunking data processing based on column ranges
    • Implemented ProcessChunks() method for handling data partitioning
    • Added chunk size configuration options

  4. Case Handling Improvements

    • Added support for lower/upper case schema and table names
    • Improved temp table name handling for different database types
    • Added new format variables: stream_schema_lower/upper, stream_table_lower/upper

  5. File System Updates

    • Enhanced file pattern matching across different file system implementations
    • Improved file deletion handling for SFTP
    • Better handling of file paths in copy operations

Minor Changes

  • Added binary data handling for DuckDB
  • Improved UTF-8 validation in string casting
  • Added environment variable controls for DuckDB compute
  • Various bug fixes and code cleanup
Assets 12
Loading